{"repo": "calliopeai/calliope-cli", "files": {"docs/providers.md": {"8466872d4af80245e3ff9db6675975ac31d0cad9": {"t": 1785674475.2266786, "n": 5296, "h": "e09fdd77d5ce8b26", "c": "# Providers\n\nCalliope supports 13 provider backends, plus a generic OpenAI-compatible\nendpoint for any other server that speaks the OpenAI chat-completions API.\nModels are discovered live from each provider — there are no hardcoded model\nlists. Use `/model` to browse what a provider offers.\n\n## Backends at a glance\n\n| Backend | Kind | Credential | Notes |\n|---------|------|-----------|-------|\n| `anthropic` | Native SDK | API key | Claude models. |\n| `google` | Native SDK | API key | Gemini models. |\n| `openai` | Native SDK | API key | GPT / o-series models. |\n| `bedrock` | AWS | region + profile, or gateway URL | AWS Bedrock, native or via gateway. |\n| `ollama` | Local | base URL | Local models; no API key. |\n| `litellm` | Proxy | base URL (+ optional key) | Unified proxy for many providers. |\n| `openrouter` | OpenAI-compatible | API key | Aggregator for many models. |\n| `together` | OpenAI-compatible | API key | Open-weight models. |\n| `groq` | OpenAI-compatible | API key | Fast inference. |\n| `fireworks` | OpenAI-compatible | API key | Open-weight models. |\n| `mistral` | OpenAI-compatible | API key | Mistral models. |\n| `ai21` | OpenAI-compatible | API key | Jamba models. |\n| `huggingface` | OpenAI-compatible | API key | Hosted inference. |\n| `openai-compat` | Generic | base URL (+ optional key) | Any OpenAI-compatible server. |\n\nSelect a provider in a session with `/provider <name>`, or set `defaultProvider`\nduring setup. `auto` selects the first configured provider (priority order:\nanthropic, openai, google, mistral, openrouter, together, groq, fireworks, ai21,\nhuggingface, bedrock, ollama, litellm).\n\n## How credentials resolve\n\nFor every provider, Calliope merges two sources:\n\n1. Stored config under `providers.<name>` (written by the setup wizard).\n2. Environment variables.\n\nEnvironment variables take precedence, except for the `openai-compat` base URL\nwhere the stored value wins. The full environment-variable table is in\n[Configuration → Environment-variable fallbacks](./configuration.md#environment-variable-fallbacks).\n\n## Native SDK backends\n\n`anthropic`, `google`, and `openai` use each vendor's own API. Provide an API key:\n\n```\nexport ANTHROPIC_API_KEY=sk-ant-...\nexport GOOGLE_API_KEY=...\nexport OPENAI_API_KEY=sk-...\n```\n\nOr store them via the setup wizard, which writes them to `providers.<name>.apiKey`.\n\n## Hosted OpenAI-compatible backends\n\n`openrouter`, `together`, `groq`, `fireworks`, `mistral`, `ai21`, and\n`huggingface` speak the OpenAI chat-completions API and each have a built-in\nbase URL, so you only supply an API key:\n\n| Provider | Base URL |\n|----------|----------|\n| openrouter | `https://openrouter.ai/api/v1` |\n| together | `https://api.together.xyz/v1` |\n| groq | `https://api.groq.com/openai/v1` |\n| fireworks | `https://api.fireworks.ai/inference/v1` |\n| mistral | `https://api.mistral.ai/v1` |\n| ai21 | `https://api.ai21.com/studio/v1` |\n| huggingface | `https://api-inference.huggingface.co/v1` |\n\n```\nexport GROQ_API_KEY=gsk_...\n```\n\n## Ollama and local models\n\n`ollama` runs models on your machine and needs no API key — its \"credential\" is\nthe base URL. The default is `http://localhost:11434`.\n\n```\n# install and pull a model\nollama pull llama3.3\n\n# optional: point Calliope at a non-default host\nexport OLLAMA_BASE_URL=http://localhost:11434\n```\n\n```\n/provider ollama\n/model llama3.3\n```\n\nLocal models run offline, cost nothing, and keep data on your machine.\n\n## LiteLLM proxy\n\n`litellm` targets a [LiteLLM](https://docs.litellm.ai/docs/proxy) proxy that\nfronts other providers. Point Calliope at the proxy URL (default\n`http://localhost:4000`) and, if the proxy requires it, an API key:\n\n```\nexport LITELLM_BASE_URL=http://localhost:4000\nexport LITELLM_API_KEY=...      # only if your proxy requires it\n```\n\n## Generic OpenAI-compatible servers\n\n`openai-compat` connects to any server implementing the OpenAI chat-completions\nAPI — for example LM Studio, vLLM, Jan, LocalAI, or AnythingLLM. Supply the base\nURL, plus a key if the server requires one:\n\n```\nexport OPENAI_COMPAT_BASE_URL=http://localhost:1234/v1\nexport OPENAI_COMPAT_API_KEY=...   # only if required\n```\n\nThe base URL you store in config is preferred over the environment variable for\nthis backend.\n\n## AWS Bedrock\n\n`bedrock` has two modes:\n\n- **Native** — calls the Bedrock Converse API directly. Provide an AWS region and\n  (optionally) a named profile, via config or the environment:\n\n  ```\n  export AWS_REGION=us-east-1        # or AWS_DEFAULT_REGION\n  export AWS_PROFILE=dev             # optional named profile\n  ```\n\n  Standard AWS credential resolution applies (`AWS_ACCESS_KEY_ID` /\n  `AWS_PROFILE` are detected automatically).\n\n- **Gateway** — if you set a Bedrock gateway/proxy base URL, Calliope treats it\n  as an OpenAI-compatible endpoint instead:\n\n  ```\n  export BEDROCK_BASE_URL=http://localhost:8080\n  export BEDROCK_API_KEY=...         # only if the gateway requires it\n  ```\n\nStore region and profile permanently under `providers.bedrock.region` and\n`providers.bedrock.profile`.\n\n## Default models\n\nEach provider has an offline emergency-fallback model used only when live\ndiscovery is unavailable (for example, with no network). In normal use the model\nis chosen by discovery or by your `/model` selection, not from a fixed list.\n"}}}}