1 line
22 KiB
JSON
1 line
22 KiB
JSON
{"repo": "blackcater/acme", "files": {"docs/anomalyco_opencode/core-application/ai-provider-and-model-management.md": {"23de4eb4f3e7da2b7cbd007f6783504f71acfc00": {"t": 1785596389.6798937, "n": 21281, "h": "4617e0b598aa99c6", "c": "# AI Provider & Model Management\n\n<details>\n<summary>Relevant source files</summary>\n\nThe following files were used as context for generating this wiki page:\n\n- [README.md](README.md)\n- [packages/opencode/script/schema.ts](packages/opencode/script/schema.ts)\n- [packages/opencode/src/auth/index.ts](packages/opencode/src/auth/index.ts)\n- [packages/opencode/src/auth/service.ts](packages/opencode/src/auth/service.ts)\n- [packages/opencode/src/cli/ui.ts](packages/opencode/src/cli/ui.ts)\n- [packages/opencode/src/config/config.ts](packages/opencode/src/config/config.ts)\n- [packages/opencode/src/env/index.ts](packages/opencode/src/env/index.ts)\n- [packages/opencode/src/provider/error.ts](packages/opencode/src/provider/error.ts)\n- [packages/opencode/src/provider/models.ts](packages/opencode/src/provider/models.ts)\n- [packages/opencode/src/provider/provider.ts](packages/opencode/src/provider/provider.ts)\n- [packages/opencode/src/provider/transform.ts](packages/opencode/src/provider/transform.ts)\n- [packages/opencode/src/server/server.ts](packages/opencode/src/server/server.ts)\n- [packages/opencode/src/session/compaction.ts](packages/opencode/src/session/compaction.ts)\n- [packages/opencode/src/session/index.ts](packages/opencode/src/session/index.ts)\n- [packages/opencode/src/session/llm.ts](packages/opencode/src/session/llm.ts)\n- [packages/opencode/src/session/message-v2.ts](packages/opencode/src/session/message-v2.ts)\n- [packages/opencode/src/session/message.ts](packages/opencode/src/session/message.ts)\n- [packages/opencode/src/session/prompt.ts](packages/opencode/src/session/prompt.ts)\n- [packages/opencode/src/session/revert.ts](packages/opencode/src/session/revert.ts)\n- [packages/opencode/src/session/summary.ts](packages/opencode/src/session/summary.ts)\n- [packages/opencode/src/tool/task.ts](packages/opencode/src/tool/task.ts)\n- [packages/opencode/test/config/config.test.ts](packages/opencode/test/config/config.test.ts)\n- [packages/opencode/test/provider/amazon-bedrock.test.ts](packages/opencode/test/provider/amazon-bedrock.test.ts)\n- [packages/opencode/test/provider/gitlab-duo.test.ts](packages/opencode/test/provider/gitlab-duo.test.ts)\n- [packages/opencode/test/provider/provider.test.ts](packages/opencode/test/provider/provider.test.ts)\n- [packages/opencode/test/provider/transform.test.ts](packages/opencode/test/provider/transform.test.ts)\n- [packages/opencode/test/session/llm.test.ts](packages/opencode/test/session/llm.test.ts)\n- [packages/opencode/test/session/message-v2.test.ts](packages/opencode/test/session/message-v2.test.ts)\n- [packages/opencode/test/session/revert-compact.test.ts](packages/opencode/test/session/revert-compact.test.ts)\n- [packages/sdk/js/src/gen/sdk.gen.ts](packages/sdk/js/src/gen/sdk.gen.ts)\n- [packages/sdk/js/src/gen/types.gen.ts](packages/sdk/js/src/gen/types.gen.ts)\n- [packages/sdk/js/src/v2/gen/sdk.gen.ts](packages/sdk/js/src/v2/gen/sdk.gen.ts)\n- [packages/sdk/js/src/v2/gen/types.gen.ts](packages/sdk/js/src/v2/gen/types.gen.ts)\n- [packages/sdk/openapi.json](packages/sdk/openapi.json)\n- [packages/web/src/components/Lander.astro](packages/web/src/components/Lander.astro)\n- [packages/web/src/content/docs/go.mdx](packages/web/src/content/docs/go.mdx)\n- [packages/web/src/content/docs/index.mdx](packages/web/src/content/docs/index.mdx)\n- [packages/web/src/content/docs/providers.mdx](packages/web/src/content/docs/providers.mdx)\n- [packages/web/src/content/docs/zen.mdx](packages/web/src/content/docs/zen.mdx)\n\n</details>\n\nThis document explains how OpenCode integrates with AI providers, discovers and manages model metadata, and routes requests through the provider abstraction layer. It covers the provider initialization system, model capabilities detection, message transformation, and the optional OpenCode Zen/Go services.\n\nFor information about how sessions use models and agents, see [Session & Agent System](#2.3). For configuration of providers, see [Configuration System](#2.2).\n\n---\n\n## Provider Architecture Overview\n\nOpenCode supports 75+ AI providers through a unified abstraction layer that normalizes differences between provider APIs while preserving provider-specific capabilities.\n\n**Provider System Architecture**\n\n```mermaid\ngraph TB\n Config[\"Config.get()\"]\n Auth[\"Auth System<br/>~/.local/share/opencode/auth.json\"]\n ModelsDev[\"Models.dev Metadata<br/>~/.cache/opencode/models.json\"]\n\n ProviderRegistry[\"Provider Registry<br/>Provider.all()\"]\n BundledProviders[\"Bundled Providers<br/>BUNDLED_PROVIDERS map\"]\n CustomLoaders[\"Custom Loaders<br/>CUSTOM_LOADERS map\"]\n\n ProviderInstance[\"Provider.Info<br/>{id, models, env, api}\"]\n ModelSelection[\"Provider.getModel()<br/>(providerID, modelID)\"]\n\n Transform[\"ProviderTransform<br/>message(), options()\"]\n SDKInstance[\"AI SDK Provider<br/>LanguageModelV2\"]\n\n Config --> ProviderRegistry\n Auth --> ProviderRegistry\n ModelsDev --> ProviderRegistry\n\n ProviderRegistry --> BundledProviders\n ProviderRegistry --> CustomLoaders\n\n BundledProviders --> ProviderInstance\n CustomLoaders --> ProviderInstance\n\n ProviderInstance --> ModelSelection\n ModelSelection --> Transform\n Transform --> SDKInstance\n```\n\nSources: [packages/opencode/src/provider/provider.ts:1-1000](), [packages/opencode/src/provider/models.ts:1-200]()\n\n---\n\n## Provider Registry and Initialization\n\nThe `Provider` namespace manages all provider integrations through a centralized registry system.\n\n### Bundled Providers\n\nOpenCode includes direct imports for common providers to avoid runtime installation overhead:\n\n| Provider Package | Factory Function | Provider ID |\n| ----------------------------- | -------------------------- | ---------------- |\n| `@ai-sdk/anthropic` | `createAnthropic` | `anthropic` |\n| `@ai-sdk/openai` | `createOpenAI` | `openai` |\n| `@ai-sdk/google` | `createGoogleGenerativeAI` | `google` |\n| `@ai-sdk/amazon-bedrock` | `createAmazonBedrock` | `amazon-bedrock` |\n| `@openrouter/ai-sdk-provider` | `createOpenRouter` | `openrouter` |\n| `@ai-sdk/azure` | `createAzure` | `azure` |\n| `@ai-sdk/xai` | `createXai` | `xai` |\n\nSources: [packages/opencode/src/provider/provider.ts:88-111]()\n\n### Provider Discovery Flow\n\n```mermaid\ngraph TB\n Start[\"Provider.all()\"]\n LoadModels[\"ModelsDev.all()<br/>fetch models.json\"]\n FilterEnabled[\"Filter by API key presence\"]\n\n CreateInfo[\"Build Provider.Info objects<br/>{id, models, env, api}\"]\n CustomLoader[\"Apply CUSTOM_LOADERS<br/>custom logic per provider\"]\n\n Autoload{\"autoload<br/>check\"}\n Include[\"Include in result\"]\n Skip[\"Skip provider\"]\n\n Start --> LoadModels\n LoadModels --> FilterEnabled\n FilterEnabled --> CreateInfo\n CreateInfo --> CustomLoader\n CustomLoader --> Autoload\n Autoload -->|true| Include\n Autoload -->|false| Skip\n```\n\nSources: [packages/opencode/src/provider/provider.ts:600-800]()\n\nThe `Provider.all()` function returns an array of enabled providers based on:\n\n1. Available models from Models.dev\n2. Presence of authentication credentials\n3. Custom loader `autoload` decisions\n\n---\n\n## Models.dev Integration\n\nOpenCode uses [Models.dev](https://models.dev) as the source of truth for model metadata, capabilities, and pricing.\n\n### Model Metadata Schema\n\n```typescript\n// Model.Info structure from Models.dev\n{\n id: string // e.g., \"anthropic/claude-3-5-sonnet\"\n name: string // Display name\n providerID: string // Provider identifier\n api: {\n id: string // API model ID\n url: string // Base URL\n npm: string // AI SDK package\n }\n capabilities: {\n temperature: boolean\n reasoning: boolean\n attachment: boolean\n toolcall: boolean\n input: {\n text: boolean\n audio: boolean\n image: boolean\n video: boolean\n pdf: boolean\n }\n interleaved: boolean | { field: string }\n }\n cost: {\n input: number // Per 1M tokens\n output: number\n cache: {\n read: number\n write: number\n }\n }\n limit: {\n context: number // Total context window\n input?: number // Max input tokens\n output: number // Max output tokens\n }\n}\n```\n\nSources: [packages/opencode/src/provider/models.ts:18-100]()\n\n### Model Cache Management\n\nModels.dev data is cached locally to avoid repeated fetches:\n\n**Cache Flow**\n\n```mermaid\ngraph LR\n Request[\"ModelsDev.all()\"]\n CacheCheck[\"Check cache<br/>~/.cache/opencode/models.json\"]\n\n Expired{\"Cache<br/>expired?\"}\n LoadCache[\"Load from cache\"]\n FetchRemote[\"Fetch from Models.dev<br/>https://models.dev/models.json\"]\n\n WriteCache[\"Write to cache<br/>1 hour TTL\"]\n Return[\"Return models\"]\n\n Request --> CacheCheck\n CacheCheck --> Expired\n Expired -->|No| LoadCache\n Expired -->|Yes| FetchRemote\n FetchRemote --> WriteCache\n LoadCache --> Return\n WriteCache --> Return\n```\n\nSources: [packages/opencode/src/provider/models.ts:80-150]()\n\nThe cache is invalidated after 1 hour or when `OPENCODE_CACHE_MODELS` flag is set to force refresh.\n\n---\n\n## Provider Transform Layer\n\nThe `ProviderTransform` namespace normalizes messages and options across different provider implementations.\n\n### Message Normalization\n\nDifferent providers have different requirements for message formatting:\n\n**Provider-Specific Transformations**\n\n| Provider | Transformation | Reason |\n| ----------------- | -------------------------------------- | ------------------------------------- |\n| Anthropic | Filter empty messages and parts | API rejects empty content |\n| Anthropic/Claude | Sanitize tool call IDs | Only alphanumeric and `_-` allowed |\n| Mistral | Normalize tool call IDs to 9 chars | Requires exactly 9 alphanumeric chars |\n| Mistral | Insert assistant message after tool | Cannot have user message after tool |\n| OpenAI-compatible | Extract reasoning to `providerOptions` | Supports `interleaved.field` |\n\nSources: [packages/opencode/src/provider/transform.ts:47-172]()\n\n### Caching Strategy\n\nOpenCode applies prompt caching to reduce costs for supported providers:\n\n```mermaid\ngraph TB\n Messages[\"Message Array\"]\n System[\"System Messages<br/>(first 2)\"]\n Recent[\"Recent Messages<br/>(last 2)\"]\n\n ApplyCache[\"Apply cacheControl\"]\n\n Anthropic[\"Anthropic:<br/>{cacheControl: {type: 'ephemeral'}}\"]\n Bedrock[\"Bedrock:<br/>{cachePoint: {type: 'default'}}\"]\n OpenAI[\"OpenAI Compatible:<br/>{cache_control: {type: 'ephemeral'}}\"]\n Copilot[\"Copilot:<br/>{copilot_cache_control: {type: 'ephemeral'}}\"]\n\n Messages --> System\n Messages --> Recent\n System --> ApplyCache\n Recent --> ApplyCache\n\n ApplyCache --> Anthropic\n ApplyCache --> Bedrock\n ApplyCache --> OpenAI\n ApplyCache --> Copilot\n```\n\nSources: [packages/opencode/src/provider/transform.ts:174-212]()\n\nCache control is applied to:\n\n- First 2 system messages (stable prompts)\n- Last 2 messages (recent context)\n\n### Model-Specific Options\n\nThe transform layer applies provider-specific defaults:\n\n**Temperature Defaults**\n\n```mermaid\ngraph LR\n Model[\"Model ID\"]\n\n Qwen[\"Qwen: 0.55\"]\n Claude[\"Claude: undefined\"]\n Gemini[\"Gemini: 1.0\"]\n GLM[\"GLM-4.x: 1.0\"]\n Kimi[\"Kimi K2 variants: 0.6-1.0\"]\n Default[\"Default: undefined\"]\n\n Model --> Qwen\n Model --> Claude\n Model --> Gemini\n Model --> GLM\n Model --> Kimi\n Model --> Default\n```\n\nSources: [packages/opencode/src/provider/transform.ts:292-317]()\n\n---\n\n## Model Selection and Routing\n\n### Provider SDK Instantiation\n\n```mermaid\ngraph TB\n GetModel[\"Provider.getModel()<br/>(providerID, modelID)\"]\n\n FindProvider[\"Find in Provider.all()\"]\n CheckBundle[\"Check BUNDLED_PROVIDERS\"]\n\n CreateSDK[\"Call factory function<br/>e.g., createAnthropic()\"]\n ApplyOptions[\"Apply provider options<br/>from config + custom loader\"]\n\n CustomGet{\"Custom<br/>getModel?\"}\n DefaultGet[\"sdk.languageModel(modelID)\"]\n CustomGet2[\"Custom getModel() logic\"]\n\n Return[\"Return LanguageModelV2\"]\n\n GetModel --> FindProvider\n FindProvider --> CheckBundle\n CheckBundle --> CreateSDK\n CreateSDK --> ApplyOptions\n ApplyOptions --> CustomGet\n CustomGet -->|No| DefaultGet\n CustomGet -->|Yes| CustomGet2\n DefaultGet --> Return\n CustomGet2 --> Return\n```\n\nSources: [packages/opencode/src/provider/provider.ts:850-950]()\n\n### Custom Model Loaders\n\nCertain providers require special handling for model instantiation:\n\n**OpenAI Responses API**\n\n```typescript\n// Custom loader for OpenAI\nopenai: async () => {\n return {\n autoload: false,\n async getModel(sdk, modelID, options) {\n return sdk.responses(modelID) // Use responses() not chat()\n },\n }\n}\n```\n\nSources: [packages/opencode/src/provider/provider.ts:154-162]()\n\n**GitHub Copilot**\n\n```typescript\n// Decides between responses() and chat() based on model\n\"github-copilot\": async () => {\n return {\n autoload: false,\n async getModel(sdk, modelID, options) {\n if (shouldUseCopilotResponsesApi(modelID)) {\n return sdk.responses(modelID)\n }\n return sdk.chat(modelID)\n }\n }\n}\n```\n\nSources: [packages/opencode/src/provider/provider.ts:163-172](), [packages/opencode/src/provider/provider.ts:52-62]()\n\n**Amazon Bedrock Region Prefixing**\n\nBedrock requires special model ID transformation based on region:\n\n```typescript\n// Prefix models with region for cross-region inference\n\"amazon-bedrock\": async () => {\n return {\n async getModel(sdk, modelID, options) {\n const region = options?.region ?? defaultRegion\n let regionPrefix = region.split(\"-\")[0]\n\n // Special handling for us., eu., ap. prefixes\n if (regionPrefix === \"us\" && requiresPrefix(modelID)) {\n modelID = `us.${modelID}`\n }\n // ... more region logic\n\n return sdk.languageModel(modelID)\n }\n }\n}\n```\n\nSources: [packages/opencode/src/provider/provider.ts:212-358]()\n\n---\n\n## OpenCode Zen and Go Services\n\nOpenCode provides two optional managed services for simplified model access.\n\n### Service Comparison\n\n| Feature | OpenCode Zen | OpenCode Go |\n| -------------- | ----------------------------- | ---------------------------- |\n| Model Access | Curated premium models | Popular open models |\n| Pricing | Per-request | Low-cost subscription |\n| Authentication | API key from console | API key from console |\n| Base URL | `https://opencode.ai/zen/v1/` | `https://opencode.ai/go/v1/` |\n\nSources: [packages/web/src/content/docs/zen.mdx:1-100]()\n\n### Zen Model Endpoints\n\nOpenCode Zen provides access to verified models through provider-specific endpoints:\n\n| Model Family | Endpoint Path | AI SDK Package |\n| ------------ | ------------------- | --------------------------- |\n| GPT-5.x | `/responses` | `@ai-sdk/openai` |\n| Claude 4.x | `/messages` | `@ai-sdk/anthropic` |\n| Gemini 3.x | `/models/gemini-*` | `@ai-sdk/google` |\n| Others | `/chat/completions` | `@ai-sdk/openai-compatible` |\n\nSources: [packages/web/src/content/docs/zen.mdx:62-95]()\n\n### Zen Provider Implementation\n\n```typescript\n// Custom loader removes paid models when no API key present\nasync opencode(input) {\n const hasKey = await checkForApiKey()\n\n if (!hasKey) {\n // Remove paid models, keep only free tier\n for (const [key, value] of Object.entries(input.models)) {\n if (value.cost.input === 0) continue\n delete input.models[key]\n }\n }\n\n return {\n autoload: Object.keys(input.models).length > 0,\n options: hasKey ? {} : { apiKey: \"public\" }\n }\n}\n```\n\nSources: [packages/opencode/src/provider/provider.ts:132-153]()\n\n---\n\n## Provider Configuration\n\n### Authentication Storage\n\nProvider credentials are stored in `~/.local/share/opencode/auth.json`:\n\n```typescript\n// Auth.Info schema\n{\n type: \"api\" // or \"oauth\" or \"wellknown\"\n key: string // API key\n access?: string // OAuth access token\n refresh?: string // OAuth refresh token\n token?: string // Well-known token\n}\n```\n\nSources: [packages/opencode/src/auth.ts:20-50]() (referenced)\n\n### Provider Config Schema\n\nProviders can be configured in `opencode.json`:\n\n```json\n{\n \"provider\": {\n \"<provider-id>\": {\n \"options\": {\n \"baseURL\": \"https://custom-endpoint.com\",\n \"apiKey\": \"${CUSTOM_API_KEY}\",\n \"headers\": {\n \"X-Custom-Header\": \"value\"\n }\n },\n \"default\": \"model-id\"\n }\n }\n}\n```\n\nSources: [packages/opencode/src/config/config.ts:1000-1200]() (referenced)\n\n### Base URL Templating\n\nProvider base URLs support environment variable interpolation:\n\n```typescript\nfunction loadBaseURL(model, options) {\n const raw = options['baseURL'] ?? model.api.url\n\n // For Google Vertex, expand special variables\n const vars =\n model.providerID === 'google-vertex' ? googleVertexVars(options) : undefined\n\n return raw.replace(/\\$\\{([^}]+)\\}/g, (match, key) => {\n const val = Env.get(key) ?? vars?.[key]\n return val ?? match\n })\n}\n```\n\nSources: [packages/opencode/src/provider/provider.ts:78-86]()\n\n---\n\n## Model Routing in Sessions\n\nWhen a session prompt is created, the model is selected and instantiated through this flow:\n\n**Session to Model Resolution**\n\n```mermaid\ngraph TB\n Prompt[\"SessionPrompt.prompt()\"]\n UserMsg[\"User Message<br/>lastUser.model\"]\n\n GetModel[\"Provider.getModel()<br/>(providerID, modelID)\"]\n ModelObj[\"Provider.Model object\"]\n\n Transform[\"ProviderTransform<br/>normalize messages\"]\n Options[\"ProviderTransform.options()<br/>build provider options\"]\n\n Stream[\"streamText()<br/>AI SDK\"]\n\n Prompt --> UserMsg\n UserMsg --> GetModel\n GetModel --> ModelObj\n ModelObj --> Transform\n Transform --> Options\n Options --> Stream\n```\n\nSources: [packages/opencode/src/session/prompt.ts:336-347](), [packages/opencode/src/session/llm.ts:1-200]()\n\n### Model Variant Support\n\nReasoning models support variants that control inference effort:\n\n**Variant Configuration**\n\n| Provider | Variants | Parameter |\n| ---------- | --------------------------------------------------- | --------------------------- |\n| OpenAI | `none`, `minimal`, `low`, `medium`, `high`, `xhigh` | `reasoningEffort` |\n| Anthropic | `low`, `medium`, `high`, `max` | `thinking.type: \"adaptive\"` |\n| Gemini | `low`, `high` | `thinkingLevel` |\n| OpenRouter | Same as underlying model | `reasoning.effort` |\n\nSources: [packages/opencode/src/provider/transform.ts:329-500]()\n\nVariants are applied via `ProviderTransform.variants()` which returns provider-specific option objects.\n\n---\n\n## Error Handling\n\n### Provider Authentication Errors\n\n```mermaid\ngraph TB\n Request[\"AI Request\"]\n Error[\"Error Response\"]\n\n LoadAPIKey[\"LoadAPIKeyError\"]\n AuthError[\"Return ProviderAuthError\"]\n\n OtherError[\"Other Error\"]\n ProcessError[\"Process error message\"]\n\n Request --> Error\n Error --> LoadAPIKey\n Error --> OtherError\n\n LoadAPIKey --> AuthError\n OtherError --> ProcessError\n```\n\nSources: [packages/opencode/src/provider/error.ts:1-100]() (referenced), [packages/opencode/src/session/message-v2.ts:33-49]()\n\nThe `MessageV2.AuthError` type captures provider authentication failures with the provider ID and error message for user-friendly display.\n\n### API Error Normalization\n\nAll provider errors are normalized to `MessageV2.APIError`:\n\n```typescript\n{\n name: \"APIError\"\n data: {\n message: string\n statusCode?: number\n isRetryable: boolean\n responseHeaders?: Record<string, string>\n responseBody?: string\n metadata?: Record<string, string>\n }\n}\n```\n\nSources: [packages/opencode/src/session/message-v2.ts:40-50]()\n\n---\n\n## Summary\n\nThe AI Provider & Model Management system in OpenCode provides:\n\n1. **Unified Provider Abstraction** - Single interface for 75+ providers via AI SDK\n2. **Model Metadata Service** - Models.dev integration for capabilities and pricing\n3. **Provider Transform Layer** - Message normalization and provider-specific optimizations\n4. **Custom Loaders** - Per-provider logic for authentication and model instantiation\n5. **Optional Managed Services** - OpenCode Zen and Go for simplified access\n6. **Flexible Configuration** - Per-provider options, base URL overrides, authentication\n\nThe architecture separates concerns between provider discovery ([packages/opencode/src/provider/provider.ts]()), model metadata ([packages/opencode/src/provider/models.ts]()), and message transformation ([packages/opencode/src/provider/transform.ts]()), allowing each component to evolve independently while maintaining a consistent interface for session prompting.\n"}}}} |