hack/tools/scripts/llm-key-hunter/results/_cache/content/bai936191-afk_baibai-guochan-llm.json

{"repo": "bai936191-afk/baibai-guochan-llm", "files": {"docs/en/guide/third-party-models.md": {"fdc412498a9f751abe10ae85d2a5755bb113c7ba": {"t": 1786067812.2321587, "n": 8118, "h": "8b77ff7ac7bb5634", "c": "# Using Third-Party Models (OpenAI / DeepSeek / Local Models)\n\nThis project communicates with LLMs via the Anthropic protocol. By using a protocol translation proxy, you can use any model including OpenAI, DeepSeek, Ollama, etc.\n\n## How It Works\n\n```\nclaude-code-haha ──Anthropic protocol──▶ LiteLLM Proxy ──OpenAI protocol──▶ Target Model API\n                                          (translation)\n```\n\nThis project sends Anthropic Messages API requests. The LiteLLM proxy automatically translates them to OpenAI Chat Completions API format and forwards them to the target model.\n\n---\n\n## Option 1: LiteLLM Proxy (Recommended)\n\n[LiteLLM](https://github.com/BerriAI/litellm) is a unified proxy gateway supporting 100+ LLMs (41k+ GitHub Stars), with native support for receiving Anthropic protocol requests.\n\n### 1. Install LiteLLM\n\n```bash\npip install 'litellm[proxy]'\n```\n\n### 2. Create Configuration File\n\nCreate `litellm_config.yaml`:\n\n#### Using OpenAI Models\n\n```yaml\nmodel_list:\n  - model_name: gpt-4o\n    litellm_params:\n      model: openai/gpt-4o\n      api_key: os.environ/OPENAI_API_KEY\n\nlitellm_settings:\n  drop_params: true  # Drop Anthropic-specific params (thinking, etc.)\n```\n\n#### Using DeepSeek Models\n\n```yaml\nmodel_list:\n  - model_name: deepseek-chat\n    litellm_params:\n      model: deepseek/deepseek-chat\n      api_key: os.environ/DEEPSEEK_API_KEY\n      api_base: https://api.deepseek.com\n\nlitellm_settings:\n  drop_params: true\n```\n\n#### Using Ollama Local Models\n\n```yaml\nmodel_list:\n  - model_name: llama3\n    litellm_params:\n      model: ollama/llama3\n      api_base: http://localhost:11434\n\nlitellm_settings:\n  drop_params: true\n```\n\n#### Using Multiple Models (switchable after startup)\n\n```yaml\nmodel_list:\n  - model_name: gpt-4o\n    litellm_params:\n      model: openai/gpt-4o\n      api_key: os.environ/OPENAI_API_KEY\n\n  - model_name: deepseek-chat\n    litellm_params:\n      model: deepseek/deepseek-chat\n      api_key: os.environ/DEEPSEEK_API_KEY\n      api_base: https://api.deepseek.com\n\n  - model_name: llama3\n    litellm_params:\n      model: ollama/llama3\n      api_base: http://localhost:11434\n\nlitellm_settings:\n  drop_params: true\n```\n\n### 3. Start the Proxy\n\n```bash\n# Set your target model's API key\nexport OPENAI_API_KEY=sk-xxx\n# or\nexport DEEPSEEK_API_KEY=sk-xxx\n\n# Start the proxy\nlitellm --config litellm_config.yaml --port 4000\n```\n\nThe proxy will listen on `http://localhost:4000` and expose an Anthropic-compatible `/v1/messages` endpoint.\n\n### 4. Configure This Project\n\nChoose one of two configuration methods:\n\n#### Method A: Via `.env` File\n\n```bash\nANTHROPIC_AUTH_TOKEN=sk-anything\nANTHROPIC_BASE_URL=http://localhost:4000\nANTHROPIC_MODEL=gpt-4o\nANTHROPIC_DEFAULT_SONNET_MODEL=gpt-4o\nANTHROPIC_DEFAULT_HAIKU_MODEL=gpt-4o\nANTHROPIC_DEFAULT_OPUS_MODEL=gpt-4o\nAPI_TIMEOUT_MS=3000000\nDISABLE_TELEMETRY=1\nCLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1\n```\n\n#### Method B: Via `~/.claude/settings.json`\n\n```json\n{\n  \"env\": {\n    \"ANTHROPIC_AUTH_TOKEN\": \"sk-anything\",\n    \"ANTHROPIC_BASE_URL\": \"http://localhost:4000\",\n    \"ANTHROPIC_MODEL\": \"gpt-4o\",\n    \"ANTHROPIC_DEFAULT_SONNET_MODEL\": \"gpt-4o\",\n    \"ANTHROPIC_DEFAULT_HAIKU_MODEL\": \"gpt-4o\",\n    \"ANTHROPIC_DEFAULT_OPUS_MODEL\": \"gpt-4o\",\n    \"API_TIMEOUT_MS\": \"3000000\",\n    \"DISABLE_TELEMETRY\": \"1\",\n    \"CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC\": \"1\"\n  }\n}\n```\n\n> **Note**: The `ANTHROPIC_AUTH_TOKEN` value can be any string when using the LiteLLM proxy (LiteLLM uses its own configured key for forwarding), unless you've set a `master_key` on the LiteLLM side.\n\n### 5. Start and Verify\n\n```bash\n./bin/claude-haha\n```\n\nIf everything is configured correctly, you should see the normal chat interface, with your configured target model handling the requests.\n\n---\n\n## Option 2: Direct Connection to Anthropic-Compatible Services\n\nSome third-party services directly support the Anthropic Messages API, no proxy needed:\n\n### OpenRouter\n\n```bash\nANTHROPIC_AUTH_TOKEN=sk-or-v1-xxx\nANTHROPIC_BASE_URL=https://openrouter.ai/api/v1\nANTHROPIC_MODEL=openai/gpt-4o\nANTHROPIC_DEFAULT_SONNET_MODEL=openai/gpt-4o\nANTHROPIC_DEFAULT_HAIKU_MODEL=openai/gpt-4o-mini\nANTHROPIC_DEFAULT_OPUS_MODEL=openai/gpt-4o\nDISABLE_TELEMETRY=1\nCLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1\n```\n\n### MiniMax (pre-configured in .env.example)\n\nMiniMax provides an Anthropic-compatible API endpoint and can be connected directly without any proxy. Available models:\n\n| Model | Description |\n|-------|-------------|\n| `MiniMax-M3` | Default recommended, latest generation with excellent overall performance and 1M context |\n| `MiniMax-M2.7` | Previous stable release |\n| `MiniMax-M2.7-highspeed` | Faster responses, suitable for latency-sensitive use cases |\n\n```bash\nANTHROPIC_AUTH_TOKEN=your_minimax_api_key_here\n# International users: api.minimax.io; China users may use api.minimaxi.com\nANTHROPIC_BASE_URL=https://api.minimax.io/anthropic\nANTHROPIC_MODEL=MiniMax-M3\nANTHROPIC_DEFAULT_SONNET_MODEL=MiniMax-M3\nANTHROPIC_DEFAULT_HAIKU_MODEL=MiniMax-M2.7-highspeed\nANTHROPIC_DEFAULT_OPUS_MODEL=MiniMax-M3\nAPI_TIMEOUT_MS=3000000\nDISABLE_TELEMETRY=1\nCLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1\n```\n\n> **Get API Key**: Visit [MiniMax Open Platform](https://platform.minimax.io) to register and obtain an API key.\n\n---\n\n## Option 3: Other Proxy Tools\n\nThe community has built several proxy tools specifically for Claude Code:\n\n| Tool | Description | Link |\n|------|-------------|------|\n| **a2o** | Anthropic → OpenAI single binary, zero dependencies | [Twitter](https://x.com/mantou543/status/2018846154855940200) |\n| **Empero Proxy** | Full Anthropic Messages API to OpenAI translation | [Twitter](https://x.com/EmperoAI/status/2036840854065762551) |\n| **Alma** | Client with built-in OpenAI → Anthropic proxy | [Twitter](https://x.com/yetone/status/2003508782127833332) |\n| **Chutes** | Docker container supporting 60+ open-source models | [Twitter](https://x.com/chutes_ai/status/2027039742915662232) |\n\n---\n\n## Known Limitations\n\n### 1. `drop_params: true` Is Essential\n\nThis project sends Anthropic-specific parameters (e.g., `thinking`, `cache_control`) that don't exist in the OpenAI API. You must set `drop_params: true` in the LiteLLM config, otherwise requests will fail.\n\n### 2. Extended Thinking Unavailable\n\nAnthropic's Extended Thinking is a proprietary feature not supported by other models. It is automatically disabled when using third-party models.\n\n### 3. Prompt Caching Unavailable\n\n`cache_control` is an Anthropic-specific feature. Prompt caching won't work with third-party models (but won't cause errors — it's silently ignored by `drop_params`).\n\n### 4. Tool Calling Compatibility\n\nThis project heavily uses tool calling (tool_use). LiteLLM automatically translates Anthropic's tool_use format to OpenAI's function_calling format. This works in most cases, but some complex tool calls may have compatibility issues. If you encounter problems, try using a more capable model (e.g., GPT-4o).\n\n### 5. Telemetry and Non-Essential Requests\n\nConfigure these environment variables to avoid unnecessary network requests:\n```\nDISABLE_TELEMETRY=1\nCLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1\n```\n\n---\n\n## FAQ\n\n### Q: LiteLLM proxy returns `/v1/responses` not found?\n\nSome OpenAI-compatible services only support `/v1/chat/completions`. Add this to your LiteLLM config:\n\n```yaml\nlitellm_settings:\n  use_chat_completions_url_for_anthropic_messages: true\n```\n\n### Q: What's the difference between `ANTHROPIC_API_KEY` and `ANTHROPIC_AUTH_TOKEN`?\n\n- `ANTHROPIC_API_KEY` → Sent via `x-api-key` header\n- `ANTHROPIC_AUTH_TOKEN` → Sent via `Authorization: Bearer` header\n\nLiteLLM proxy accepts Bearer Token format by default, so `ANTHROPIC_AUTH_TOKEN` is recommended.\n\n### Q: Can I configure multiple models?\n\nYes. Define multiple `model_name` entries in `litellm_config.yaml`, then switch by changing the `ANTHROPIC_MODEL` value.\n\n### Q: Local Ollama models don't work well?\n\nThis project's system prompts and tool calls require strong model capabilities. Use larger models (e.g., Llama 3 70B+, Qwen 72B+). Smaller models may fail to handle tool calling correctly.\n"}}}}