1 line
75 KiB
JSON
1 line
75 KiB
JSON
{"repo": "artcc/openclient-llm", "files": {"LiteLLM.md": {"870be6d78f723c2d8c0b3beffe6a5b6ba4ce796c": {"t": 1785746511.6736376, "n": 5619, "h": "f091c245bf1f6d56", "c": "# Running LiteLLM with Docker\n\nThis guide shows how to self-host [LiteLLM](https://docs.litellm.ai) as an OpenAI-compatible proxy using Docker Compose so you can use it as a backend with OpenClient LLM. It covers the full stack: Postgres for persistence, a reference `config.yaml` with both local (Ollama) and cloud models, and common operational commands.\n\nConfigure OpenClient with the proxy root, for example `http://your-server:4000`, not\n`http://your-server:4000/v1`. OpenClient appends endpoint paths verbatim: core requests use `/models`,\n`/model/info`, and `/chat/completions`; audio and search requests use their explicit `/v1/...` paths.\n\n## Reference\n\n- https://docs.litellm.ai/docs/proxy/deploy\n- https://docs.litellm.ai/docs/proxy/configs\n- https://github.com/BerriAI/litellm\n\n## docker-compose.yml\n\n```yaml\nservices:\n litellm-db:\n image: postgres:16-alpine\n container_name: litellm-db\n restart: unless-stopped\n environment:\n POSTGRES_USER: litellm\n POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}\n POSTGRES_DB: litellm\n volumes:\n - /opt/docker/litellm/pgdata:/var/lib/postgresql/data\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U litellm -d litellm\"]\n interval: 10s\n timeout: 5s\n retries: 5\n\n litellm:\n image: docker.litellm.ai/berriai/litellm-database:main-stable\n container_name: litellm\n restart: unless-stopped\n depends_on:\n litellm-db:\n condition: service_healthy\n volumes:\n - /opt/docker/litellm/config.yaml:/app/config.yaml\n ports:\n - \"4000:4000\"\n environment:\n LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}\n DATABASE_URL: ${DATABASE_URL}\n OPENAI_API_KEY: ${OPENAI_API_KEY}\n ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}\n GOOGLE_API_KEY: ${GOOGLE_API_KEY}\n DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY}\n FIRECRAWL_API_KEY: ${FIRECRAWL_API_KEY}\n BRAVE_API_KEY: ${BRAVE_API_KEY}\n command: [\"--config\", \"/app/config.yaml\", \"--port\", \"4000\"]\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n```\n\n## Reference .env\n\n```env\nLITELLM_MASTER_KEY=sk-your-master-key\nPOSTGRES_PASSWORD=your-postgres-password\nDATABASE_URL=postgresql://litellm:your-postgres-password@litellm-db:5432/litellm\nOPENAI_API_KEY=your-openai-api-key\nANTHROPIC_API_KEY=your-anthropic-api-key\nGOOGLE_API_KEY=your-google-api-key\nDEEPSEEK_API_KEY=your-deepseek-api-key\nFIRECRAWL_API_KEY=your-firecrawl-api-key\nBRAVE_API_KEY=your-brave-api-key\n```\n\n## Reference config.yaml\n\nCreate at `/opt/docker/litellm/config.yaml`:\n\n```yaml\nmodel_list:\n # Anthropic\n - model_name: claude-sonnet-4-6\n litellm_params:\n model: anthropic/claude-sonnet-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n # DeepSeek\n - model_name: deepseek-chat\n litellm_params:\n model: deepseek/deepseek-chat\n api_key: os.environ/DEEPSEEK_API_KEY\n\n # Google\n - model_name: gemini-2.5-pro\n litellm_params:\n model: gemini/gemini-2.5-pro\n api_key: os.environ/GOOGLE_API_KEY\n\n # Ollama\n - model_name: gemma4:e4b\n litellm_params:\n model: ollama/gemma4:e4b\n api_base: http://your_ollama_ip_server:11434\n\n # OpenAI\n - model_name: gpt-4o\n litellm_params:\n model: openai/gpt-4o\n api_key: os.environ/OPENAI_API_KEY\n\nsearch_tools:\n - search_tool_name: brave-search\n litellm_params:\n search_provider: brave\n api_key: os.environ/BRAVE_API_KEY\n\n - search_tool_name: firecrawl-search\n litellm_params:\n search_provider: firecrawl\n api_key: os.environ/FIRECRAWL_API_KEY\n\n - search_tool_name: searxng-search\n litellm_params:\n search_provider: searxng\n api_base: https://serxng-deployment-production.up.railway.app\n\ngeneral_settings:\n master_key: os.environ/LITELLM_MASTER_KEY\n database_url: os.environ/DATABASE_URL\n```\n\n## Operational notes\n\nPrepare the data directory:\n\n```bash\nsudo mkdir -p /opt/docker/litellm\nsudo chown -R 1000:1000 /opt/docker/litellm\n```\n\nCreate the configuration file:\n\n```bash\nsudo nano /opt/docker/litellm/config.yaml\n```\n\nStart the service:\n\n```bash\ndocker compose up -d\n```\n\nVerify the proxy is running:\n\n```bash\ncurl http://localhost:4000/health/readiness\n```\n\nList available models:\n\n```bash\ncurl http://localhost:4000/models \\\n -H \"Authorization: Bearer sk-your-master-key\"\n```\n\nTest a chat completion against a local model:\n\n```bash\ncurl http://localhost:4000/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer sk-your-master-key\" \\\n -d '{\n \"model\": \"qwen3:14b\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]\n }'\n```\n\nInternal DNS (e.g. Pi-hole):\n\n- `litellm.yourdomain.com` → `192.168.x.x`\n\n## Notes\n\n- `LITELLM_MASTER_KEY` must start with `sk-`. Generate one with:\n\n```bash\necho \"sk-$(openssl rand -hex 32)\"\n```\n\n It is used in the `Authorization` header for every API call:\n\n```bash\nAuthorization: Bearer sk-xxxxxxxx...\n```\n\nAny client (app, n8n, curl, etc.) must include it to use LiteLLM.\n\n- `host.docker.internal` resolves to the host IP via `extra_hosts`, required for LiteLLM to reach Ollama running on the host.\n- The `litellm-database` image includes Postgres support for virtual keys and cost tracking. Use `litellm:main-stable` if you don't need a database.\n- The admin dashboard is available at `https://litellm.yourdomain.com/ui`.\n- OpenClient's basic model listing and chat work without `/model/info`; model metadata and capabilities are then best-effort.\n- Image output is supported when a configured model/provider returns image data in chat-completion stream chunks.\n OpenClient does not call LiteLLM's dedicated image-generation endpoint.\n"}, "c8175ab4596d71eddc42c78cd376f4027509e17e": {"t": 1785765098.6831787, "n": 5619, "h": "f091c245bf1f6d56", "c": "# Running LiteLLM with Docker\n\nThis guide shows how to self-host [LiteLLM](https://docs.litellm.ai) as an OpenAI-compatible proxy using Docker Compose so you can use it as a backend with OpenClient LLM. It covers the full stack: Postgres for persistence, a reference `config.yaml` with both local (Ollama) and cloud models, and common operational commands.\n\nConfigure OpenClient with the proxy root, for example `http://your-server:4000`, not\n`http://your-server:4000/v1`. OpenClient appends endpoint paths verbatim: core requests use `/models`,\n`/model/info`, and `/chat/completions`; audio and search requests use their explicit `/v1/...` paths.\n\n## Reference\n\n- https://docs.litellm.ai/docs/proxy/deploy\n- https://docs.litellm.ai/docs/proxy/configs\n- https://github.com/BerriAI/litellm\n\n## docker-compose.yml\n\n```yaml\nservices:\n litellm-db:\n image: postgres:16-alpine\n container_name: litellm-db\n restart: unless-stopped\n environment:\n POSTGRES_USER: litellm\n POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}\n POSTGRES_DB: litellm\n volumes:\n - /opt/docker/litellm/pgdata:/var/lib/postgresql/data\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U litellm -d litellm\"]\n interval: 10s\n timeout: 5s\n retries: 5\n\n litellm:\n image: docker.litellm.ai/berriai/litellm-database:main-stable\n container_name: litellm\n restart: unless-stopped\n depends_on:\n litellm-db:\n condition: service_healthy\n volumes:\n - /opt/docker/litellm/config.yaml:/app/config.yaml\n ports:\n - \"4000:4000\"\n environment:\n LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}\n DATABASE_URL: ${DATABASE_URL}\n OPENAI_API_KEY: ${OPENAI_API_KEY}\n ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}\n GOOGLE_API_KEY: ${GOOGLE_API_KEY}\n DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY}\n FIRECRAWL_API_KEY: ${FIRECRAWL_API_KEY}\n BRAVE_API_KEY: ${BRAVE_API_KEY}\n command: [\"--config\", \"/app/config.yaml\", \"--port\", \"4000\"]\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n```\n\n## Reference .env\n\n```env\nLITELLM_MASTER_KEY=sk-your-master-key\nPOSTGRES_PASSWORD=your-postgres-password\nDATABASE_URL=postgresql://litellm:your-postgres-password@litellm-db:5432/litellm\nOPENAI_API_KEY=your-openai-api-key\nANTHROPIC_API_KEY=your-anthropic-api-key\nGOOGLE_API_KEY=your-google-api-key\nDEEPSEEK_API_KEY=your-deepseek-api-key\nFIRECRAWL_API_KEY=your-firecrawl-api-key\nBRAVE_API_KEY=your-brave-api-key\n```\n\n## Reference config.yaml\n\nCreate at `/opt/docker/litellm/config.yaml`:\n\n```yaml\nmodel_list:\n # Anthropic\n - model_name: claude-sonnet-4-6\n litellm_params:\n model: anthropic/claude-sonnet-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n # DeepSeek\n - model_name: deepseek-chat\n litellm_params:\n model: deepseek/deepseek-chat\n api_key: os.environ/DEEPSEEK_API_KEY\n\n # Google\n - model_name: gemini-2.5-pro\n litellm_params:\n model: gemini/gemini-2.5-pro\n api_key: os.environ/GOOGLE_API_KEY\n\n # Ollama\n - model_name: gemma4:e4b\n litellm_params:\n model: ollama/gemma4:e4b\n api_base: http://your_ollama_ip_server:11434\n\n # OpenAI\n - model_name: gpt-4o\n litellm_params:\n model: openai/gpt-4o\n api_key: os.environ/OPENAI_API_KEY\n\nsearch_tools:\n - search_tool_name: brave-search\n litellm_params:\n search_provider: brave\n api_key: os.environ/BRAVE_API_KEY\n\n - search_tool_name: firecrawl-search\n litellm_params:\n search_provider: firecrawl\n api_key: os.environ/FIRECRAWL_API_KEY\n\n - search_tool_name: searxng-search\n litellm_params:\n search_provider: searxng\n api_base: https://serxng-deployment-production.up.railway.app\n\ngeneral_settings:\n master_key: os.environ/LITELLM_MASTER_KEY\n database_url: os.environ/DATABASE_URL\n```\n\n## Operational notes\n\nPrepare the data directory:\n\n```bash\nsudo mkdir -p /opt/docker/litellm\nsudo chown -R 1000:1000 /opt/docker/litellm\n```\n\nCreate the configuration file:\n\n```bash\nsudo nano /opt/docker/litellm/config.yaml\n```\n\nStart the service:\n\n```bash\ndocker compose up -d\n```\n\nVerify the proxy is running:\n\n```bash\ncurl http://localhost:4000/health/readiness\n```\n\nList available models:\n\n```bash\ncurl http://localhost:4000/models \\\n -H \"Authorization: Bearer sk-your-master-key\"\n```\n\nTest a chat completion against a local model:\n\n```bash\ncurl http://localhost:4000/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer sk-your-master-key\" \\\n -d '{\n \"model\": \"qwen3:14b\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]\n }'\n```\n\nInternal DNS (e.g. Pi-hole):\n\n- `litellm.yourdomain.com` → `192.168.x.x`\n\n## Notes\n\n- `LITELLM_MASTER_KEY` must start with `sk-`. Generate one with:\n\n```bash\necho \"sk-$(openssl rand -hex 32)\"\n```\n\n It is used in the `Authorization` header for every API call:\n\n```bash\nAuthorization: Bearer sk-xxxxxxxx...\n```\n\nAny client (app, n8n, curl, etc.) must include it to use LiteLLM.\n\n- `host.docker.internal` resolves to the host IP via `extra_hosts`, required for LiteLLM to reach Ollama running on the host.\n- The `litellm-database` image includes Postgres support for virtual keys and cost tracking. Use `litellm:main-stable` if you don't need a database.\n- The admin dashboard is available at `https://litellm.yourdomain.com/ui`.\n- OpenClient's basic model listing and chat work without `/model/info`; model metadata and capabilities are then best-effort.\n- Image output is supported when a configured model/provider returns image data in chat-completion stream chunks.\n OpenClient does not call LiteLLM's dedicated image-generation endpoint.\n"}, "ba23c7568f959bf94950bacfeb0ac2cf5f7e6373": {"t": 1785765100.0468273, "n": 5102, "h": "2dde3e37f5b47fea", "c": "# Running LiteLLM with Docker\n\nThis guide shows how to self-host [LiteLLM](https://docs.litellm.ai) as an OpenAI-compatible proxy using Docker Compose so you can use it as a backend with OpenClient LLM. It covers the full stack: Postgres for persistence, a reference `config.yaml` with both local (Ollama) and cloud models, and common operational commands.\n\n## Reference\n\n- https://docs.litellm.ai/docs/proxy/deploy\n- https://docs.litellm.ai/docs/proxy/configs\n- https://github.com/BerriAI/litellm\n\n## docker-compose.yml\n\n```yaml\nservices:\n litellm-db:\n image: postgres:16-alpine\n container_name: litellm-db\n restart: unless-stopped\n environment:\n POSTGRES_USER: litellm\n POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}\n POSTGRES_DB: litellm\n volumes:\n - /opt/docker/litellm/pgdata:/var/lib/postgresql/data\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U litellm -d litellm\"]\n interval: 10s\n timeout: 5s\n retries: 5\n\n litellm:\n image: docker.litellm.ai/berriai/litellm-database:main-stable\n container_name: litellm\n restart: unless-stopped\n depends_on:\n litellm-db:\n condition: service_healthy\n volumes:\n - /opt/docker/litellm/config.yaml:/app/config.yaml\n ports:\n - \"4000:4000\"\n environment:\n LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}\n DATABASE_URL: ${DATABASE_URL}\n OPENAI_API_KEY: ${OPENAI_API_KEY}\n ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}\n GOOGLE_API_KEY: ${GOOGLE_API_KEY}\n DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY}\n FIRECRAWL_API_KEY: ${FIRECRAWL_API_KEY}\n BRAVE_API_KEY: ${BRAVE_API_KEY}\n command: [\"--config\", \"/app/config.yaml\", \"--port\", \"4000\"]\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n```\n\n## Reference .env\n\n```env\nLITELLM_MASTER_KEY=sk-your-master-key\nPOSTGRES_PASSWORD=your-postgres-password\nDATABASE_URL=postgresql://litellm:your-postgres-password@litellm-db:5432/litellm\nOPENAI_API_KEY=your-openai-api-key\nANTHROPIC_API_KEY=your-anthropic-api-key\nGOOGLE_API_KEY=your-google-api-key\nDEEPSEEK_API_KEY=your-deepseek-api-key\nFIRECRAWL_API_KEY=your-firecrawl-api-key\nBRAVE_API_KEY=your-brave-api-key\n```\n\n## Reference config.yaml\n\nCreate at `/opt/docker/litellm/config.yaml`:\n\n```yaml\nmodel_list:\n # Anthropic\n - model_name: claude-sonnet-4-6\n litellm_params:\n model: anthropic/claude-sonnet-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n # DeepSeek\n - model_name: deepseek-chat\n litellm_params:\n model: deepseek/deepseek-chat\n api_key: os.environ/DEEPSEEK_API_KEY\n\n # Google\n - model_name: gemini-2.5-pro\n litellm_params:\n model: gemini/gemini-2.5-pro\n api_key: os.environ/GOOGLE_API_KEY\n\n # Ollama\n - model_name: gemma4:e4b\n litellm_params:\n model: ollama/gemma4:e4b\n api_base: http://your_ollama_ip_server:11434\n\n # OpenAI\n - model_name: gpt-4o\n litellm_params:\n model: openai/gpt-4o\n api_key: os.environ/OPENAI_API_KEY\n\nsearch_tools:\n - search_tool_name: brave-search\n litellm_params:\n search_provider: brave\n api_key: os.environ/BRAVE_API_KEY\n\n - search_tool_name: firecrawl-search\n litellm_params:\n search_provider: firecrawl\n api_key: os.environ/FIRECRAWL_API_KEY\n\n - search_tool_name: searxng-search\n litellm_params:\n search_provider: searxng\n api_base: https://serxng-deployment-production.up.railway.app\n\ngeneral_settings:\n master_key: os.environ/LITELLM_MASTER_KEY\n database_url: os.environ/DATABASE_URL\n```\n\n## Operational notes\n\nPrepare the data directory:\n\n```bash\nsudo mkdir -p /opt/docker/litellm\nsudo chown -R 1000:1000 /opt/docker/litellm\n```\n\nCreate the configuration file:\n\n```bash\nsudo nano /opt/docker/litellm/config.yaml\n```\n\nStart the service:\n\n```bash\ndocker compose up -d\n```\n\nVerify the proxy is running:\n\n```bash\ncurl http://localhost:4000/health\n```\n\nList available models:\n\n```bash\ncurl http://localhost:4000/models \\\n -H \"Authorization: Bearer sk-your-master-key\"\n```\n\nTest a chat completion against a local model:\n\n```bash\ncurl http://localhost:4000/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer sk-your-master-key\" \\\n -d '{\n \"model\": \"qwen3:14b\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]\n }'\n```\n\nInternal DNS (e.g. Pi-hole):\n\n- `litellm.yourdomain.com` → `192.168.x.x`\n\n## Notes\n\n- `LITELLM_MASTER_KEY` must start with `sk-`. Generate one with:\n\n```bash\necho \"sk-$(openssl rand -hex 32)\"\n```\n\n It is used in the `Authorization` header for every API call:\n\n```bash\nAuthorization: Bearer sk-xxxxxxxx...\n```\n\nAny client (app, n8n, curl, etc.) must include it to use LiteLLM.\n\n- `host.docker.internal` resolves to the host IP via `extra_hosts`, required for LiteLLM to reach Ollama running on the host.\n- The `litellm-database` image includes Postgres support for virtual keys and cost tracking. Use `litellm:main-stable` if you don't need a database.\n- The admin dashboard is available at `https://litellm.yourdomain.com/ui`.\n- Any OpenAI-compatible client can point to LiteLLM by changing only the `base_url` and `api_key`."}, "a83ac2f0a379cb8970f10752e4cca2aedf79a537": {"t": 1785765101.9834168, "n": 6175, "h": "13d215144d564dda", "c": "# Running LiteLLM with Docker\n\nThis guide shows how to self-host [LiteLLM](https://docs.litellm.ai) as an OpenAI-compatible proxy using Docker Compose so you can use it as a backend with OpenClient LLM. It covers the full stack: Postgres for persistence, Traefik as a reverse proxy, a reference `config.yaml` with both local (Ollama) and cloud models, and common operational commands.\n\n## Reference\n\n- https://docs.litellm.ai/docs/proxy/deploy\n- https://docs.litellm.ai/docs/proxy/configs\n- https://github.com/BerriAI/litellm\n\n## docker-compose.yml\n\n```yaml\nservices:\n litellm-db:\n image: postgres:16-alpine\n container_name: litellm-db\n restart: unless-stopped\n environment:\n POSTGRES_USER: litellm\n POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}\n POSTGRES_DB: litellm\n volumes:\n - /opt/docker/litellm/pgdata:/var/lib/postgresql/data\n networks:\n - proxy\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U litellm -d litellm\"]\n interval: 10s\n timeout: 5s\n retries: 5\n\n litellm:\n image: docker.litellm.ai/berriai/litellm-database:main-stable\n container_name: litellm\n restart: unless-stopped\n depends_on:\n litellm-db:\n condition: service_healthy\n volumes:\n - /opt/docker/litellm/config.yaml:/app/config.yaml\n ports:\n - \"4000:4000\"\n networks:\n - proxy\n labels:\n - \"traefik.enable=true\"\n - \"traefik.docker.network=proxy\"\n - \"traefik.http.routers.litellm.rule=Host(`litellm.yourdomain.com`)\"\n - \"traefik.http.routers.litellm.entrypoints=websecure\"\n - \"traefik.http.routers.litellm.tls.certresolver=le\"\n - \"traefik.http.services.litellm.loadbalancer.server.port=4000\"\n environment:\n LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}\n DATABASE_URL: ${DATABASE_URL}\n OPENAI_API_KEY: ${OPENAI_API_KEY}\n ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}\n GOOGLE_API_KEY: ${GOOGLE_API_KEY}\n DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY}\n FIRECRAWL_API_KEY: ${FIRECRAWL_API_KEY}\n BRAVE_API_KEY: ${BRAVE_API_KEY}\n command: [\"--config\", \"/app/config.yaml\", \"--port\", \"4000\"]\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n\nnetworks:\n proxy:\n external: true\n```\n\n## Reference .env\n\n```env\nLITELLM_MASTER_KEY=sk-your-master-key\nPOSTGRES_PASSWORD=your-postgres-password\nDATABASE_URL=postgresql://litellm:your-postgres-password@litellm-db:5432/litellm\nOPENAI_API_KEY=your-openai-api-key\nANTHROPIC_API_KEY=your-anthropic-api-key\nGOOGLE_API_KEY=your-google-api-key\nDEEPSEEK_API_KEY=your-deepseek-api-key\nFIRECRAWL_API_KEY=your-firecrawl-api-key\nBRAVE_API_KEY=your-brave-api-key\n```\n\n## Reference config.yaml\n\nCreate at `/opt/docker/litellm/config.yaml`:\n\n```yaml\nmodel_list:\n # Ollama — Google\n - model_name: gemma4:e4b\n litellm_params:\n model: ollama/gemma4:e4b\n api_base: http://your_ollama_ip_server:11434\n\n # APIs cloud\n # OpenAI\n - model_name: gpt-5.4\n litellm_params:\n model: openai/gpt-5.4\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-4o-mini-tts\n litellm_params:\n model: openai/gpt-4o-mini-tts\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-4o-mini-transcribe\n litellm_params:\n model: openai/gpt-4o-mini-transcribe\n api_key: os.environ/OPENAI_API_KEY\n\n # Anthropic\n - model_name: claude-opus-4-6\n litellm_params:\n model: anthropic/claude-opus-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n - model_name: claude-sonnet-4-6\n litellm_params:\n model: anthropic/claude-sonnet-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n # Google\n - model_name: gemini-3.1-pro-preview\n litellm_params:\n model: gemini/gemini-3.1-pro-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n - model_name: gemini-3-pro-image-preview\n litellm_params:\n model: gemini/gemini-3-pro-image-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n # DeepSeek\n - model_name: deepseek-chat\n litellm_params:\n model: deepseek/deepseek-chat\n api_key: os.environ/DEEPSEEK_API_KEY\n\nsearch_tools:\n - search_tool_name: brave-search\n litellm_params:\n search_provider: brave\n api_key: os.environ/BRAVE_API_KEY\n\n - search_tool_name: firecrawl-search\n litellm_params:\n search_provider: firecrawl\n api_key: os.environ/FIRECRAWL_API_KEY\n\n - search_tool_name: searxng-search\n litellm_params:\n search_provider: searxng\n api_base: https://serxng-deployment-production.up.railway.app\n\ngeneral_settings:\n master_key: os.environ/LITELLM_MASTER_KEY\n database_url: os.environ/DATABASE_URL\n```\n\n## Operational notes\n\nPrepare the data directory:\n\n```bash\nsudo mkdir -p /opt/docker/litellm\nsudo chown -R 1000:1000 /opt/docker/litellm\n```\n\nCreate the configuration file:\n\n```bash\nsudo nano /opt/docker/litellm/config.yaml\n```\n\nStart the service:\n\n```bash\ndocker compose up -d\n```\n\nVerify the proxy is running:\n\n```bash\ncurl http://localhost:4000/health\n```\n\nList available models:\n\n```bash\ncurl http://localhost:4000/models \\\n -H \"Authorization: Bearer sk-your-master-key\"\n```\n\nTest a chat completion against a local model:\n\n```bash\ncurl http://localhost:4000/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer sk-your-master-key\" \\\n -d '{\n \"model\": \"qwen3:14b\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]\n }'\n```\n\nInternal DNS (e.g. Pi-hole):\n\n- `litellm.yourdomain.com` → `192.168.x.x`\n\n## Notes\n\n- `LITELLM_MASTER_KEY` must start with `sk-`. Generate one with:\n\n```bash\necho \"sk-$(openssl rand -hex 32)\"\n```\n\n It is used in the `Authorization` header for every API call:\n\n```bash\nAuthorization: Bearer sk-xxxxxxxx...\n```\n\nAny client (app, n8n, curl, etc.) must include it to use LiteLLM.\n\n- `host.docker.internal` resolves to the host IP via `extra_hosts`, required for LiteLLM to reach Ollama running on the host.\n- The `litellm-database` image includes Postgres support for virtual keys and cost tracking. Use `litellm:main-stable` if you don't need a database.\n- The admin dashboard is available at `https://litellm.yourdomain.com/ui`.\n- Any OpenAI-compatible client can point to LiteLLM by changing only the `base_url` and `api_key`."}, "b66a08aed70363d7878fa83150848d304ce7757a": {"t": 1785765103.6131015, "n": 6495, "h": "8a1f0ac78e50828a", "c": "# Running LiteLLM with Docker\n\nThis guide shows how to self-host [LiteLLM](https://docs.litellm.ai) as an OpenAI-compatible proxy using Docker Compose so you can use it as a backend with OpenClient LLM. It covers the full stack: Postgres for persistence, Traefik as a reverse proxy, a reference `config.yaml` with both local (Ollama) and cloud models, and common operational commands.\n\n## Reference\n\n- https://docs.litellm.ai/docs/proxy/deploy\n- https://docs.litellm.ai/docs/proxy/configs\n- https://github.com/BerriAI/litellm\n\n## docker-compose.yml\n\n```yaml\nservices:\n litellm-db:\n image: postgres:16-alpine\n container_name: litellm-db\n restart: unless-stopped\n environment:\n POSTGRES_USER: litellm\n POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}\n POSTGRES_DB: litellm\n volumes:\n - /opt/docker/litellm/pgdata:/var/lib/postgresql/data\n networks:\n - proxy\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U litellm -d litellm\"]\n interval: 10s\n timeout: 5s\n retries: 5\n\n litellm:\n image: docker.litellm.ai/berriai/litellm-database:main-stable\n container_name: litellm\n restart: unless-stopped\n depends_on:\n litellm-db:\n condition: service_healthy\n volumes:\n - /opt/docker/litellm/config.yaml:/app/config.yaml\n ports:\n - \"4000:4000\"\n networks:\n - proxy\n labels:\n - \"traefik.enable=true\"\n - \"traefik.docker.network=proxy\"\n - \"traefik.http.routers.litellm.rule=Host(`litellm.yourdomain.com`)\"\n - \"traefik.http.routers.litellm.entrypoints=websecure\"\n - \"traefik.http.routers.litellm.tls.certresolver=le\"\n - \"traefik.http.services.litellm.loadbalancer.server.port=4000\"\n environment:\n LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}\n DATABASE_URL: ${DATABASE_URL}\n OPENAI_API_KEY: ${OPENAI_API_KEY}\n ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}\n GOOGLE_API_KEY: ${GOOGLE_API_KEY}\n DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY}\n FIRECRAWL_API_KEY: ${FIRECRAWL_API_KEY}\n BRAVE_API_KEY: ${BRAVE_API_KEY}\n command: [\"--config\", \"/app/config.yaml\", \"--port\", \"4000\"]\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n\nnetworks:\n proxy:\n external: true\n```\n\n## Reference .env\n\n```env\nLITELLM_MASTER_KEY=sk-your-master-key\nPOSTGRES_PASSWORD=your-postgres-password\nDATABASE_URL=postgresql://litellm:your-postgres-password@litellm-db:5432/litellm\nOPENAI_API_KEY=your-openai-api-key\nANTHROPIC_API_KEY=your-anthropic-api-key\nGOOGLE_API_KEY=your-google-api-key\nDEEPSEEK_API_KEY=your-deepseek-api-key\nFIRECRAWL_API_KEY=your-firecrawl-api-key\nBRAVE_API_KEY=your-brave-api-key\n```\n\n## Reference config.yaml\n\nCreate at `/opt/docker/litellm/config.yaml`:\n\n```yaml\nmodel_list:\n # Ollama — Google\n - model_name: gemma4:e4b\n litellm_params:\n model: ollama/gemma4:e4b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: true\n supports_function_calling: false\n\n # Ollama — Qwen (Alibaba)\n - model_name: qwen3:14b\n litellm_params:\n model: ollama/qwen3:14b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n # APIs cloud\n # OpenAI\n - model_name: gpt-5.4\n litellm_params:\n model: openai/gpt-5.4\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-4o-mini-tts\n litellm_params:\n model: openai/gpt-4o-mini-tts\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-4o-mini-transcribe\n litellm_params:\n model: openai/gpt-4o-mini-transcribe\n api_key: os.environ/OPENAI_API_KEY\n\n # Anthropic\n - model_name: claude-opus-4-6\n litellm_params:\n model: anthropic/claude-opus-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n - model_name: claude-sonnet-4-6\n litellm_params:\n model: anthropic/claude-sonnet-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n # Google\n - model_name: gemini-3.1-pro-preview\n litellm_params:\n model: gemini/gemini-3.1-pro-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n - model_name: gemini-3-pro-image-preview\n litellm_params:\n model: gemini/gemini-3-pro-image-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n # DeepSeek\n - model_name: deepseek-chat\n litellm_params:\n model: deepseek/deepseek-chat\n api_key: os.environ/DEEPSEEK_API_KEY\n\nsearch_tools:\n - search_tool_name: brave-search\n litellm_params:\n search_provider: brave\n api_key: os.environ/BRAVE_API_KEY\n\n - search_tool_name: firecrawl-search\n litellm_params:\n search_provider: firecrawl\n api_key: os.environ/FIRECRAWL_API_KEY\n\n - search_tool_name: searxng-search\n litellm_params:\n search_provider: searxng\n api_base: https://serxng-deployment-production.up.railway.app\n\ngeneral_settings:\n master_key: os.environ/LITELLM_MASTER_KEY\n database_url: os.environ/DATABASE_URL\n```\n\n## Operational notes\n\nPrepare the data directory:\n\n```bash\nsudo mkdir -p /opt/docker/litellm\nsudo chown -R 1000:1000 /opt/docker/litellm\n```\n\nCreate the configuration file:\n\n```bash\nsudo nano /opt/docker/litellm/config.yaml\n```\n\nStart the service:\n\n```bash\ndocker compose up -d\n```\n\nVerify the proxy is running:\n\n```bash\ncurl http://localhost:4000/health\n```\n\nList available models:\n\n```bash\ncurl http://localhost:4000/models \\\n -H \"Authorization: Bearer sk-your-master-key\"\n```\n\nTest a chat completion against a local model:\n\n```bash\ncurl http://localhost:4000/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer sk-your-master-key\" \\\n -d '{\n \"model\": \"qwen3:14b\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]\n }'\n```\n\nInternal DNS (e.g. Pi-hole):\n\n- `litellm.yourdomain.com` → `192.168.x.x`\n\n## Notes\n\n- `LITELLM_MASTER_KEY` must start with `sk-`. Generate one with:\n\n```bash\necho \"sk-$(openssl rand -hex 32)\"\n```\n\n It is used in the `Authorization` header for every API call:\n\n```bash\nAuthorization: Bearer sk-xxxxxxxx...\n```\n\nAny client (app, n8n, curl, etc.) must include it to use LiteLLM.\n\n- `host.docker.internal` resolves to the host IP via `extra_hosts`, required for LiteLLM to reach Ollama running on the host.\n- The `litellm-database` image includes Postgres support for virtual keys and cost tracking. Use `litellm:main-stable` if you don't need a database.\n- The admin dashboard is available at `https://litellm.yourdomain.com/ui`.\n- Any OpenAI-compatible client can point to LiteLLM by changing only the `base_url` and `api_key`."}, "e7165733ce996ff5c043a1dc58c5d25c5a53ce99": {"t": 1785765105.3785267, "n": 6497, "h": "785b931d577bef7b", "c": "# Running LiteLLM with Docker\n\nThis guide shows how to self-host [LiteLLM](https://docs.litellm.ai) as an OpenAI-compatible proxy using Docker Compose so you can use it as a backend with OpenClient LLM. It covers the full stack: Postgres for persistence, Traefik as a reverse proxy, a reference `config.yaml` with both local (Ollama) and cloud models, and common operational commands.\n\n## Reference\n\n- https://docs.litellm.ai/docs/proxy/deploy\n- https://docs.litellm.ai/docs/proxy/configs\n- https://github.com/BerriAI/litellm\n\n## docker-compose.yml\n\n```yaml\nservices:\n litellm-db:\n image: postgres:16-alpine\n container_name: litellm-db\n restart: unless-stopped\n environment:\n POSTGRES_USER: litellm\n POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}\n POSTGRES_DB: litellm\n volumes:\n - /opt/docker/litellm/pgdata:/var/lib/postgresql/data\n networks:\n - proxy\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U litellm -d litellm\"]\n interval: 10s\n timeout: 5s\n retries: 5\n\n litellm:\n image: docker.litellm.ai/berriai/litellm-database:main-stable\n container_name: litellm\n restart: unless-stopped\n depends_on:\n litellm-db:\n condition: service_healthy\n volumes:\n - /opt/docker/litellm/config.yaml:/app/config.yaml\n ports:\n - \"4000:4000\"\n networks:\n - proxy\n labels:\n - \"traefik.enable=true\"\n - \"traefik.docker.network=proxy\"\n - \"traefik.http.routers.litellm.rule=Host(`litellm.yourdomain.com`)\"\n - \"traefik.http.routers.litellm.entrypoints=websecure\"\n - \"traefik.http.routers.litellm.tls.certresolver=le\"\n - \"traefik.http.services.litellm.loadbalancer.server.port=4000\"\n environment:\n LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}\n DATABASE_URL: ${DATABASE_URL}\n OPENAI_API_KEY: ${OPENAI_API_KEY}\n ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}\n GOOGLE_API_KEY: ${GOOGLE_API_KEY}\n DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY}\n FIRECRAWL_API_KEY: ${FIRECRAWL_API_KEY}\n BRAVE_API_KEY: ${BRAVE_API_KEY}\n command: [\"--config\", \"/app/config.yaml\", \"--port\", \"4000\"]\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n\nnetworks:\n proxy:\n external: true\n```\n\n## Reference .env\n\n```env\nLITELLM_MASTER_KEY=sk-your-master-key\nPOSTGRES_PASSWORD=your-postgres-password\nDATABASE_URL=postgresql://litellm:your-postgres-password@litellm-db:5432/litellm\nOPENAI_API_KEY=your-openai-api-key\nANTHROPIC_API_KEY=your-anthropic-api-key\nGOOGLE_API_KEY=your-google-api-key\nDEEPSEEK_API_KEY=your-deepseek-api-key\nFIRECRAWL_API_KEY=your-firecrawl-api-key\nBRAVE_API_KEY=your-brave-api-key\n```\n\n## Reference config.yaml\n\nCreate at `/opt/docker/litellm/config.yaml`:\n\n```yaml\nmodel_list:\n # Ollama — Google\n - model_name: gemma4:e4b\n litellm_params:\n model: ollama/gemma4:e4b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: true\n supports_function_calling: false\n\n # Ollama — Qwen (Alibaba)\n - model_name: qwen3:14b\n litellm_params:\n model: ollama/qwen3:14b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n # APIs cloud\n # OpenAI\n - model_name: gpt-5.4\n litellm_params:\n model: openai/gpt-5.4\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-4o-mini-tts\n litellm_params:\n model: openai/gpt-4o-mini-tts\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-4o-mini-transcribe\n litellm_params:\n model: openai/gpt-4o-mini-transcribe\n api_key: os.environ/OPENAI_API_KEY\n\n # Anthropic\n - model_name: claude-opus-4-6\n litellm_params:\n model: anthropic/claude-opus-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n - model_name: claude-sonnet-4-6\n litellm_params:\n model: anthropic/claude-sonnet-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n # Google\n - model_name: gemini-3.1-pro-preview\n litellm_params:\n model: gemini/gemini-3.1-pro-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n - model_name: gemini-3-pro-image-preview\n litellm_params:\n model: gemini/gemini-3-pro-image-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n # DeepSeek\n - model_name: deepseek-chat\n litellm_params:\n model: deepseek/deepseek-chat\n api_key: os.environ/DEEPSEEK_API_KEY\n\nsearch_tools:\n - search_tool_name: brave-search\n litellm_params:\n search_provider: brave\n api_key: os.environ/BRAVE_API_KEY\n\n - search_tool_name: firecrawl-search\n litellm_params:\n search_provider: firecrawl\n api_key: os.environ/FIRECRAWL_API_KEY\n\n - search_tool_name: searxng-search\n litellm_params:\n search_provider: searxng\n api_base: https://serxng-deployment-production.up.railway.app\n\ngeneral_settings:\n master_key: os.environ/LITELLM_MASTER_KEY\n database_url: os.environ/DATABASE_URL\n```\n\n## Operational notes\n\nPrepare the data directory:\n\n```bash\nsudo mkdir -p /opt/docker/litellm\nsudo chown -R 1000:1000 /opt/docker/litellm\n```\n\nCreate the configuration file:\n\n```bash\nsudo nano /opt/docker/litellm/config.yaml\n```\n\nStart the service:\n\n```bash\ndocker compose up -d\n```\n\nVerify the proxy is running:\n\n```bash\ncurl http://localhost:4000/health\n```\n\nList available models:\n\n```bash\ncurl http://localhost:4000/models \\\n -H \"Authorization: Bearer sk-your-master-key\"\n```\n\nTest a chat completion against a local model:\n\n```bash\ncurl http://localhost:4000/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer sk-your-master-key\" \\\n -d '{\n \"model\": \"qwen3:14b\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]\n }'\n```\n\nInternal DNS (e.g. Pi-hole):\n\n- `litellm.yourdomain.com` → `192.168.x.x`\n\n## Notes\n\n- `LITELLM_MASTER_KEY` must start with `sk-`. Generate one with:\n\n```bash\necho \"sk-$(openssl rand -hex 32)\"\n```\n\n It is used in the `Authorization` header for every API call:\n\n```bash\nAuthorization: Bearer sk-xxxxxxxx...\n```\n\nAny client (app, n8n, curl, etc.) must include it to use LiteLLM.\n\n- `host.docker.internal` resolves to the host IP via `extra_hosts`, required for LiteLLM to reach Ollama running on the host.\n- The `litellm-database` image includes Postgres support for virtual keys and cost tracking. Use `litellm:main-stable` if you don't need a database.\n- The admin dashboard is available at `https://litellm.yourdomain.com/ui`.\n- Any OpenAI-compatible client can point to LiteLLM by changing only the `base_url` and `api_key`."}, "4c3a29924ca4d8ba3cf94a57e492ae940b4bf901": {"t": 1785765106.9888654, "n": 8249, "h": "a9cfdf238ebe8f1a", "c": "# Running LiteLLM with Docker\n\nThis guide shows how to self-host [LiteLLM](https://docs.litellm.ai) as an OpenAI-compatible proxy using Docker Compose so you can use it as a backend with OpenClient LLM. It covers the full stack: Postgres for persistence, Traefik as a reverse proxy, a reference `config.yaml` with both local (Ollama) and cloud models, and common operational commands.\n\n## Reference\n\n- https://docs.litellm.ai/docs/proxy/deploy\n- https://docs.litellm.ai/docs/proxy/configs\n- https://github.com/BerriAI/litellm\n\n## docker-compose.yml\n\n```yaml\nservices:\n litellm-db:\n image: postgres:16-alpine\n container_name: litellm-db\n restart: unless-stopped\n environment:\n POSTGRES_USER: litellm\n POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}\n POSTGRES_DB: litellm\n volumes:\n - /opt/docker/litellm/pgdata:/var/lib/postgresql/data\n networks:\n - proxy\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U litellm -d litellm\"]\n interval: 10s\n timeout: 5s\n retries: 5\n\n litellm:\n image: docker.litellm.ai/berriai/litellm-database:main-stable\n container_name: litellm\n restart: unless-stopped\n depends_on:\n litellm-db:\n condition: service_healthy\n volumes:\n - /opt/docker/litellm/config.yaml:/app/config.yaml\n ports:\n - \"4000:4000\"\n networks:\n - proxy\n labels:\n - \"traefik.enable=true\"\n - \"traefik.docker.network=proxy\"\n - \"traefik.http.routers.litellm.rule=Host(`litellm.yourdomain.com`)\"\n - \"traefik.http.routers.litellm.entrypoints=websecure\"\n - \"traefik.http.routers.litellm.tls.certresolver=le\"\n - \"traefik.http.services.litellm.loadbalancer.server.port=4000\"\n environment:\n LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}\n DATABASE_URL: ${DATABASE_URL}\n OPENAI_API_KEY: ${OPENAI_API_KEY}\n ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}\n GOOGLE_API_KEY: ${GOOGLE_API_KEY}\n DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY}\n FIRECRAWL_API_KEY: ${FIRECRAWL_API_KEY}\n BRAVE_API_KEY: ${BRAVE_API_KEY}\n command: [\"--config\", \"/app/config.yaml\", \"--port\", \"4000\"]\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n\nnetworks:\n proxy:\n external: true\n```\n\n## Reference .env\n\n```env\nLITELLM_MASTER_KEY=sk-your-master-key\nPOSTGRES_PASSWORD=your-postgres-password\nDATABASE_URL=postgresql://litellm:your-postgres-password@litellm-db:5432/litellm\nOPENAI_API_KEY=your-openai-api-key\nANTHROPIC_API_KEY=your-anthropic-api-key\nGOOGLE_API_KEY=your-google-api-key\nDEEPSEEK_API_KEY=your-deepseek-api-key\nFIRECRAWL_API_KEY=your-firecrawl-api-key\nBRAVE_API_KEY=your-brave-api-key\n```\n\n## Reference config.yaml\n\nCreate at `/opt/docker/litellm/config.yaml`:\n\n```yaml\nmodel_list:\n # Ollama — Google\n - model_name: gemma4:e4b\n litellm_params:\n model: ollama/gemma4:e4b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: true\n supports_function_calling: false\n\n - model_name: gemma3:12b\n litellm_params:\n model: ollama/gemma3:12b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: true\n supports_function_calling: false\n\n # Ollama — Qwen (Alibaba)\n - model_name: qwen3:14b\n litellm_params:\n model: ollama/qwen3:14b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen3.5:cloud\n litellm_params:\n model: ollama/qwen3.5:cloud\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen2.5-coder:14b\n litellm_params:\n model: ollama/qwen2.5-coder:14b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen3-coder:480b-cloud\n litellm_params:\n model: ollama/qwen3-coder:480b-cloud\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen3-coder-next:cloud\n litellm_params:\n model: ollama/qwen3-coder-next:cloud\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n # APIs cloud\n # OpenAI\n - model_name: gpt-5.4\n litellm_params:\n model: openai/gpt-5.4\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-5.4-mini\n litellm_params:\n model: openai/gpt-5.4-mini\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-5.4-nano\n litellm_params:\n model: openai/gpt-5.4-nano\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-4o-mini-tts\n litellm_params:\n model: openai/gpt-4o-mini-tts\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-4o-mini-transcribe\n litellm_params:\n model: openai/gpt-4o-mini-transcribe\n api_key: os.environ/OPENAI_API_KEY\n\n # Anthropic\n - model_name: claude-opus-4-6\n litellm_params:\n model: anthropic/claude-opus-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n - model_name: claude-sonnet-4-6\n litellm_params:\n model: anthropic/claude-sonnet-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n - model_name: claude-haiku-4-5\n litellm_params:\n model: anthropic/claude-haiku-4-5\n api_key: os.environ/ANTHROPIC_API_KEY\n\n # Google\n - model_name: gemini-3.1-pro-preview\n litellm_params:\n model: gemini/gemini-3.1-pro-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n - model_name: gemini-3-pro-image-preview\n litellm_params:\n model: gemini/gemini-3-pro-image-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n - model_name: gemini-2.5-pro\n litellm_params:\n model: gemini/gemini-2.5-pro\n api_key: os.environ/GOOGLE_API_KEY\n\n # DeepSeek\n - model_name: deepseek-chat\n litellm_params:\n model: deepseek/deepseek-chat\n api_key: os.environ/DEEPSEEK_API_KEY\n\n - model_name: deepseek-reasoner\n litellm_params:\n model: deepseek/deepseek-reasoner\n api_key: os.environ/DEEPSEEK_API_KEY\n\nsearch_tools:\n - search_tool_name: brave-search\n litellm_params:\n search_provider: brave\n api_key: os.environ/BRAVE_API_KEY\n\n - search_tool_name: firecrawl-search\n litellm_params:\n search_provider: firecrawl\n api_key: os.environ/FIRECRAWL_API_KEY\n\n - search_tool_name: searxng-search\n litellm_params:\n search_provider: searxng\n api_base: https://search.rhscz.eu\n\ngeneral_settings:\n master_key: os.environ/LITELLM_MASTER_KEY\n database_url: os.environ/DATABASE_URL\n```\n\n## Operational notes\n\nPrepare the data directory:\n\n```bash\nsudo mkdir -p /opt/docker/litellm\nsudo chown -R 1000:1000 /opt/docker/litellm\n```\n\nCreate the configuration file:\n\n```bash\nsudo nano /opt/docker/litellm/config.yaml\n```\n\nStart the service:\n\n```bash\ndocker compose up -d\n```\n\nVerify the proxy is running:\n\n```bash\ncurl http://localhost:4000/health\n```\n\nList available models:\n\n```bash\ncurl http://localhost:4000/models \\\n -H \"Authorization: Bearer sk-your-master-key\"\n```\n\nTest a chat completion against a local model:\n\n```bash\ncurl http://localhost:4000/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer sk-your-master-key\" \\\n -d '{\n \"model\": \"qwen3:14b\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]\n }'\n```\n\nInternal DNS (e.g. Pi-hole):\n\n- `litellm.yourdomain.com` → `192.168.x.x`\n\n## Notes\n\n- `LITELLM_MASTER_KEY` must start with `sk-`. Generate one with:\n\n```bash\necho \"sk-$(openssl rand -hex 32)\"\n```\n\n It is used in the `Authorization` header for every API call:\n\n```bash\nAuthorization: Bearer sk-xxxxxxxx...\n```\n\n Any client (app, n8n, curl, etc.) must include it to use LiteLLM.\n\n- `host.docker.internal` resolves to the host IP via `extra_hosts`, required for LiteLLM to reach Ollama running on the host.\n- The `litellm-database` image includes Postgres support for virtual keys and cost tracking. Use `litellm:main-stable` if you don't need a database.\n- The admin dashboard is available at `https://litellm.yourdomain.com/ui`.\n- Any OpenAI-compatible client can point to LiteLLM by changing only the `base_url` and `api_key`."}, "8936644734c2ee68fb093c313f6a09e98630c427": {"t": 1785765109.0014434, "n": 8053, "h": "886b7d3196d3c01c", "c": "# Running LiteLLM with Docker\n\nThis guide shows how to self-host [LiteLLM](https://docs.litellm.ai) as an OpenAI-compatible proxy using Docker Compose so you can use it as a backend with OpenClient LLM. It covers the full stack: Postgres for persistence, Traefik as a reverse proxy, a reference `config.yaml` with both local (Ollama) and cloud models, and common operational commands.\n\n## Reference\n\n- https://docs.litellm.ai/docs/proxy/deploy\n- https://docs.litellm.ai/docs/proxy/configs\n- https://github.com/BerriAI/litellm\n\n## docker-compose.yml\n\n```yaml\nservices:\n litellm-db:\n image: postgres:16-alpine\n container_name: litellm-db\n restart: unless-stopped\n environment:\n POSTGRES_USER: litellm\n POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}\n POSTGRES_DB: litellm\n volumes:\n - /opt/docker/litellm/pgdata:/var/lib/postgresql/data\n networks:\n - proxy\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U litellm -d litellm\"]\n interval: 10s\n timeout: 5s\n retries: 5\n\n litellm:\n image: docker.litellm.ai/berriai/litellm-database:main-stable\n container_name: litellm\n restart: unless-stopped\n depends_on:\n litellm-db:\n condition: service_healthy\n volumes:\n - /opt/docker/litellm/config.yaml:/app/config.yaml\n ports:\n - \"4000:4000\"\n networks:\n - proxy\n labels:\n - \"traefik.enable=true\"\n - \"traefik.docker.network=proxy\"\n - \"traefik.http.routers.litellm.rule=Host(`litellm.yourdomain.com`)\"\n - \"traefik.http.routers.litellm.entrypoints=websecure\"\n - \"traefik.http.routers.litellm.tls.certresolver=le\"\n - \"traefik.http.services.litellm.loadbalancer.server.port=4000\"\n environment:\n LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}\n DATABASE_URL: ${DATABASE_URL}\n OPENAI_API_KEY: ${OPENAI_API_KEY}\n ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}\n GOOGLE_API_KEY: ${GOOGLE_API_KEY}\n DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY}\n FIRECRAWL_API_KEY: ${FIRECRAWL_API_KEY}\n command: [\"--config\", \"/app/config.yaml\", \"--port\", \"4000\"]\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n\nnetworks:\n proxy:\n external: true\n```\n\n## Reference .env\n\n```env\nLITELLM_MASTER_KEY=sk-your-master-key\nPOSTGRES_PASSWORD=your-postgres-password\nDATABASE_URL=postgresql://litellm:your-postgres-password@litellm-db:5432/litellm\nOPENAI_API_KEY=your-openai-api-key\nANTHROPIC_API_KEY=your-anthropic-api-key\nGOOGLE_API_KEY=your-google-api-key\nDEEPSEEK_API_KEY=your-deepseek-api-key\nFIRECRAWL_API_KEY=your-firecrawl-api-key\n```\n\n## Reference config.yaml\n\nCreate at `/opt/docker/litellm/config.yaml`:\n\n```yaml\nmodel_list:\n # Ollama — Google\n - model_name: gemma4:e4b\n litellm_params:\n model: ollama/gemma4:e4b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: true\n supports_function_calling: false\n\n - model_name: gemma3:12b\n litellm_params:\n model: ollama/gemma3:12b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: true\n supports_function_calling: false\n\n # Ollama — Qwen (Alibaba)\n - model_name: qwen3:14b\n litellm_params:\n model: ollama/qwen3:14b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen3.5:cloud\n litellm_params:\n model: ollama/qwen3.5:cloud\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen2.5-coder:14b\n litellm_params:\n model: ollama/qwen2.5-coder:14b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen3-coder:480b-cloud\n litellm_params:\n model: ollama/qwen3-coder:480b-cloud\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen3-coder-next:cloud\n litellm_params:\n model: ollama/qwen3-coder-next:cloud\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n # APIs cloud\n # OpenAI\n - model_name: gpt-5.4\n litellm_params:\n model: openai/gpt-5.4\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-5.4-mini\n litellm_params:\n model: openai/gpt-5.4-mini\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-5.4-nano\n litellm_params:\n model: openai/gpt-5.4-nano\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-4o-mini-tts\n litellm_params:\n model: openai/gpt-4o-mini-tts\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-4o-mini-transcribe\n litellm_params:\n model: openai/gpt-4o-mini-transcribe\n api_key: os.environ/OPENAI_API_KEY\n\n # Anthropic\n - model_name: claude-opus-4-6\n litellm_params:\n model: anthropic/claude-opus-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n - model_name: claude-sonnet-4-6\n litellm_params:\n model: anthropic/claude-sonnet-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n - model_name: claude-haiku-4-5\n litellm_params:\n model: anthropic/claude-haiku-4-5\n api_key: os.environ/ANTHROPIC_API_KEY\n\n # Google\n - model_name: gemini-3.1-pro-preview\n litellm_params:\n model: gemini/gemini-3.1-pro-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n - model_name: gemini-3-pro-image-preview\n litellm_params:\n model: gemini/gemini-3-pro-image-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n - model_name: gemini-2.5-pro\n litellm_params:\n model: gemini/gemini-2.5-pro\n api_key: os.environ/GOOGLE_API_KEY\n\n # DeepSeek\n - model_name: deepseek-chat\n litellm_params:\n model: deepseek/deepseek-chat\n api_key: os.environ/DEEPSEEK_API_KEY\n\n - model_name: deepseek-reasoner\n litellm_params:\n model: deepseek/deepseek-reasoner\n api_key: os.environ/DEEPSEEK_API_KEY\n\nsearch_tools:\n - search_tool_name: firecrawl-search\n litellm_params:\n search_provider: firecrawl\n api_key: os.environ/FIRECRAWL_API_KEY\n\n - search_tool_name: searxng-search\n litellm_params:\n search_provider: searxng\n api_base: https://search.rhscz.eu\n\ngeneral_settings:\n master_key: os.environ/LITELLM_MASTER_KEY\n database_url: os.environ/DATABASE_URL\n```\n\n## Operational notes\n\nPrepare the data directory:\n\n```bash\nsudo mkdir -p /opt/docker/litellm\nsudo chown -R 1000:1000 /opt/docker/litellm\n```\n\nCreate the configuration file:\n\n```bash\nsudo nano /opt/docker/litellm/config.yaml\n```\n\nStart the service:\n\n```bash\ndocker compose up -d\n```\n\nVerify the proxy is running:\n\n```bash\ncurl http://localhost:4000/health\n```\n\nList available models:\n\n```bash\ncurl http://localhost:4000/models \\\n -H \"Authorization: Bearer sk-your-master-key\"\n```\n\nTest a chat completion against a local model:\n\n```bash\ncurl http://localhost:4000/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer sk-your-master-key\" \\\n -d '{\n \"model\": \"qwen3:14b\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]\n }'\n```\n\nInternal DNS (e.g. Pi-hole):\n\n- `litellm.yourdomain.com` → `192.168.x.x`\n\n## Notes\n\n- `LITELLM_MASTER_KEY` must start with `sk-`. Generate one with:\n\n```bash\necho \"sk-$(openssl rand -hex 32)\"\n```\n\n It is used in the `Authorization` header for every API call:\n\n```bash\nAuthorization: Bearer sk-xxxxxxxx...\n```\n\n Any client (app, n8n, curl, etc.) must include it to use LiteLLM.\n\n- `host.docker.internal` resolves to the host IP via `extra_hosts`, required for LiteLLM to reach Ollama running on the host.\n- The `litellm-database` image includes Postgres support for virtual keys and cost tracking. Use `litellm:main-stable` if you don't need a database.\n- The admin dashboard is available at `https://litellm.yourdomain.com/ui`.\n- Any OpenAI-compatible client can point to LiteLLM by changing only the `base_url` and `api_key`."}, "fbb46a8d29aad9fe4bfb7a79fcb4567f0ff3e92f": {"t": 1785765132.9753869, "n": 7773, "h": "11e2016608d068e9", "c": "# Running LiteLLM with Docker\n\nThis guide shows how to self-host [LiteLLM](https://docs.litellm.ai) as an OpenAI-compatible proxy using Docker Compose so you can use it as a backend with OpenClient LLM. It covers the full stack: Postgres for persistence, Traefik as a reverse proxy, a reference `config.yaml` with both local (Ollama) and cloud models, and common operational commands.\n\n## Reference\n\n- https://docs.litellm.ai/docs/proxy/deploy\n- https://docs.litellm.ai/docs/proxy/configs\n- https://github.com/BerriAI/litellm\n\n## docker-compose.yml\n\n```yaml\nservices:\n litellm-db:\n image: postgres:16-alpine\n container_name: litellm-db\n restart: unless-stopped\n environment:\n POSTGRES_USER: litellm\n POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}\n POSTGRES_DB: litellm\n volumes:\n - /opt/docker/litellm/pgdata:/var/lib/postgresql/data\n networks:\n - proxy\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U litellm -d litellm\"]\n interval: 10s\n timeout: 5s\n retries: 5\n\n litellm:\n image: docker.litellm.ai/berriai/litellm-database:main-stable\n container_name: litellm\n restart: unless-stopped\n depends_on:\n litellm-db:\n condition: service_healthy\n volumes:\n - /opt/docker/litellm/config.yaml:/app/config.yaml\n ports:\n - \"4000:4000\"\n networks:\n - proxy\n labels:\n - \"traefik.enable=true\"\n - \"traefik.docker.network=proxy\"\n - \"traefik.http.routers.litellm.rule=Host(`litellm.yourdomain.com`)\"\n - \"traefik.http.routers.litellm.entrypoints=websecure\"\n - \"traefik.http.routers.litellm.tls.certresolver=le\"\n - \"traefik.http.services.litellm.loadbalancer.server.port=4000\"\n environment:\n LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}\n DATABASE_URL: ${DATABASE_URL}\n OPENAI_API_KEY: ${OPENAI_API_KEY}\n ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}\n GOOGLE_API_KEY: ${GOOGLE_API_KEY}\n DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY}\n FIRECRAWL_API_KEY: ${FIRECRAWL_API_KEY}\n command: [\"--config\", \"/app/config.yaml\", \"--port\", \"4000\"]\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n\nnetworks:\n proxy:\n external: true\n```\n\n## Reference .env\n\n```env\nLITELLM_MASTER_KEY=sk-your-master-key\nPOSTGRES_PASSWORD=your-postgres-password\nDATABASE_URL=postgresql://litellm:your-postgres-password@litellm-db:5432/litellm\nOPENAI_API_KEY=your-openai-api-key\nANTHROPIC_API_KEY=your-anthropic-api-key\nGOOGLE_API_KEY=your-google-api-key\nDEEPSEEK_API_KEY=your-deepseek-api-key\nFIRECRAWL_API_KEY=your-firecrawl-api-key\n```\n\n## Reference config.yaml\n\nCreate at `/opt/docker/litellm/config.yaml`:\n\n```yaml\nmodel_list:\n # Ollama — Google\n - model_name: gemma4:e4b\n litellm_params:\n model: ollama/gemma4:e4b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: true\n supports_function_calling: false\n\n - model_name: gemma3:12b\n litellm_params:\n model: ollama/gemma3:12b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: true\n supports_function_calling: false\n\n # Ollama — Qwen (Alibaba)\n - model_name: qwen3:14b\n litellm_params:\n model: ollama/qwen3:14b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen3.5:cloud\n litellm_params:\n model: ollama/qwen3.5:cloud\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen2.5-coder:14b\n litellm_params:\n model: ollama/qwen2.5-coder:14b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen3-coder:480b-cloud\n litellm_params:\n model: ollama/qwen3-coder:480b-cloud\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen3-coder-next:cloud\n litellm_params:\n model: ollama/qwen3-coder-next:cloud\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n # APIs cloud\n # OpenAI\n - model_name: gpt-5.4\n litellm_params:\n model: openai/gpt-5.4\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-5.4-mini\n litellm_params:\n model: openai/gpt-5.4-mini\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-5.4-nano\n litellm_params:\n model: openai/gpt-5.4-nano\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-4o-mini-tts\n litellm_params:\n model: openai/gpt-4o-mini-tts\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-4o-mini-transcribe\n litellm_params:\n model: openai/gpt-4o-mini-transcribe\n api_key: os.environ/OPENAI_API_KEY\n\n # Anthropic\n - model_name: claude-opus-4-6\n litellm_params:\n model: anthropic/claude-opus-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n - model_name: claude-sonnet-4-6\n litellm_params:\n model: anthropic/claude-sonnet-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n - model_name: claude-haiku-4-5\n litellm_params:\n model: anthropic/claude-haiku-4-5\n api_key: os.environ/ANTHROPIC_API_KEY\n\n # Google\n - model_name: gemini-3.1-pro-preview\n litellm_params:\n model: gemini/gemini-3.1-pro-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n - model_name: gemini-3-pro-image-preview\n litellm_params:\n model: gemini/gemini-3-pro-image-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n - model_name: gemini-2.5-pro\n litellm_params:\n model: gemini/gemini-2.5-pro\n api_key: os.environ/GOOGLE_API_KEY\n\n # DeepSeek\n - model_name: deepseek-chat\n litellm_params:\n model: deepseek/deepseek-chat\n api_key: os.environ/DEEPSEEK_API_KEY\n\n - model_name: deepseek-reasoner\n litellm_params:\n model: deepseek/deepseek-reasoner\n api_key: os.environ/DEEPSEEK_API_KEY\n\ngeneral_settings:\n master_key: os.environ/LITELLM_MASTER_KEY\n database_url: os.environ/DATABASE_URL\n```\n\n## Operational notes\n\nPrepare the data directory:\n\n```bash\nsudo mkdir -p /opt/docker/litellm\nsudo chown -R 1000:1000 /opt/docker/litellm\n```\n\nCreate the configuration file:\n\n```bash\nsudo nano /opt/docker/litellm/config.yaml\n```\n\nStart the service:\n\n```bash\ndocker compose up -d\n```\n\nVerify the proxy is running:\n\n```bash\ncurl http://localhost:4000/health\n```\n\nList available models:\n\n```bash\ncurl http://localhost:4000/models \\\n -H \"Authorization: Bearer sk-your-master-key\"\n```\n\nTest a chat completion against a local model:\n\n```bash\ncurl http://localhost:4000/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer sk-your-master-key\" \\\n -d '{\n \"model\": \"qwen3:14b\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]\n }'\n```\n\nInternal DNS (e.g. Pi-hole):\n\n- `litellm.yourdomain.com` → `192.168.x.x`\n\n## Notes\n\n- `LITELLM_MASTER_KEY` must start with `sk-`. Generate one with:\n\n```bash\necho \"sk-$(openssl rand -hex 32)\"\n```\n\n It is used in the `Authorization` header for every API call:\n\n```bash\nAuthorization: Bearer sk-xxxxxxxx...\n```\n\n Any client (app, n8n, curl, etc.) must include it to use LiteLLM.\n\n- `host.docker.internal` resolves to the host IP via `extra_hosts`, required for LiteLLM to reach Ollama running on the host.\n- The `litellm-database` image includes Postgres support for virtual keys and cost tracking. Use `litellm:main-stable` if you don't need a database.\n- The admin dashboard is available at `https://litellm.yourdomain.com/ui`.\n- Any OpenAI-compatible client can point to LiteLLM by changing only the `base_url` and `api_key`."}, "5d054bcdc11cb7c1a3ac436da7956525bc36f584": {"t": 1785765134.4697654, "n": 7412, "h": "29b10bd1fdb1346f", "c": "# Running LiteLLM with Docker\n\nThis guide shows how to self-host [LiteLLM](https://docs.litellm.ai) as an OpenAI-compatible proxy using Docker Compose so you can use it as a backend with OpenClient LLM. It covers the full stack: Postgres for persistence, Traefik as a reverse proxy, a reference `config.yaml` with both local (Ollama) and cloud models, and common operational commands.\n\n## Reference\n\n- https://docs.litellm.ai/docs/proxy/deploy\n- https://docs.litellm.ai/docs/proxy/configs\n- https://github.com/BerriAI/litellm\n\n## docker-compose.yml\n\n```yaml\nservices:\n litellm-db:\n image: postgres:16-alpine\n container_name: litellm-db\n restart: unless-stopped\n environment:\n POSTGRES_USER: litellm\n POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}\n POSTGRES_DB: litellm\n volumes:\n - /opt/docker/litellm/pgdata:/var/lib/postgresql/data\n networks:\n - proxy\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U litellm -d litellm\"]\n interval: 10s\n timeout: 5s\n retries: 5\n\n litellm:\n image: docker.litellm.ai/berriai/litellm-database:main-stable\n container_name: litellm\n restart: unless-stopped\n depends_on:\n litellm-db:\n condition: service_healthy\n volumes:\n - /opt/docker/litellm/config.yaml:/app/config.yaml\n ports:\n - \"4000:4000\"\n networks:\n - proxy\n labels:\n - \"traefik.enable=true\"\n - \"traefik.docker.network=proxy\"\n - \"traefik.http.routers.litellm.rule=Host(`litellm.yourdomain.com`)\"\n - \"traefik.http.routers.litellm.entrypoints=websecure\"\n - \"traefik.http.routers.litellm.tls.certresolver=le\"\n - \"traefik.http.services.litellm.loadbalancer.server.port=4000\"\n environment:\n LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}\n DATABASE_URL: ${DATABASE_URL}\n OPENAI_API_KEY: ${OPENAI_API_KEY}\n ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}\n GOOGLE_API_KEY: ${GOOGLE_API_KEY}\n DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY}\n command: [\"--config\", \"/app/config.yaml\", \"--port\", \"4000\"]\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n\nnetworks:\n proxy:\n external: true\n```\n\n## Reference .env\n\n```env\nLITELLM_MASTER_KEY=sk-your-master-key\nPOSTGRES_PASSWORD=your-postgres-password\nDATABASE_URL=postgresql://litellm:your-postgres-password@litellm-db:5432/litellm\nOPENAI_API_KEY=your-openai-api-key\nANTHROPIC_API_KEY=your-anthropic-api-key\nGOOGLE_API_KEY=your-google-api-key\nDEEPSEEK_API_KEY=your-deepseek-api-key\n```\n\n## Reference config.yaml\n\nCreate at `/opt/docker/litellm/config.yaml`:\n\n```yaml\nmodel_list:\n # Ollama — Google\n - model_name: gemma4:e4b\n litellm_params:\n model: ollama/gemma4:e4b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: true\n supports_function_calling: false\n\n - model_name: gemma3:12b\n litellm_params:\n model: ollama/gemma3:12b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: true\n supports_function_calling: false\n\n # Ollama — Qwen (Alibaba)\n - model_name: qwen3:14b\n litellm_params:\n model: ollama/qwen3:14b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen3.5:cloud\n litellm_params:\n model: ollama/qwen3.5:cloud\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen2.5-coder:14b\n litellm_params:\n model: ollama/qwen2.5-coder:14b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen3-coder:480b-cloud\n litellm_params:\n model: ollama/qwen3-coder:480b-cloud\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen3-coder-next:cloud\n litellm_params:\n model: ollama/qwen3-coder-next:cloud\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n # APIs cloud\n # OpenAI\n - model_name: gpt-5.4\n litellm_params:\n model: openai/gpt-5.4\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-5.4-mini\n litellm_params:\n model: openai/gpt-5.4-mini\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: gpt-5.4-nano\n litellm_params:\n model: openai/gpt-5.4-nano\n api_key: os.environ/OPENAI_API_KEY\n\n # Anthropic\n - model_name: claude-opus-4-6\n litellm_params:\n model: anthropic/claude-opus-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n - model_name: claude-sonnet-4-6\n litellm_params:\n model: anthropic/claude-sonnet-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n - model_name: claude-haiku-4-5\n litellm_params:\n model: anthropic/claude-haiku-4-5\n api_key: os.environ/ANTHROPIC_API_KEY\n\n # Google\n - model_name: gemini-3.1-pro-preview\n litellm_params:\n model: gemini/gemini-3.1-pro-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n - model_name: gemini-3-pro-image-preview\n litellm_params:\n model: gemini/gemini-3-pro-image-preview\n api_key: os.environ/GOOGLE_API_KEY\n\n - model_name: gemini-2.5-pro\n litellm_params:\n model: gemini/gemini-2.5-pro\n api_key: os.environ/GOOGLE_API_KEY\n\n # DeepSeek\n - model_name: deepseek-chat\n litellm_params:\n model: deepseek/deepseek-chat\n api_key: os.environ/DEEPSEEK_API_KEY\n\n - model_name: deepseek-reasoner\n litellm_params:\n model: deepseek/deepseek-reasoner\n api_key: os.environ/DEEPSEEK_API_KEY\n\ngeneral_settings:\n master_key: os.environ/LITELLM_MASTER_KEY\n database_url: os.environ/DATABASE_URL\n```\n\n## Operational notes\n\nPrepare the data directory:\n\n```bash\nsudo mkdir -p /opt/docker/litellm\nsudo chown -R 1000:1000 /opt/docker/litellm\n```\n\nCreate the configuration file:\n\n```bash\nsudo nano /opt/docker/litellm/config.yaml\n```\n\nStart the service:\n\n```bash\ndocker compose up -d\n```\n\nVerify the proxy is running:\n\n```bash\ncurl http://localhost:4000/health\n```\n\nList available models:\n\n```bash\ncurl http://localhost:4000/models \\\n -H \"Authorization: Bearer sk-your-master-key\"\n```\n\nTest a chat completion against a local model:\n\n```bash\ncurl http://localhost:4000/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer sk-your-master-key\" \\\n -d '{\n \"model\": \"qwen3:14b\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]\n }'\n```\n\nInternal DNS (e.g. Pi-hole):\n\n- `litellm.yourdomain.com` → `192.168.x.x`\n\n## Notes\n\n- `LITELLM_MASTER_KEY` must start with `sk-`. Generate one with:\n\n```bash\necho \"sk-$(openssl rand -hex 32)\"\n```\n\n It is used in the `Authorization` header for every API call:\n\n```bash\nAuthorization: Bearer sk-xxxxxxxx...\n```\n\n Any client (app, n8n, curl, etc.) must include it to use LiteLLM.\n\n- `host.docker.internal` resolves to the host IP via `extra_hosts`, required for LiteLLM to reach Ollama running on the host.\n- The `litellm-database` image includes Postgres support for virtual keys and cost tracking. Use `litellm:main-stable` if you don't need a database.\n- The admin dashboard is available at `https://litellm.yourdomain.com/ui`.\n- Any OpenAI-compatible client can point to LiteLLM by changing only the `base_url` and `api_key`."}, "d69a9d2c15c0dc4599196bcaa18c46ae06be8388": {"t": 1785765136.976659, "n": 5476, "h": "3d0179a81d3d9af7", "c": "# Running LiteLLM with Docker\n\nThis guide shows how to self-host [LiteLLM](https://docs.litellm.ai) as an OpenAI-compatible proxy using Docker Compose so you can use it as a backend with OpenClient LLM. It covers the full stack: Postgres for persistence, Traefik as a reverse proxy, a reference `config.yaml` with both local (Ollama) and cloud models, and common operational commands.\n\n## Reference\n\n- https://docs.litellm.ai/docs/proxy/deploy\n- https://docs.litellm.ai/docs/proxy/configs\n- https://github.com/BerriAI/litellm\n\n## docker-compose.yml\n\n```yaml\nservices:\n litellm-db:\n image: postgres:16-alpine\n container_name: litellm-db\n restart: unless-stopped\n environment:\n POSTGRES_USER: litellm\n POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}\n POSTGRES_DB: litellm\n volumes:\n - /opt/docker/litellm/pgdata:/var/lib/postgresql/data\n networks:\n - proxy\n healthcheck:\n test: [\"CMD-SHELL\", \"pg_isready -U litellm -d litellm\"]\n interval: 10s\n timeout: 5s\n retries: 5\n\n litellm:\n image: docker.litellm.ai/berriai/litellm-database:main-stable\n container_name: litellm\n restart: unless-stopped\n depends_on:\n litellm-db:\n condition: service_healthy\n volumes:\n - /opt/docker/litellm/config.yaml:/app/config.yaml\n ports:\n - \"4000:4000\"\n networks:\n - proxy\n labels:\n - \"traefik.enable=true\"\n - \"traefik.docker.network=proxy\"\n - \"traefik.http.routers.litellm.rule=Host(`litellm.yourdomain.com`)\"\n - \"traefik.http.routers.litellm.entrypoints=websecure\"\n - \"traefik.http.routers.litellm.tls.certresolver=le\"\n - \"traefik.http.services.litellm.loadbalancer.server.port=4000\"\n environment:\n LITELLM_MASTER_KEY: ${LITELLM_MASTER_KEY}\n DATABASE_URL: ${DATABASE_URL}\n OPENAI_API_KEY: ${OPENAI_API_KEY}\n ANTHROPIC_API_KEY: ${ANTHROPIC_API_KEY}\n GOOGLE_API_KEY: ${GOOGLE_API_KEY}\n DEEPSEEK_API_KEY: ${DEEPSEEK_API_KEY}\n command: [\"--config\", \"/app/config.yaml\", \"--port\", \"4000\"]\n extra_hosts:\n - \"host.docker.internal:host-gateway\"\n\nnetworks:\n proxy:\n external: true\n```\n\n## Reference .env\n\n```env\nLITELLM_MASTER_KEY=sk-your-master-key\nPOSTGRES_PASSWORD=your-postgres-password\nDATABASE_URL=postgresql://litellm:your-postgres-password@litellm-db:5432/litellm\nOPENAI_API_KEY=your-openai-api-key\nANTHROPIC_API_KEY=your-anthropic-api-key\nGOOGLE_API_KEY=your-google-api-key\nDEEPSEEK_API_KEY=your-deepseek-api-key\n```\n\n## Reference config.yaml\n\nCreate at `/opt/docker/litellm/config.yaml`:\n\n```yaml\nmodel_list:\n # Ollama (local)\n - model_name: qwen3:14b\n litellm_params:\n model: ollama/qwen3:14b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: qwen2.5-coder:14b\n litellm_params:\n model: ollama/qwen2.5-coder:14b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: false\n supports_function_calling: true\n\n - model_name: gemma3:12b\n litellm_params:\n model: ollama/gemma3:12b\n api_base: http://host.docker.internal:11434\n model_info:\n supports_vision: true\n supports_function_calling: false\n\n # Cloud APIs\n - model_name: gpt-5.4\n litellm_params:\n model: openai/gpt-5.4\n api_key: os.environ/OPENAI_API_KEY\n\n - model_name: claude-sonnet-4-6\n litellm_params:\n model: anthropic/claude-sonnet-4-6\n api_key: os.environ/ANTHROPIC_API_KEY\n\n - model_name: gemini-2.5-pro\n litellm_params:\n model: gemini/gemini-2.5-pro\n api_key: os.environ/GOOGLE_API_KEY\n\n - model_name: deepseek-chat\n litellm_params:\n model: deepseek/deepseek-chat\n api_key: os.environ/DEEPSEEK_API_KEY\n\ngeneral_settings:\n master_key: os.environ/LITELLM_MASTER_KEY\n database_url: os.environ/DATABASE_URL\n```\n\n## Operational notes\n\nPrepare the data directory:\n\n```bash\nsudo mkdir -p /opt/docker/litellm\nsudo chown -R 1000:1000 /opt/docker/litellm\n```\n\nCreate the configuration file:\n\n```bash\nsudo nano /opt/docker/litellm/config.yaml\n```\n\nStart the service:\n\n```bash\ndocker compose up -d\n```\n\nVerify the proxy is running:\n\n```bash\ncurl http://localhost:4000/health\n```\n\nList available models:\n\n```bash\ncurl http://localhost:4000/models \\\n -H \"Authorization: Bearer sk-your-master-key\"\n```\n\nTest a chat completion against a local model:\n\n```bash\ncurl http://localhost:4000/chat/completions \\\n -H \"Content-Type: application/json\" \\\n -H \"Authorization: Bearer sk-your-master-key\" \\\n -d '{\n \"model\": \"qwen3:14b\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello\"}]\n }'\n```\n\nInternal DNS (e.g. Pi-hole):\n\n- `litellm.yourdomain.com` → `192.168.x.x`\n\n## Notes\n\n- `LITELLM_MASTER_KEY` must start with `sk-`. Generate one with:\n\n```bash\necho \"sk-$(openssl rand -hex 32)\"\n```\n\n It is used in the `Authorization` header for every API call:\n\n```bash\nAuthorization: Bearer sk-xxxxxxxx...\n```\n\n Any client (app, n8n, curl, etc.) must include it to use LiteLLM.\n\n- `host.docker.internal` resolves to the host IP via `extra_hosts`, required for LiteLLM to reach Ollama running on the host.\n- The `litellm-database` image includes Postgres support for virtual keys and cost tracking. Use `litellm:main-stable` if you don't need a database.\n- The admin dashboard is available at `https://litellm.yourdomain.com/ui`.\n- Any OpenAI-compatible client can point to LiteLLM by changing only the `base_url` and `api_key`."}}}} |