1 line
35 KiB
JSON
1 line
35 KiB
JSON
{"repo": "akashkesav/aethercloud", "files": {"docs/features/ai-gateway/page.mdx": {"0d89657d43c9361d57ff2f20b40fdd038b5183ee": {"t": 1785746550.8197122, "n": 16850, "h": "9f251ac558958a64", "c": "export const metadata = {\n title: 'AI Gateway',\n description: 'OpenAI-compatible AI gateway built into Temps. Route requests to OpenAI, Anthropic, Gemini, and more with centralized API key management.',\n alternates: {\n canonical: '/docs/ai-gateway',\n },\n}\n\nexport const sections = [\n { title: 'Overview', id: 'overview' },\n { title: 'Quick Start', id: 'quick-start' },\n { title: 'Supported Providers', id: 'providers' },\n { title: 'Chat Completions', id: 'chat-completions' },\n { title: 'Vision & Image Attachments', id: 'vision' },\n { title: 'Tool Calling (Function Calling)', id: 'tool-calling' },\n { title: 'Streaming', id: 'streaming' },\n { title: 'Embeddings', id: 'embeddings' },\n { title: 'Provider Key Management', id: 'key-management' },\n { title: 'Usage & Analytics', id: 'usage' },\n]\n\n# AI Gateway\n\nOpenAI-compatible AI gateway built into Temps. Use any OpenAI SDK to call OpenAI, Anthropic, Gemini, xAI, Mistral, and DeepSeek — just change the base URL. {{ className: 'lead' }}\n\n---\n\n## Overview {{ anchor: true, id: 'overview' }}\n\nThe AI Gateway provides a unified, OpenAI-compatible API for all major LLM providers. Your existing OpenAI SDK code works unchanged — just point it at your Temps instance.\n\n<Row>\n <Col>\n **What's Included**\n - OpenAI API wire-compatible endpoints\n - Multi-provider routing (OpenAI, Anthropic, Gemini, xAI, Mistral, DeepSeek)\n - Tool calling / function calling across all providers\n - Vision / image attachments (base64 and URL)\n - SSE streaming with consistent format\n - Centralized API key management\n - Per-user usage tracking and analytics\n - JSON mode / structured output\n </Col>\n <Col>\n **Why It Matters**\n - Zero code changes — works with any OpenAI SDK\n - Centralized key management (devs never see API keys)\n - Switch providers without changing client code\n - Built-in usage analytics and cost tracking\n - Self-hosted — your data stays on your infrastructure\n - No per-request fees beyond provider costs\n - Transparent translation for Anthropic and Gemini\n </Col>\n</Row>\n\n### Key Features\n\n<Properties>\n <Property name=\"Drop-In Replacement\">\n 100% OpenAI SDK compatible. Works with Python, Node.js, Go, Rust, and any other OpenAI client library.\n </Property>\n <Property name=\"Transparent Translation\">\n Automatically translates requests/responses for Anthropic Messages API and Gemini generateContent API.\n </Property>\n <Property name=\"Vision Support\">\n Send images as base64 data URIs or HTTP URLs. Translated to each provider's native format automatically.\n </Property>\n <Property name=\"Tool Calling\">\n OpenAI-format tools work across all providers. Function definitions, tool_choice, and tool results are translated transparently.\n </Property>\n <Property name=\"Streaming\">\n Server-Sent Events (SSE) streaming with consistent `data: {...}\\n\\n` format across all providers.\n </Property>\n <Property name=\"Usage Analytics\">\n Track token usage, costs, and request counts per user, model, and provider.\n </Property>\n</Properties>\n\n---\n\n## Quick Start {{ anchor: true, id: 'quick-start' }}\n\nPoint any OpenAI SDK at your Temps instance. The only changes are `base_url` and `api_key`.\n\n<CodeGroup title=\"Setup\">\n\n```python {{ title: 'Python' }}\nfrom openai import OpenAI\n\nclient = OpenAI(\n base_url=\"https://your-temps-instance.com/ai/v1\",\n api_key=\"your-temps-api-key\",\n)\n\nresponse = client.chat.completions.create(\n model=\"gpt-4o\", # or \"claude-sonnet-4-20250514\", \"gemini-2.5-pro\", etc.\n messages=[\n {\"role\": \"user\", \"content\": \"Hello!\"}\n ],\n)\n\nprint(response.choices[0].message.content)\n```\n\n```typescript {{ title: 'Node.js' }}\nimport OpenAI from \"openai\";\n\nconst client = new OpenAI({\n baseURL: \"https://your-temps-instance.com/ai/v1\",\n apiKey: \"your-temps-api-key\",\n});\n\nconst response = await client.chat.completions.create({\n model: \"gpt-4o\", // or \"claude-sonnet-4-20250514\", \"gemini-2.5-pro\", etc.\n messages: [\n { role: \"user\", content: \"Hello!\" }\n ],\n});\n\nconsole.log(response.choices[0].message.content);\n```\n\n```bash {{ title: 'cURL' }}\ncurl https://your-temps-instance.com/ai/v1/chat/completions \\\n -H \"Authorization: Bearer your-temps-api-key\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"model\": \"gpt-4o\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]\n }'\n```\n\n</CodeGroup>\n\n---\n\n## Supported Providers {{ anchor: true, id: 'providers' }}\n\nThe gateway routes requests to the correct provider based on the model name:\n\n| Model Prefix | Provider | Examples |\n|---|---|---|\n| `gpt-*`, `o1-*`, `o3-*` | OpenAI | `gpt-4o`, `gpt-4o-mini`, `o3-mini` |\n| `claude-*` | Anthropic | `claude-sonnet-4-20250514`, `claude-haiku-4-5-20251001` |\n| `gemini-*` | Google Gemini | `gemini-2.5-pro`, `gemini-2.5-flash` |\n| `grok-*` | xAI | `grok-3`, `grok-3-mini` |\n| `mistral-*`, `codestral-*` | Mistral | `mistral-large-latest`, `codestral-latest` |\n| `deepseek-*` | DeepSeek | `deepseek-chat`, `deepseek-reasoner` |\n\n<Note>\n All providers use the same OpenAI-compatible API format. The gateway handles protocol translation automatically for Anthropic and Gemini.\n</Note>\n\n---\n\n## Chat Completions {{ anchor: true, id: 'chat-completions' }}\n\nStandard chat completions work identically to the OpenAI API.\n\n```python {{ title: 'Multi-turn conversation' }}\nresponse = client.chat.completions.create(\n model=\"claude-sonnet-4-20250514\",\n messages=[\n {\"role\": \"system\", \"content\": \"You are a helpful coding assistant.\"},\n {\"role\": \"user\", \"content\": \"Write a Python function to calculate fibonacci numbers.\"},\n {\"role\": \"assistant\", \"content\": \"Here's a fibonacci function:\\n\\n```python\\ndef fib(n):\\n if n <= 1:\\n return n\\n return fib(n-1) + fib(n-2)\\n```\"},\n {\"role\": \"user\", \"content\": \"Can you make it iterative for better performance?\"},\n ],\n temperature=0.7,\n max_tokens=500,\n)\n```\n\n---\n\n## Vision & Image Attachments {{ anchor: true, id: 'vision' }}\n\nSend images to vision-capable models. The gateway translates image formats automatically for each provider.\n\n### Base64 Image\n\n<CodeGroup title=\"Send base64 image\">\n\n```python {{ title: 'Python' }}\nimport base64\n\n# Read image and encode to base64\nwith open(\"screenshot.png\", \"rb\") as f:\n image_data = base64.standard_b64encode(f.read()).decode(\"utf-8\")\n\nresponse = client.chat.completions.create(\n model=\"gpt-4o\", # or claude-sonnet-4-20250514, gemini-2.5-pro\n messages=[\n {\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"text\", \"text\": \"What's in this image? Describe it in detail.\"},\n {\n \"type\": \"image_url\",\n \"image_url\": {\n \"url\": f\"data:image/png;base64,{image_data}\"\n },\n },\n ],\n }\n ],\n max_tokens=1000,\n)\n\nprint(response.choices[0].message.content)\n```\n\n```typescript {{ title: 'Node.js' }}\nimport fs from \"fs\";\n\n// Read image and encode to base64\nconst imageData = fs.readFileSync(\"screenshot.png\").toString(\"base64\");\n\nconst response = await client.chat.completions.create({\n model: \"gpt-4o\", // or claude-sonnet-4-20250514, gemini-2.5-pro\n messages: [\n {\n role: \"user\",\n content: [\n { type: \"text\", text: \"What's in this image? Describe it in detail.\" },\n {\n type: \"image_url\",\n image_url: {\n url: `data:image/png;base64,${imageData}`,\n },\n },\n ],\n },\n ],\n max_tokens: 1000,\n});\n\nconsole.log(response.choices[0].message.content);\n```\n\n```bash {{ title: 'cURL' }}\n# Base64 encode an image (macOS)\nIMAGE_B64=$(base64 -i screenshot.png)\n\ncurl https://your-temps-instance.com/ai/v1/chat/completions \\\n -H \"Authorization: Bearer your-temps-api-key\" \\\n -H \"Content-Type: application/json\" \\\n -d \"{\n \\\"model\\\": \\\"gpt-4o\\\",\n \\\"messages\\\": [{\n \\\"role\\\": \\\"user\\\",\n \\\"content\\\": [\n {\\\"type\\\": \\\"text\\\", \\\"text\\\": \\\"What's in this image?\\\"},\n {\\\"type\\\": \\\"image_url\\\", \\\"image_url\\\": {\\\"url\\\": \\\"data:image/png;base64,${IMAGE_B64}\\\"}}\n ]\n }],\n \\\"max_tokens\\\": 1000\n }\"\n```\n\n</CodeGroup>\n\n### Image from URL\n\n```python {{ title: 'Image URL' }}\nresponse = client.chat.completions.create(\n model=\"gemini-2.5-pro\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"text\", \"text\": \"Describe what you see in this image.\"},\n {\n \"type\": \"image_url\",\n \"image_url\": {\n \"url\": \"https://example.com/photo.jpg\"\n },\n },\n ],\n }\n ],\n)\n```\n\n### Multiple Images\n\n```python {{ title: 'Multiple images' }}\nresponse = client.chat.completions.create(\n model=\"gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"text\", \"text\": \"Compare these two screenshots. What changed?\"},\n {\n \"type\": \"image_url\",\n \"image_url\": {\"url\": f\"data:image/png;base64,{before_image}\"},\n },\n {\n \"type\": \"image_url\",\n \"image_url\": {\"url\": f\"data:image/png;base64,{after_image}\"},\n },\n ],\n }\n ],\n)\n```\n\n<Note>\n Supported image formats: PNG, JPEG, GIF, WebP. Base64 data URIs work with all providers. HTTP URLs work with OpenAI and Gemini; Anthropic requires base64.\n</Note>\n\n---\n\n## Tool Calling (Function Calling) {{ anchor: true, id: 'tool-calling' }}\n\nDefine tools using the OpenAI format. The gateway translates them to each provider's native format.\n\n<CodeGroup title=\"Tool calling\">\n\n```python {{ title: 'Python' }}\nimport json\n\ntools = [\n {\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_weather\",\n \"description\": \"Get current weather for a city\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"description\": \"City name, e.g. 'London'\"\n },\n \"unit\": {\n \"type\": \"string\",\n \"enum\": [\"celsius\", \"fahrenheit\"],\n \"description\": \"Temperature unit\"\n }\n },\n \"required\": [\"location\"]\n }\n }\n }\n]\n\n# Step 1: Send message with tools\nresponse = client.chat.completions.create(\n model=\"claude-sonnet-4-20250514\", # works with any provider\n messages=[\n {\"role\": \"user\", \"content\": \"What's the weather like in London?\"}\n ],\n tools=tools,\n)\n\nmessage = response.choices[0].message\n\n# Step 2: Check if the model wants to call a tool\nif message.tool_calls:\n tool_call = message.tool_calls[0]\n args = json.loads(tool_call.function.arguments)\n print(f\"Model wants to call: {tool_call.function.name}({args})\")\n\n # Step 3: Execute the tool and send the result back\n weather_result = get_weather(args[\"location\"]) # your function\n\n follow_up = client.chat.completions.create(\n model=\"claude-sonnet-4-20250514\",\n messages=[\n {\"role\": \"user\", \"content\": \"What's the weather like in London?\"},\n message, # assistant message with tool_calls\n {\n \"role\": \"tool\",\n \"tool_call_id\": tool_call.id,\n \"content\": json.dumps(weather_result),\n },\n ],\n tools=tools,\n )\n\n print(follow_up.choices[0].message.content)\n```\n\n```typescript {{ title: 'Node.js' }}\nconst tools = [\n {\n type: \"function\" as const,\n function: {\n name: \"get_weather\",\n description: \"Get current weather for a city\",\n parameters: {\n type: \"object\",\n properties: {\n location: { type: \"string\", description: \"City name\" },\n unit: { type: \"string\", enum: [\"celsius\", \"fahrenheit\"] },\n },\n required: [\"location\"],\n },\n },\n },\n];\n\n// Step 1: Send message with tools\nconst response = await client.chat.completions.create({\n model: \"claude-sonnet-4-20250514\", // works with any provider\n messages: [\n { role: \"user\", content: \"What's the weather like in London?\" }\n ],\n tools,\n});\n\nconst message = response.choices[0].message;\n\n// Step 2: Check if the model wants to call a tool\nif (message.tool_calls) {\n const toolCall = message.tool_calls[0];\n const args = JSON.parse(toolCall.function.arguments);\n console.log(`Calling: ${toolCall.function.name}(${JSON.stringify(args)})`);\n\n // Step 3: Execute the tool and send result back\n const weatherResult = await getWeather(args.location);\n\n const followUp = await client.chat.completions.create({\n model: \"claude-sonnet-4-20250514\",\n messages: [\n { role: \"user\", content: \"What's the weather like in London?\" },\n message, // assistant message with tool_calls\n {\n role: \"tool\",\n tool_call_id: toolCall.id,\n content: JSON.stringify(weatherResult),\n },\n ],\n tools,\n });\n\n console.log(followUp.choices[0].message.content);\n}\n```\n\n</CodeGroup>\n\n### Tool Choice\n\nControl whether the model should call tools:\n\n```python {{ title: 'Tool choice options' }}\n# Let the model decide (default)\ntool_choice=\"auto\"\n\n# Force the model to call a tool\ntool_choice=\"required\"\n\n# Force a specific tool\ntool_choice={\"type\": \"function\", \"function\": {\"name\": \"get_weather\"}}\n\n# Don't use tools (even if provided)\ntool_choice=\"none\"\n```\n\n---\n\n## Streaming {{ anchor: true, id: 'streaming' }}\n\nSSE streaming works identically to the OpenAI API. All providers produce the same `data: {...}\\n\\n` format.\n\n<CodeGroup title=\"Streaming\">\n\n```python {{ title: 'Python' }}\nstream = client.chat.completions.create(\n model=\"gemini-2.5-flash\",\n messages=[\n {\"role\": \"user\", \"content\": \"Write a haiku about deployment automation.\"}\n ],\n stream=True,\n)\n\nfor chunk in stream:\n content = chunk.choices[0].delta.content\n if content:\n print(content, end=\"\", flush=True)\n\nprint() # newline at end\n```\n\n```typescript {{ title: 'Node.js' }}\nconst stream = await client.chat.completions.create({\n model: \"gemini-2.5-flash\",\n messages: [\n { role: \"user\", content: \"Write a haiku about deployment automation.\" }\n ],\n stream: true,\n});\n\nfor await (const chunk of stream) {\n const content = chunk.choices[0]?.delta?.content;\n if (content) {\n process.stdout.write(content);\n }\n}\n\nconsole.log(); // newline at end\n```\n\n</CodeGroup>\n\n---\n\n## Embeddings {{ anchor: true, id: 'embeddings' }}\n\nGenerate embeddings using OpenAI-compatible models.\n\n```python {{ title: 'Embeddings' }}\nresponse = client.embeddings.create(\n model=\"text-embedding-3-small\",\n input=\"The quick brown fox jumps over the lazy dog\",\n)\n\nembedding = response.data[0].embedding\nprint(f\"Dimensions: {len(embedding)}\")\n```\n\n<Note>\n Embeddings are currently supported for OpenAI models only. Anthropic and Gemini embedding support is planned.\n</Note>\n\n---\n\n## Provider Key Management {{ anchor: true, id: 'key-management' }}\n\nManage provider API keys through the Temps admin panel or API. Keys are encrypted at rest using AES-256-GCM.\n\n```bash {{ title: 'Add a provider key via API' }}\n# Add an OpenAI key\ncurl -X POST https://your-temps-instance.com/ai/providers/keys \\\n -H \"Authorization: Bearer your-admin-api-key\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"provider\": \"openai\",\n \"api_key\": \"sk-...\",\n \"label\": \"Production OpenAI Key\"\n }'\n\n# Add an Anthropic key\ncurl -X POST https://your-temps-instance.com/ai/providers/keys \\\n -H \"Authorization: Bearer your-admin-api-key\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"provider\": \"anthropic\",\n \"api_key\": \"sk-ant-...\",\n \"label\": \"Production Anthropic Key\"\n }'\n```\n\n<Note>\n Developers only need a Temps API key. Provider API keys are managed centrally by administrators and never exposed to end users.\n</Note>\n\n---\n\n## Usage & Analytics {{ anchor: true, id: 'usage' }}\n\nTrack token usage and costs per user, model, and provider through the Temps dashboard or API.\n\n```bash {{ title: 'Query usage' }}\n# Get usage summary for a project\ncurl https://your-temps-instance.com/ai/usage/summary?period=7d \\\n -H \"Authorization: Bearer your-admin-api-key\"\n```\n\nResponse:\n\n```json\n{\n \"total_requests\": 1523,\n \"total_tokens\": 2847392,\n \"prompt_tokens\": 1923847,\n \"completion_tokens\": 923545,\n \"by_model\": [\n {\"model\": \"gpt-4o\", \"requests\": 842, \"total_tokens\": 1523000},\n {\"model\": \"claude-sonnet-4-20250514\", \"requests\": 481, \"total_tokens\": 924392},\n {\"model\": \"gemini-2.5-pro\", \"requests\": 200, \"total_tokens\": 400000}\n ]\n}\n```\n"}, "accdcd671dc77a4ffdf84af522f181085d815be2": {"t": 1785765563.7249703, "n": 16850, "h": "9f251ac558958a64", "c": "export const metadata = {\n title: 'AI Gateway',\n description: 'OpenAI-compatible AI gateway built into Temps. Route requests to OpenAI, Anthropic, Gemini, and more with centralized API key management.',\n alternates: {\n canonical: '/docs/ai-gateway',\n },\n}\n\nexport const sections = [\n { title: 'Overview', id: 'overview' },\n { title: 'Quick Start', id: 'quick-start' },\n { title: 'Supported Providers', id: 'providers' },\n { title: 'Chat Completions', id: 'chat-completions' },\n { title: 'Vision & Image Attachments', id: 'vision' },\n { title: 'Tool Calling (Function Calling)', id: 'tool-calling' },\n { title: 'Streaming', id: 'streaming' },\n { title: 'Embeddings', id: 'embeddings' },\n { title: 'Provider Key Management', id: 'key-management' },\n { title: 'Usage & Analytics', id: 'usage' },\n]\n\n# AI Gateway\n\nOpenAI-compatible AI gateway built into Temps. Use any OpenAI SDK to call OpenAI, Anthropic, Gemini, xAI, Mistral, and DeepSeek — just change the base URL. {{ className: 'lead' }}\n\n---\n\n## Overview {{ anchor: true, id: 'overview' }}\n\nThe AI Gateway provides a unified, OpenAI-compatible API for all major LLM providers. Your existing OpenAI SDK code works unchanged — just point it at your Temps instance.\n\n<Row>\n <Col>\n **What's Included**\n - OpenAI API wire-compatible endpoints\n - Multi-provider routing (OpenAI, Anthropic, Gemini, xAI, Mistral, DeepSeek)\n - Tool calling / function calling across all providers\n - Vision / image attachments (base64 and URL)\n - SSE streaming with consistent format\n - Centralized API key management\n - Per-user usage tracking and analytics\n - JSON mode / structured output\n </Col>\n <Col>\n **Why It Matters**\n - Zero code changes — works with any OpenAI SDK\n - Centralized key management (devs never see API keys)\n - Switch providers without changing client code\n - Built-in usage analytics and cost tracking\n - Self-hosted — your data stays on your infrastructure\n - No per-request fees beyond provider costs\n - Transparent translation for Anthropic and Gemini\n </Col>\n</Row>\n\n### Key Features\n\n<Properties>\n <Property name=\"Drop-In Replacement\">\n 100% OpenAI SDK compatible. Works with Python, Node.js, Go, Rust, and any other OpenAI client library.\n </Property>\n <Property name=\"Transparent Translation\">\n Automatically translates requests/responses for Anthropic Messages API and Gemini generateContent API.\n </Property>\n <Property name=\"Vision Support\">\n Send images as base64 data URIs or HTTP URLs. Translated to each provider's native format automatically.\n </Property>\n <Property name=\"Tool Calling\">\n OpenAI-format tools work across all providers. Function definitions, tool_choice, and tool results are translated transparently.\n </Property>\n <Property name=\"Streaming\">\n Server-Sent Events (SSE) streaming with consistent `data: {...}\\n\\n` format across all providers.\n </Property>\n <Property name=\"Usage Analytics\">\n Track token usage, costs, and request counts per user, model, and provider.\n </Property>\n</Properties>\n\n---\n\n## Quick Start {{ anchor: true, id: 'quick-start' }}\n\nPoint any OpenAI SDK at your Temps instance. The only changes are `base_url` and `api_key`.\n\n<CodeGroup title=\"Setup\">\n\n```python {{ title: 'Python' }}\nfrom openai import OpenAI\n\nclient = OpenAI(\n base_url=\"https://your-temps-instance.com/ai/v1\",\n api_key=\"your-temps-api-key\",\n)\n\nresponse = client.chat.completions.create(\n model=\"gpt-4o\", # or \"claude-sonnet-4-20250514\", \"gemini-2.5-pro\", etc.\n messages=[\n {\"role\": \"user\", \"content\": \"Hello!\"}\n ],\n)\n\nprint(response.choices[0].message.content)\n```\n\n```typescript {{ title: 'Node.js' }}\nimport OpenAI from \"openai\";\n\nconst client = new OpenAI({\n baseURL: \"https://your-temps-instance.com/ai/v1\",\n apiKey: \"your-temps-api-key\",\n});\n\nconst response = await client.chat.completions.create({\n model: \"gpt-4o\", // or \"claude-sonnet-4-20250514\", \"gemini-2.5-pro\", etc.\n messages: [\n { role: \"user\", content: \"Hello!\" }\n ],\n});\n\nconsole.log(response.choices[0].message.content);\n```\n\n```bash {{ title: 'cURL' }}\ncurl https://your-temps-instance.com/ai/v1/chat/completions \\\n -H \"Authorization: Bearer your-temps-api-key\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"model\": \"gpt-4o\",\n \"messages\": [{\"role\": \"user\", \"content\": \"Hello!\"}]\n }'\n```\n\n</CodeGroup>\n\n---\n\n## Supported Providers {{ anchor: true, id: 'providers' }}\n\nThe gateway routes requests to the correct provider based on the model name:\n\n| Model Prefix | Provider | Examples |\n|---|---|---|\n| `gpt-*`, `o1-*`, `o3-*` | OpenAI | `gpt-4o`, `gpt-4o-mini`, `o3-mini` |\n| `claude-*` | Anthropic | `claude-sonnet-4-20250514`, `claude-haiku-4-5-20251001` |\n| `gemini-*` | Google Gemini | `gemini-2.5-pro`, `gemini-2.5-flash` |\n| `grok-*` | xAI | `grok-3`, `grok-3-mini` |\n| `mistral-*`, `codestral-*` | Mistral | `mistral-large-latest`, `codestral-latest` |\n| `deepseek-*` | DeepSeek | `deepseek-chat`, `deepseek-reasoner` |\n\n<Note>\n All providers use the same OpenAI-compatible API format. The gateway handles protocol translation automatically for Anthropic and Gemini.\n</Note>\n\n---\n\n## Chat Completions {{ anchor: true, id: 'chat-completions' }}\n\nStandard chat completions work identically to the OpenAI API.\n\n```python {{ title: 'Multi-turn conversation' }}\nresponse = client.chat.completions.create(\n model=\"claude-sonnet-4-20250514\",\n messages=[\n {\"role\": \"system\", \"content\": \"You are a helpful coding assistant.\"},\n {\"role\": \"user\", \"content\": \"Write a Python function to calculate fibonacci numbers.\"},\n {\"role\": \"assistant\", \"content\": \"Here's a fibonacci function:\\n\\n```python\\ndef fib(n):\\n if n <= 1:\\n return n\\n return fib(n-1) + fib(n-2)\\n```\"},\n {\"role\": \"user\", \"content\": \"Can you make it iterative for better performance?\"},\n ],\n temperature=0.7,\n max_tokens=500,\n)\n```\n\n---\n\n## Vision & Image Attachments {{ anchor: true, id: 'vision' }}\n\nSend images to vision-capable models. The gateway translates image formats automatically for each provider.\n\n### Base64 Image\n\n<CodeGroup title=\"Send base64 image\">\n\n```python {{ title: 'Python' }}\nimport base64\n\n# Read image and encode to base64\nwith open(\"screenshot.png\", \"rb\") as f:\n image_data = base64.standard_b64encode(f.read()).decode(\"utf-8\")\n\nresponse = client.chat.completions.create(\n model=\"gpt-4o\", # or claude-sonnet-4-20250514, gemini-2.5-pro\n messages=[\n {\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"text\", \"text\": \"What's in this image? Describe it in detail.\"},\n {\n \"type\": \"image_url\",\n \"image_url\": {\n \"url\": f\"data:image/png;base64,{image_data}\"\n },\n },\n ],\n }\n ],\n max_tokens=1000,\n)\n\nprint(response.choices[0].message.content)\n```\n\n```typescript {{ title: 'Node.js' }}\nimport fs from \"fs\";\n\n// Read image and encode to base64\nconst imageData = fs.readFileSync(\"screenshot.png\").toString(\"base64\");\n\nconst response = await client.chat.completions.create({\n model: \"gpt-4o\", // or claude-sonnet-4-20250514, gemini-2.5-pro\n messages: [\n {\n role: \"user\",\n content: [\n { type: \"text\", text: \"What's in this image? Describe it in detail.\" },\n {\n type: \"image_url\",\n image_url: {\n url: `data:image/png;base64,${imageData}`,\n },\n },\n ],\n },\n ],\n max_tokens: 1000,\n});\n\nconsole.log(response.choices[0].message.content);\n```\n\n```bash {{ title: 'cURL' }}\n# Base64 encode an image (macOS)\nIMAGE_B64=$(base64 -i screenshot.png)\n\ncurl https://your-temps-instance.com/ai/v1/chat/completions \\\n -H \"Authorization: Bearer your-temps-api-key\" \\\n -H \"Content-Type: application/json\" \\\n -d \"{\n \\\"model\\\": \\\"gpt-4o\\\",\n \\\"messages\\\": [{\n \\\"role\\\": \\\"user\\\",\n \\\"content\\\": [\n {\\\"type\\\": \\\"text\\\", \\\"text\\\": \\\"What's in this image?\\\"},\n {\\\"type\\\": \\\"image_url\\\", \\\"image_url\\\": {\\\"url\\\": \\\"data:image/png;base64,${IMAGE_B64}\\\"}}\n ]\n }],\n \\\"max_tokens\\\": 1000\n }\"\n```\n\n</CodeGroup>\n\n### Image from URL\n\n```python {{ title: 'Image URL' }}\nresponse = client.chat.completions.create(\n model=\"gemini-2.5-pro\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"text\", \"text\": \"Describe what you see in this image.\"},\n {\n \"type\": \"image_url\",\n \"image_url\": {\n \"url\": \"https://example.com/photo.jpg\"\n },\n },\n ],\n }\n ],\n)\n```\n\n### Multiple Images\n\n```python {{ title: 'Multiple images' }}\nresponse = client.chat.completions.create(\n model=\"gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": [\n {\"type\": \"text\", \"text\": \"Compare these two screenshots. What changed?\"},\n {\n \"type\": \"image_url\",\n \"image_url\": {\"url\": f\"data:image/png;base64,{before_image}\"},\n },\n {\n \"type\": \"image_url\",\n \"image_url\": {\"url\": f\"data:image/png;base64,{after_image}\"},\n },\n ],\n }\n ],\n)\n```\n\n<Note>\n Supported image formats: PNG, JPEG, GIF, WebP. Base64 data URIs work with all providers. HTTP URLs work with OpenAI and Gemini; Anthropic requires base64.\n</Note>\n\n---\n\n## Tool Calling (Function Calling) {{ anchor: true, id: 'tool-calling' }}\n\nDefine tools using the OpenAI format. The gateway translates them to each provider's native format.\n\n<CodeGroup title=\"Tool calling\">\n\n```python {{ title: 'Python' }}\nimport json\n\ntools = [\n {\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_weather\",\n \"description\": \"Get current weather for a city\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"description\": \"City name, e.g. 'London'\"\n },\n \"unit\": {\n \"type\": \"string\",\n \"enum\": [\"celsius\", \"fahrenheit\"],\n \"description\": \"Temperature unit\"\n }\n },\n \"required\": [\"location\"]\n }\n }\n }\n]\n\n# Step 1: Send message with tools\nresponse = client.chat.completions.create(\n model=\"claude-sonnet-4-20250514\", # works with any provider\n messages=[\n {\"role\": \"user\", \"content\": \"What's the weather like in London?\"}\n ],\n tools=tools,\n)\n\nmessage = response.choices[0].message\n\n# Step 2: Check if the model wants to call a tool\nif message.tool_calls:\n tool_call = message.tool_calls[0]\n args = json.loads(tool_call.function.arguments)\n print(f\"Model wants to call: {tool_call.function.name}({args})\")\n\n # Step 3: Execute the tool and send the result back\n weather_result = get_weather(args[\"location\"]) # your function\n\n follow_up = client.chat.completions.create(\n model=\"claude-sonnet-4-20250514\",\n messages=[\n {\"role\": \"user\", \"content\": \"What's the weather like in London?\"},\n message, # assistant message with tool_calls\n {\n \"role\": \"tool\",\n \"tool_call_id\": tool_call.id,\n \"content\": json.dumps(weather_result),\n },\n ],\n tools=tools,\n )\n\n print(follow_up.choices[0].message.content)\n```\n\n```typescript {{ title: 'Node.js' }}\nconst tools = [\n {\n type: \"function\" as const,\n function: {\n name: \"get_weather\",\n description: \"Get current weather for a city\",\n parameters: {\n type: \"object\",\n properties: {\n location: { type: \"string\", description: \"City name\" },\n unit: { type: \"string\", enum: [\"celsius\", \"fahrenheit\"] },\n },\n required: [\"location\"],\n },\n },\n },\n];\n\n// Step 1: Send message with tools\nconst response = await client.chat.completions.create({\n model: \"claude-sonnet-4-20250514\", // works with any provider\n messages: [\n { role: \"user\", content: \"What's the weather like in London?\" }\n ],\n tools,\n});\n\nconst message = response.choices[0].message;\n\n// Step 2: Check if the model wants to call a tool\nif (message.tool_calls) {\n const toolCall = message.tool_calls[0];\n const args = JSON.parse(toolCall.function.arguments);\n console.log(`Calling: ${toolCall.function.name}(${JSON.stringify(args)})`);\n\n // Step 3: Execute the tool and send result back\n const weatherResult = await getWeather(args.location);\n\n const followUp = await client.chat.completions.create({\n model: \"claude-sonnet-4-20250514\",\n messages: [\n { role: \"user\", content: \"What's the weather like in London?\" },\n message, // assistant message with tool_calls\n {\n role: \"tool\",\n tool_call_id: toolCall.id,\n content: JSON.stringify(weatherResult),\n },\n ],\n tools,\n });\n\n console.log(followUp.choices[0].message.content);\n}\n```\n\n</CodeGroup>\n\n### Tool Choice\n\nControl whether the model should call tools:\n\n```python {{ title: 'Tool choice options' }}\n# Let the model decide (default)\ntool_choice=\"auto\"\n\n# Force the model to call a tool\ntool_choice=\"required\"\n\n# Force a specific tool\ntool_choice={\"type\": \"function\", \"function\": {\"name\": \"get_weather\"}}\n\n# Don't use tools (even if provided)\ntool_choice=\"none\"\n```\n\n---\n\n## Streaming {{ anchor: true, id: 'streaming' }}\n\nSSE streaming works identically to the OpenAI API. All providers produce the same `data: {...}\\n\\n` format.\n\n<CodeGroup title=\"Streaming\">\n\n```python {{ title: 'Python' }}\nstream = client.chat.completions.create(\n model=\"gemini-2.5-flash\",\n messages=[\n {\"role\": \"user\", \"content\": \"Write a haiku about deployment automation.\"}\n ],\n stream=True,\n)\n\nfor chunk in stream:\n content = chunk.choices[0].delta.content\n if content:\n print(content, end=\"\", flush=True)\n\nprint() # newline at end\n```\n\n```typescript {{ title: 'Node.js' }}\nconst stream = await client.chat.completions.create({\n model: \"gemini-2.5-flash\",\n messages: [\n { role: \"user\", content: \"Write a haiku about deployment automation.\" }\n ],\n stream: true,\n});\n\nfor await (const chunk of stream) {\n const content = chunk.choices[0]?.delta?.content;\n if (content) {\n process.stdout.write(content);\n }\n}\n\nconsole.log(); // newline at end\n```\n\n</CodeGroup>\n\n---\n\n## Embeddings {{ anchor: true, id: 'embeddings' }}\n\nGenerate embeddings using OpenAI-compatible models.\n\n```python {{ title: 'Embeddings' }}\nresponse = client.embeddings.create(\n model=\"text-embedding-3-small\",\n input=\"The quick brown fox jumps over the lazy dog\",\n)\n\nembedding = response.data[0].embedding\nprint(f\"Dimensions: {len(embedding)}\")\n```\n\n<Note>\n Embeddings are currently supported for OpenAI models only. Anthropic and Gemini embedding support is planned.\n</Note>\n\n---\n\n## Provider Key Management {{ anchor: true, id: 'key-management' }}\n\nManage provider API keys through the Temps admin panel or API. Keys are encrypted at rest using AES-256-GCM.\n\n```bash {{ title: 'Add a provider key via API' }}\n# Add an OpenAI key\ncurl -X POST https://your-temps-instance.com/ai/providers/keys \\\n -H \"Authorization: Bearer your-admin-api-key\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"provider\": \"openai\",\n \"api_key\": \"sk-...\",\n \"label\": \"Production OpenAI Key\"\n }'\n\n# Add an Anthropic key\ncurl -X POST https://your-temps-instance.com/ai/providers/keys \\\n -H \"Authorization: Bearer your-admin-api-key\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\n \"provider\": \"anthropic\",\n \"api_key\": \"sk-ant-...\",\n \"label\": \"Production Anthropic Key\"\n }'\n```\n\n<Note>\n Developers only need a Temps API key. Provider API keys are managed centrally by administrators and never exposed to end users.\n</Note>\n\n---\n\n## Usage & Analytics {{ anchor: true, id: 'usage' }}\n\nTrack token usage and costs per user, model, and provider through the Temps dashboard or API.\n\n```bash {{ title: 'Query usage' }}\n# Get usage summary for a project\ncurl https://your-temps-instance.com/ai/usage/summary?period=7d \\\n -H \"Authorization: Bearer your-admin-api-key\"\n```\n\nResponse:\n\n```json\n{\n \"total_requests\": 1523,\n \"total_tokens\": 2847392,\n \"prompt_tokens\": 1923847,\n \"completion_tokens\": 923545,\n \"by_model\": [\n {\"model\": \"gpt-4o\", \"requests\": 842, \"total_tokens\": 1523000},\n {\"model\": \"claude-sonnet-4-20250514\", \"requests\": 481, \"total_tokens\": 924392},\n {\"model\": \"gemini-2.5-pro\", \"requests\": 200, \"total_tokens\": 400000}\n ]\n}\n```\n"}}}} |