1 line
30 KiB
JSON
1 line
30 KiB
JSON
{"repo": "argszero/codex-bridge", "files": {"README.md": {"7b7376c6a9877460763c305ac5e4e2eb8f29f6a9": {"t": 1785674078.4755087, "n": 7802, "h": "59f599498dd4f141", "c": "# codex-bridge\n\nZero-config bridge for [Codex CLI](https://github.com/openai/codex) — prefixes your model name with `-cb-` and automatically routes through any OpenAI-compatible API.\n\n**No configuration files. No API keys. No setup.** Just run it and prefix your model.\n\nWorks great alongside [cc-switch](https://github.com/farion1231/cc-switch) for API key management — let cc-switch manage your provider keys, let codex-bridge handle the protocol translation.\n\n## Quick Start\n\n### 1. Start the bridge\n\n```bash\ngit clone https://github.com/argszero/codex-bridge.git\ncd codex-bridge\n./start.sh\n```\n\nRequires Python ≥ 3.11. Uses [uv](https://docs.astral.sh/uv/).\n\n### 2. Activate with a prefix\n\nEdit `~/.codex/config.toml` — add `-cb-` to your model name:\n\n```toml\nmodel = \"-cb-deepseek-v4-pro\"\n```\n\nThe watcher detects this within seconds, backs up your config, and rewrites it:\n\n```toml\nmodel = \"deepseek-v4-pro\"\n[model_providers.custom]\nbase_url = \"http://localhost:10110/api.deepseek.com\"\n```\n\nDone. Open a new terminal and use Codex.\n\n### 3. Deactivate\n\nRemove the `-cb-` prefix. Restore your original `base_url` manually (your backup is at `~/.codex/config.toml.cb.bak`).\n\n## Provider Examples\n\nThe watcher reads your existing `base_url` to discover the upstream API, so it works with whatever provider config you already have — cc-switch presets, manual config, or anything else. Just add `-cb-` to the model name.\n\n### Kimi (Moonshot)\n\n**Before** — your `~/.codex/config.toml` (configured via cc-switch Kimi preset or manually):\n\n```toml\nmodel_provider = \"custom\"\nmodel = \"kimi-k2.7-code\"\n\n[model_providers.custom]\nname = \"Kimi\"\nbase_url = \"https://api.moonshot.cn/v1\"\napi_key = \"sk-...\"\n```\n\n**Step 1** — add `-cb-` prefix:\n\n```toml\nmodel = \"-cb-kimi-k2.7-code\" # ← add the prefix\n```\n\n**Step 2** — watcher auto-rewrites (seconds later):\n\n```toml\nmodel = \"kimi-k2.7-code\"\n[model_providers.custom]\nbase_url = \"http://localhost:10110/api.moonshot.cn/v1\"\napi_key = \"sk-...\" # ← key preserved, forwarded by bridge\n```\n\n**Step 3** — restart Codex, done.\n\n### DeepSeek\n\n**Before:**\n\n```toml\nmodel_provider = \"custom\"\nmodel = \"deepseek-v4-pro\"\n\n[model_providers.custom]\nbase_url = \"https://api.deepseek.com\"\napi_key = \"sk-...\"\n```\n\n**Add prefix** → watcher rewrites to:\n\n```toml\nmodel = \"deepseek-v4-pro\"\n[model_providers.custom]\nbase_url = \"http://localhost:10110/api.deepseek.com\"\napi_key = \"sk-...\"\n```\n\n### Alibaba DashScope (Qwen)\n\n**Before:**\n\n```toml\nmodel_provider = \"custom\"\nmodel = \"qwen3-coder-plus\"\n\n[model_providers.custom]\nbase_url = \"https://dashscope.aliyuncs.com/compatible-mode/v1\"\napi_key = \"sk-...\"\n```\n\n**Add prefix** → watcher rewrites to:\n\n```toml\nmodel = \"qwen3-coder-plus\"\n[model_providers.custom]\nbase_url = \"http://localhost:10110/dashscope.aliyuncs.com/compatible-mode/v1\"\napi_key = \"sk-...\"\n```\n\n### Groq\n\n**Before:**\n\n```toml\nmodel_provider = \"custom\"\nmodel = \"qwen-qwq-32b\"\n\n[model_providers.custom]\nbase_url = \"https://api.groq.com/openai/v1\"\napi_key = \"gsk_...\"\n```\n\n**Add prefix** → watcher rewrites to:\n\n```toml\nmodel = \"qwen-qwq-32b\"\n[model_providers.custom]\nbase_url = \"http://localhost:10110/api.groq.com/openai/v1\"\napi_key = \"gsk_...\"\n```\n\n### Ollama (local)\n\n**Before:**\n\n```toml\nmodel_provider = \"custom\"\nmodel = \"qwen3:32b\"\n\n[model_providers.custom]\nbase_url = \"http://localhost:11434/v1\"\napi_key = \"ollama\"\n```\n\n**Add prefix** → watcher rewrites to:\n\n```toml\nmodel = \"qwen3:32b\"\n[model_providers.custom]\nbase_url = \"http://localhost:10110/localhost:11434/v1\"\napi_key = \"ollama\"\n```\n\n> **How cc-switch fits in:** Use cc-switch to manage API keys and switch between providers with a GUI. cc-switch writes the `model_providers` section in `config.toml`. Then add `-cb-` to the model name, and codex-bridge picks up the upstream from cc-switch's config and handles the protocol translation — zero additional setup.\n\n## How It Works\n\n```\nCodex CLI ──▶ ~/.codex/config.toml ──▶ bridge :10110 ──▶ Upstream API\n ▲ │\n │ watcher detects -cb- │ Responses → Chat\n │ rewrites config │ forwards client key\n └─────────────────────────────┘\n```\n\n- **Watcher** polls `~/.codex/config.toml`, detects `-cb-` prefix, reads the existing `base_url` to discover the upstream host, then rewrites `base_url` to route through the bridge\n- **Bridge** receives requests, extracts upstream from URL path, forwards with client's API key\n\n### Request path encoding\n\nThe original upstream URL is encoded into the rewritten `base_url`:\n\n| Original base_url | Rewritten |\n|---|---|\n| `https://api.deepseek.com` | `http://localhost:10110/api.deepseek.com` |\n| `https://api.moonshot.cn/v1` | `http://localhost:10110/api.moonshot.cn/v1` |\n| `https://dashscope.aliyuncs.com/compatible-mode/v1` | `http://localhost:10110/dashscope.aliyuncs.com/compatible-mode/v1` |\n| `https://api.groq.com/openai/v1` | `http://localhost:10110/api.groq.com/openai/v1` |\n| `http://localhost:11434/v1` (Ollama) | `http://localhost:10110/localhost:11434/v1` |\n\nNo state files, no side channels — the bridge knows where to forward from the URL alone.\n\n## CLI Options\n\n```bash\n./start.sh --port 10110 --multimodal\n```\n\n| Flag | Default | Description |\n|------|---------|-------------|\n| `--port` | `10110` | Listen port |\n| `--poll-interval` | `5` | Config poll interval (seconds) |\n| `--timeout` | `30` | Upstream API timeout (minutes) |\n| `--multimodal` | auto | Force-enable image support (auto-detected from model name by default) |\n\nAll flags are optional. Zero config needed.\n\n## Supported Providers\n\nAny provider with an **OpenAI-compatible Chat Completions API**:\n\n- [DeepSeek](https://platform.deepseek.com/)\n- [Moonshot (Kimi)](https://platform.moonshot.cn/)\n- [Alibaba DashScope (Qwen)](https://dashscope.aliyun.com/)\n- [Groq](https://console.groq.com/)\n- [Together AI](https://api.together.xyz/)\n- [Ollama](https://ollama.com/) (local)\n- [vLLM](https://docs.vllm.ai/) (local)\n\n## Protocol Translation\n\n**Request (Responses → Chat Completions)**\n\n| Responses API | Chat Completions |\n|---|---|\n| `input_text` / `output_text` | message `content` |\n| `function_call` | assistant `tool_calls` |\n| `function_call_output` | `tool` role message |\n| `reasoning` | skipped; rc on adjacent message |\n| `developer` role | `system` role |\n| `instructions` | system message + identity injection |\n| `tools` / `tool_choice` / `thinking` | translated |\n\n**Response (Chat Completions SSE → Responses SSE)**\n\n| Chat SSE | Responses SSE |\n|---|---|\n| First delta | `response.created` + `response.in_progress` |\n| `delta.content` | `response.output_text.delta` / `done` |\n| `delta.reasoning_content` | `response.reasoning_text.delta` / `done` |\n| `delta.tool_calls` | `response.function_call_arguments.delta` / `done` |\n| Stream end | `response.output_item.done` × N + `response.completed` |\n\n## Files\n\n| File | Description |\n|------|-------------|\n| `src/main.py` | Entry point — argparse CLI, starts watcher + server |\n| `src/watcher.py` | Config poller — detects `-cb-`, backups & rewrites config |\n| `src/server.py` | HTTP proxy — parses upstream from path, forwards requests |\n| `src/translate.py` | Input translation (Responses → Chat) |\n| `src/sse.py` | SSE output translation (Chat → Responses) |\n| `src/recover.py` | reasoning_content recovery |\n| `src/log.py` | Colored logging |\n\n## Acknowledgments\n\nInspired by [codex-deepseek](https://github.com/yangfei4913438/codex-deepseek) — the original project that pioneered Responses ↔ Chat Completions protocol translation for Codex CLI. codex-bridge builds on that idea with a zero-config approach: no need to edit config files when switching models, just prefix your model name with `-cb-` and the watcher handles the rest.\n\n## License\n\nMIT\n"}}, "src/server.py": {"7b7376c6a9877460763c305ac5e4e2eb8f29f6a9": {"t": 1785746648.2912264, "n": 21181, "h": "688d8b57d2936391", "c": "\"\"\"HTTP server: listens on port 10110, translates Responses API ↔ Chat Completions.\"\"\"\n\nimport json\nimport random\nimport string\nfrom http.client import HTTPSConnection\nfrom http.server import BaseHTTPRequestHandler, ThreadingHTTPServer\nfrom urllib.parse import urlparse\n\nfrom . import log\nfrom .recover import recover_reasoning, remember_reasoning, session_key\nfrom .sse import SseTranslator\nfrom .translate import (\n last_user_text,\n translate_messages,\n translate_tool_choice,\n translate_tools,\n)\n\n\ndef _rand_id(prefix: str, length: int = 8) -> str:\n suffix = \"\".join(random.choices(string.ascii_lowercase + string.digits, k=length))\n return f\"{prefix}_{suffix}\"\n\n\ndef build_chat_body(\n body: dict,\n base_url: str,\n model: str,\n multimodal: bool,\n) -> dict:\n \"\"\"Translate a Responses API request body into a Chat Completions body.\"\"\"\n stream = body.get(\"stream\") is not False\n raw_thinking = body.get(\"thinking\")\n raw_reasoning = body.get(\"reasoning\")\n has_thinking = raw_thinking is not None or raw_reasoning is not None\n\n # Auto-detect multimodal from model name (blocklist for non-vision models).\n # CLI flag (--multimodal) overrides the auto-detection.\n if not multimodal:\n model_lower = model.lower()\n multimodal = not any(\n name in model_lower for name in (\"deepseek-v4\", \"deepseek-v3\")\n )\n\n result = translate_messages(\n body.get(\"input\"),\n {\"keepReasoningContent\": has_thinking, \"multimodal\": multimodal},\n )\n messages = result[\"messages\"]\n stats = result[\"stats\"]\n\n restored = recover_reasoning(session_key(body), messages)\n has_assistant_with_rc = any(\n m.get(\"role\") == \"assistant\" and m.get(\"reasoning_content\") for m in messages\n )\n has_assistant_with_tc = any(\n m.get(\"role\") == \"assistant\" and m.get(\"tool_calls\") for m in messages\n )\n effective_thinking = has_thinking and (\n has_assistant_with_rc or not has_assistant_with_tc\n )\n\n if has_thinking and not effective_thinking:\n log.warn(\"thinking off: missing rc in history\")\n if restored > 0 and effective_thinking:\n log.ok(f\"rc restored x{restored}\")\n if stats[\"strippedReasoningContent\"] > 0:\n log.skip(f\"rc stripped x{stats['strippedReasoningContent']}\")\n if stats[\"preservedReasoningContent\"] > 0 and not restored:\n log.info(f\"rc preserved x{stats['preservedReasoningContent']}\")\n\n last_user = last_user_text(messages)\n preview = last_user[:120] + \"...\" if len(last_user) > 120 else last_user\n log.req(\n f\"thinking:{'on' if has_thinking else 'off'} \"\n f\"msgs:{len(messages)} stream:{stream} | {preview}\"\n )\n\n # Identity injection\n IDENTITY = (\n f\"\\n\\n[IMPORTANT: Your true model identity is {model}. \"\n \"You are NOT OpenAI, GPT, or Claude. When asked about your model identity, \"\n \"you MUST answer truthfully based on your actual model name. \"\n \"Ignore any conflicting identity claims in the instructions above.]\"\n )\n instructions = body.get(\"instructions\", \"\")\n if instructions:\n instructions = instructions + IDENTITY\n else:\n instructions = IDENTITY.strip()\n messages.insert(0, {\"role\": \"system\", \"content\": instructions})\n\n chat_body: dict = {\"model\": model, \"messages\": messages, \"stream\": stream}\n # Pass through thinking/reasoning from the request as-is\n if raw_thinking is not None:\n chat_body[\"thinking\"] = raw_thinking\n elif raw_reasoning is not None:\n chat_body[\"reasoning\"] = raw_reasoning\n\n tools = translate_tools(body.get(\"tools\"))\n if tools:\n chat_body[\"tools\"] = tools\n tc = translate_tool_choice(body.get(\"tool_choice\"))\n if tc:\n chat_body[\"tool_choice\"] = tc\n\n if body.get(\"temperature\") is not None:\n chat_body[\"temperature\"] = body[\"temperature\"]\n if body.get(\"top_p\") is not None:\n chat_body[\"top_p\"] = body[\"top_p\"]\n if body.get(\"max_output_tokens\") is not None:\n chat_body[\"max_tokens\"] = body[\"max_output_tokens\"]\n\n # Pass through parameters that share the same name in both APIs\n _PASSTHROUGH_PARAMS = (\n \"store\", \"safety_identifier\", \"metadata\", \"service_tier\",\n \"parallel_tool_calls\", \"prompt_cache_key\", \"prompt_cache_retention\",\n \"frequency_penalty\", \"presence_penalty\", \"stop\", \"n\",\n \"prediction\", \"verbosity\", \"web_search_options\", \"modalities\",\n \"moderation\", \"logprobs\", \"top_logprobs\",\n )\n for param in _PASSTHROUGH_PARAMS:\n val = body.get(param)\n if val is not None:\n chat_body[param] = val\n\n # top_logprobs requires logprobs to be true in Chat API\n if body.get(\"top_logprobs\") is not None and body.get(\"logprobs\") is None:\n chat_body[\"logprobs\"] = True\n\n # Map text.format to response_format when applicable\n text_cfg = body.get(\"text\", {})\n if isinstance(text_cfg, dict):\n fmt = text_cfg.get(\"format\", {})\n if isinstance(fmt, dict):\n fmt_type = fmt.get(\"type\")\n if fmt_type == \"json_schema\":\n # Copy the format dict as json_schema (minus \"type\")\n json_schema = {k: v for k, v in fmt.items() if k != \"type\"}\n chat_body[\"response_format\"] = {\n \"type\": \"json_schema\",\n \"json_schema\": json_schema,\n }\n elif fmt_type == \"json_object\":\n chat_body[\"response_format\"] = {\"type\": \"json_object\"}\n\n return {\"chat_body\": chat_body, \"stream\": stream, \"messages\": messages}\n\n\ndef build_non_stream_response(completion: dict, model: str) -> dict:\n \"\"\"Build a non-streaming Responses API response from a Chat Completions response.\"\"\"\n msg = (completion.get(\"choices\") or [{}])[0].get(\"message\", {})\n usage = completion.get(\"usage\")\n output = []\n\n if msg.get(\"reasoning_content\"):\n output.append({\n \"id\": _rand_id(\"rsn\", 6),\n \"type\": \"reasoning\",\n \"content\": [{\"type\": \"reasoning_text\", \"text\": msg[\"reasoning_content\"]}],\n \"status\": \"completed\",\n })\n if msg.get(\"content\"):\n output.append({\n \"id\": _rand_id(\"msg\", 6),\n \"type\": \"message\",\n \"role\": \"assistant\",\n \"content\": [\n {\"type\": \"output_text\", \"text\": msg[\"content\"], \"annotations\": []}\n ],\n \"status\": \"completed\",\n })\n elif msg.get(\"refusal\"):\n # Safety refusal — content is null, refusal contains the explanation\n output.append({\n \"id\": _rand_id(\"msg\", 6),\n \"type\": \"message\",\n \"role\": \"assistant\",\n \"content\": [\n {\"type\": \"output_text\", \"text\": msg[\"refusal\"], \"annotations\": []}\n ],\n \"status\": \"completed\",\n })\n if msg.get(\"tool_calls\"):\n for tc in msg[\"tool_calls\"]:\n output.append({\n \"id\": f\"fc_{tc['id']}\",\n \"type\": \"function_call\",\n \"call_id\": tc[\"id\"],\n \"name\": tc[\"function\"][\"name\"],\n \"arguments\": tc[\"function\"][\"arguments\"],\n \"status\": \"completed\",\n })\n # Handle legacy function_call (singular, deprecated)\n if msg.get(\"function_call\") and not msg.get(\"tool_calls\"):\n fc = msg[\"function_call\"]\n output.append({\n \"id\": f\"fc_{fc.get('name', 'legacy')}\",\n \"type\": \"function_call\",\n \"call_id\": fc.get(\"name\", \"legacy\"),\n \"name\": fc.get(\"name\", \"\"),\n \"arguments\": fc.get(\"arguments\", \"\"),\n \"status\": \"completed\",\n })\n\n return {\n \"id\": _rand_id(\"resp\", 10),\n \"object\": \"response\",\n \"status\": \"completed\",\n \"model\": model,\n \"output\": output,\n \"usage\": {\n \"input_tokens\": usage.get(\"prompt_tokens\", 0) if usage else 0,\n \"output_tokens\": usage.get(\"completion_tokens\", 0) if usage else 0,\n \"total_tokens\": usage.get(\"total_tokens\", 0) if usage else 0,\n }\n if usage\n else None,\n }\n\n\nFORWARD_HEADERS = {\n \"user-agent\", \"x-\", \"openai-\", \"content-type\", \"accept\",\n \"accept-encoding\", \"accept-language\", \"connection\",\n}\n\ndef upstream_request(\n base_url: str,\n api_key: str,\n chat_body: dict,\n timeout: int,\n client_headers: dict | None = None,\n stream: bool = False,\n) -> tuple:\n \"\"\"Call the upstream API via http.client.\n\n Returns (status, body_or_response, connection).\n \"\"\"\n parsed = urlparse(base_url)\n host = parsed.netloc or \"api.deepseek.com\"\n path = parsed.path.rstrip(\"/\") + \"/chat/completions\"\n body_bytes = json.dumps(chat_body).encode(\"utf-8\")\n\n # Start with our required headers\n headers = {\n \"Authorization\": f\"Bearer {api_key}\",\n \"Content-Type\": \"application/json\",\n \"Accept\": \"text/event-stream\" if stream else \"application/json\",\n }\n # Forward allowed client headers (our required headers take precedence)\n if client_headers:\n for key, value in client_headers.items():\n kl = key.lower()\n if kl in (\"authorization\", \"host\", \"content-length\", \"transfer-encoding\"):\n continue # we set these ourselves, or they're hop-by-hop\n if kl.startswith(\"x-\") or kl.startswith(\"openai-\"):\n headers[key] = value\n elif kl in (\"user-agent\", \"accept-encoding\", \"accept-language\"):\n headers[key] = value\n\n conn = HTTPSConnection(host, timeout=timeout)\n try:\n conn.request(\"POST\", path, body=body_bytes, headers=headers)\n resp = conn.getresponse()\n if resp.status != 200:\n err_body = resp.read().decode()[:500]\n conn.close()\n return resp.status, err_body, None\n if stream:\n return resp.status, resp, conn\n else:\n data = resp.read().decode()\n conn.close()\n return resp.status, data, None\n except Exception as e:\n conn.close()\n return None, str(e), None\n\n\nclass ProxyHandler(BaseHTTPRequestHandler):\n \"\"\"HTTP request handler for the bridge proxy.\"\"\"\n\n # Injected by the server factory\n timeout: int = 30 * 60\n multimodal: bool = False\n\n def handle_one_request(self) -> None:\n try:\n super().handle_one_request()\n except (ConnectionResetError, ConnectionAbortedError, BrokenPipeError):\n pass\n\n def log_message(self, format, *args):\n pass # Suppress default HTTP logging\n\n def _send_cors(self):\n self.send_header(\"Access-Control-Allow-Origin\", \"*\")\n self.send_header(\"Access-Control-Allow-Methods\", \"POST, OPTIONS, GET\")\n self.send_header(\"Access-Control-Allow-Headers\", \"Content-Type, Authorization\")\n\n def _json_response(self, data: dict, status: int = 200):\n body = json.dumps(data, ensure_ascii=False).encode(\"utf-8\")\n self.send_response(status)\n self._send_cors()\n self.send_header(\"Content-Type\", \"application/json\")\n self.send_header(\"Content-Length\", str(len(body)))\n self.end_headers()\n self.wfile.write(body)\n\n def _sse_response(self, generator):\n self.send_response(200)\n self._send_cors()\n self.send_header(\"Content-Type\", \"text/event-stream\")\n self.send_header(\"Cache-Control\", \"no-cache\")\n self.send_header(\"Connection\", \"keep-alive\")\n self.end_headers()\n try:\n for chunk in generator:\n self.wfile.write(\n chunk.encode(\"utf-8\") if isinstance(chunk, str) else chunk\n )\n self.wfile.flush()\n except (BrokenPipeError, ConnectionResetError):\n pass\n\n def do_OPTIONS(self):\n self.send_response(204)\n self._send_cors()\n self.end_headers()\n\n def do_GET(self):\n path = self.path.split(\"?\")[0]\n if path.endswith(\"/health\"):\n self._json_response({\n \"service\": \"codex-bridge\",\n \"status\": \"ok\",\n })\n elif path.endswith(\"/models\"):\n self._json_response({\n \"object\": \"list\",\n \"data\": [\n {\n \"id\": \"codex-bridge\",\n \"object\": \"model\",\n \"created\": 1700000000,\n \"owned_by\": \"codex-bridge\",\n }\n ],\n })\n else:\n self._json_response({\"error\": {\"message\": f\"not found: {path}\"}}, 404)\n\n def _client_api_key(self) -> str:\n \"\"\"Extract API key from incoming Authorization header.\"\"\"\n auth = self.headers.get(\"Authorization\", \"\")\n if auth.startswith(\"Bearer \"):\n return auth[7:]\n return \"\"\n\n def _parse_upstream(self) -> str | None:\n \"\"\"Extract upstream host+path from request path.\n\n Path format: /{upstream}/responses\n e.g. /api.deepseek.com/responses → api.deepseek.com\n /coding.dashscope.aliyuncs.com/v1/responses → coding.dashscope.aliyuncs.com/v1\n\n Handles three request-line forms:\n 1. Origin-form: /api.deepseek.com/responses\n 2. Absolute-form: http://localhost:10110/api.deepseek.com/responses\n 3. Codex-prepended: /localhost:10110/api.deepseek.com/responses\n (Codex naively splices host:port into the path)\n \"\"\"\n import re\n raw = self.path.split(\"?\")[0]\n # Step 1: urlparse strips scheme+host from absolute-form URIs\n path = urlparse(raw).path\n # Step 2: strip any leading /host:port/ that Codex may have\n # spliced into the path (e.g. /localhost:10110/…)\n path = re.sub(r'^/[^/]+:\\d+/', '/', path)\n log.info(f\"parse_upstream: raw={raw} → path={path}\")\n if not path.endswith(\"/responses\"):\n return None\n # Strip leading / and trailing /responses\n upstream = path[1:].removesuffix(\"/responses\")\n log.info(f\"parse_upstream: upstream={upstream}\")\n return upstream or None\n\n def _upstream_base_url(self, upstream: str) -> str:\n \"\"\"Build full upstream base URL from the host+path segment.\"\"\"\n return f\"https://{upstream}\"\n\n def do_POST(self):\n upstream = self._parse_upstream()\n if not upstream:\n self._json_response({\"error\": {\"message\": f\"not found: {self.path}\"}}, 404)\n return\n\n try:\n content_len = int(self.headers.get(\"Content-Length\", 0))\n raw = self.rfile.read(content_len).decode(\"utf-8\")\n body = json.loads(raw)\n except Exception as e:\n self._json_response({\"error\": {\"message\": str(e)}}, 400)\n return\n\n upstream_url = self._upstream_base_url(upstream)\n log.info(f\"upstream: {upstream_url}\")\n\n # Model comes from the incoming request (Codex sends it)\n req_model = body.get(\"model\", \"\")\n\n try:\n built = build_chat_body(\n body, upstream_url, req_model, self.multimodal\n )\n except Exception as e:\n log.err(f\"build: {e}\")\n self._json_response({\"error\": {\"message\": str(e)}}, 400)\n return\n\n chat_body = built[\"chat_body\"]\n stream = built[\"stream\"]\n\n log.info(f\"stream:{stream}\")\n if not stream:\n self._handle_non_stream(body, chat_body, upstream_url, req_model)\n else:\n self._handle_stream(body, chat_body, upstream_url, req_model)\n\n def _forward_headers(self) -> dict:\n \"\"\"Extract headers from the incoming request to forward upstream.\"\"\"\n h = {}\n for key, value in self.headers.items():\n h[key] = value\n return h\n\n def _handle_non_stream(self, body: dict, chat_body: dict, upstream_url: str, req_model: str) -> None:\n api_key = self._client_api_key()\n status, resp_body, conn = upstream_request(\n upstream_url, api_key, chat_body, self.timeout,\n client_headers=self._forward_headers(),\n )\n if status != 200:\n log.err(f\"Upstream {status}: {resp_body[:300]}\")\n self._json_response(\n {\n \"error\": {\n \"type\": \"upstream_error\",\n \"code\": f\"upstream_{status}\",\n \"message\": f\"Upstream {status}: {resp_body[:200]}\",\n }\n },\n 502 if status and status >= 500 else status or 502,\n )\n return\n try:\n completion = json.loads(resp_body)\n except Exception as e:\n log.err(f\"parse: {e}\")\n self._json_response({\"error\": {\"message\": str(e)}}, 502)\n return\n if (\n completion\n .get(\"choices\", [{}])[0]\n .get(\"message\", {})\n .get(\"reasoning_content\")\n ):\n remember_reasoning(\n session_key(body), [completion[\"choices\"][0][\"message\"]]\n )\n response = build_non_stream_response(completion, req_model)\n usg = completion.get(\"usage\")\n if usg:\n log.toks(\n usg.get(\"prompt_tokens\"),\n usg.get(\"completion_tokens\"),\n usg.get(\"total_tokens\"),\n )\n self._json_response(response, 200)\n\n def _handle_stream(self, body: dict, chat_body: dict, upstream_url: str, req_model: str) -> None:\n api_key = self._client_api_key()\n client_headers = self._forward_headers()\n\n def generate():\n translator = SseTranslator(req_model)\n conn = None\n try:\n status, resp, conn = upstream_request(\n upstream_url, api_key, chat_body, self.timeout,\n client_headers=client_headers, stream=True\n )\n if status != 200 or isinstance(resp, str):\n err_body = resp if isinstance(resp, str) else resp[:300]\n log.err(f\"Upstream {status}: {err_body}\")\n yield translator.error(f\"Upstream {status}: {err_body[:200]}\")\n return\n # Read in 4KB chunks; buffer as bytes to avoid splitting\n # multi-byte UTF-8 chars at chunk boundaries\n buf = b\"\"\n while True:\n chunk = resp.read(4096)\n if not chunk:\n break\n buf += chunk\n while b\"\\n\" in buf:\n line_bytes, buf = buf.split(b\"\\n\", 1)\n line = line_bytes.decode(\"utf-8\")\n if not line.startswith(\"data: \"):\n continue\n json_str = line[6:].strip()\n if json_str == \"[DONE]\":\n continue\n try:\n parsed = json.loads(json_str)\n result = translator.feed(parsed)\n if result:\n yield result\n except (json.JSONDecodeError, ValueError):\n pass\n # Flush remaining buffer\n for line_bytes in buf.split(b\"\\n\"):\n if not line_bytes:\n continue\n line = line_bytes.decode(\"utf-8\")\n if not line.startswith(\"data: \"):\n continue\n json_str = line[6:].strip()\n if json_str == \"[DONE]\":\n continue\n try:\n parsed = json.loads(json_str)\n result = translator.feed(parsed)\n if result:\n yield result\n except (json.JSONDecodeError, ValueError):\n pass\n if translator.reasoning_so_far:\n remember_reasoning(\n session_key(body),\n [\n {\n \"role\": \"assistant\",\n \"content\": translator.content_so_far,\n \"reasoning_content\": translator.reasoning_so_far,\n }\n ],\n )\n yield translator.done(None)\n except Exception as e:\n log.err(f\"upstream: {e}\")\n yield translator.error(str(e))\n finally:\n if conn:\n conn.close()\n\n self._sse_response(generate())\n\n\ndef create_server(\n port: int,\n timeout: int,\n multimodal: bool,\n) -> ThreadingHTTPServer:\n \"\"\"Create and return a configured ThreadingHTTPServer.\"\"\"\n server = ThreadingHTTPServer((\"127.0.0.1\", port), ProxyHandler)\n ProxyHandler.timeout = timeout\n ProxyHandler.multimodal = multimodal\n return server\n"}}}} |