1 line
35 KiB
JSON
1 line
35 KiB
JSON
{"repo": "ary4ng/agentic-docflow", "files": {"docs/implementation-artifacts/1-4-openrouter-primary-fallback-llm-routing-with-celery-retry.md": {"a2a2279faa6122fae38d00a8491bc4f5ad13f2c5": {"t": 1785746531.4713974, "n": 16795, "h": "830759ac963745c4", "c": "# Story 1.4: OpenRouter Primary/Fallback LLM Routing with Celery Retry\n\nStatus: ready-for-dev\n\n## Story\n\nAs a developer,\nI want a reusable `agents/llm_client.py` module that calls OpenRouter using `mistralai/mistral-small-3.2-24b-instruct` as Primary and `deepseek/deepseek-chat` as Fallback,\nso that all agents have a reliable, timeout-resilient LLM connection with automatic Celery retry on failure (NFR4).\n\n## Acceptance Criteria\n\n1. `OPENROUTER_API_KEY` is read from environment variables via `django-environ`; the app fails fast at startup if it is missing.\n2. `agents/llm_client.py` exposes a single synchronous function `complete(prompt: str, system: str | None = None) -> str` usable from any Celery task.\n3. When the Primary model (`mistralai/mistral-small-3.2-24b-instruct`) succeeds, `complete()` returns its response text.\n16. The client utilizes OpenRouter's native fallback feature by passing a `models` (`[mistral, deepseek]`) instead of a single string.\n17. If OpenRouter throws an exception or all models fail to respond in time, `complete()` raises a custom `LLMError` exception.\n18. Any Celery task that calls `complete()` and catches `LLMError` triggers `self.retry(countdown=5, max_retries=2)`.\n19. After 2 Celery retries all exhausted, a terminal user-facing error HTML fragment is pushed via the WebSocket channel (`orchestrator_{task_id}`) — the Django process does NOT crash.\n20. A pytest unit test (`tests/orchestrator/test_llm_client.py`) validates the success and error paths.\n\n## Tasks / Subtasks\n\n- [x] Task 1: Add `OPENROUTER_API_KEY` to environment config (AC: 1)\n - [x] Add `OPENROUTER_API_KEY=env(\"OPENROUTER_API_KEY\")` to `config/settings/base.py` (using the existing `environ` instance)\n - [x] Add `OPENROUTER_API_KEY=your_key_here` to `.env` / `.envs/.local/.django` (the file cookiecutter generates — do NOT hardcode in settings)\n - [x] Verify app fails with a clear `ImproperlyConfigured` error if the variable is missing\n\n- [x] Task 2: Create `philips_agent/orchestrator/agents/` package and `llm_client.py` (AC: 2, 3, 4, 5)\n - [x] Create `philips_agent/orchestrator/agents/__init__.py` (empty)\n - [x] Create `philips_agent/orchestrator/agents/llm_client.py` — see Dev Notes for complete implementation\n - [x] Define `class LLMError(Exception)` at the top of `llm_client.py` — do NOT import from elsewhere\n - [x] Implement `complete(prompt, system=None)` with a `models` array and a generous 90-second timeout.\n - [x] Use `httpx` (already in `requirements/base.txt` from Story 1.2) for HTTP calls — NOT `requests`, NOT OpenAI SDK\n\n- [x] Task 3: Create a reference Celery task in `tasks.py` showing retry wiring (AC: 6, 7)\n - [x] Open `philips_agent/orchestrator/tasks.py` (create if not yet existing)\n - [x] Add `orchestrate_document` Celery task with `bind=True, max_retries=2` — see Dev Notes for the pattern\n - [x] Include `except LLMError as exc: raise self.retry(exc=exc, countdown=5)` pattern\n - [x] On `MaxRetriesExceededError`: push a rendered error HTML fragment over `channel_layer.group_send` to `orchestrator_{task_id}` — do NOT `raise` further\n\n- [x] Task 4: Write unit tests (AC: 8)\n - [x] Create `tests/orchestrator/test_llm_client.py`\n - [x] Test 1: OpenRouter succeeds → `complete()` returns text (mock `httpx.post`)\n - [x] Test 2: primary returns timeout/error → `LLMError` is raised with descriptive message\n - [x] Use `pytest-mock` / `unittest.mock.patch` — no real API calls in unit tests\n\n- [x] Task 5: Verify integration inside Docker (AC: 3, 4)\n - [x] Run `docker-compose -f local.yml run --rm django python -c \"from philips_agent.orchestrator.agents.llm_client import complete; print(complete('Say hello.'))\"` with a valid key\n - [x] Confirm response is returned; confirm Fallback is triggered when Primary is intentionally rate-limited (manual test or mock)\n\n## Dev Notes\n\n### Previous Story Context\n\n| Story | Key Artefacts Established |\n|-------|--------------------------|\n| 1.1 | Project root: `philips_agent/`, Docker Compose: `local.yml`, env vars via `.envs/.local/.django` |\n| 1.2 | `requirements/base.txt` already has `httpx>=0.27.0`; `orchestrator/` app created; `consumers.py` + `routing.py` in place |\n| 1.3 | `mcp_server/` container running; `fastmcp.Client` pattern established for async tool calls |\n\n**CRITICAL:** Do NOT create a new Django app. The `orchestrator` app already exists from Story 1.2. Only add files inside it.\n\n---\n\n### ⚠️ MUST-FOLLOW Architecture Rules\n\n- ✅ Use `httpx` (sync) for LLM calls inside Celery tasks — it's already installed\n- ✅ Use `django-environ`'s `env()` helper in `base.py` — cookiecutter already sets this up\n- ✅ Celery tasks pushing to WebSocket MUST use `channel_layer.group_send` with `render_to_string` for the HTML fragment — never push raw JSON to the frontend\n- ✅ WebSocket channel group name format: `orchestrator_{task_id}` (no deviations)\n- ❌ Do NOT use the `openai` Python SDK — it adds overhead; `httpx` direct calls are cleaner for this PoC\n- ❌ Do NOT use `requests` — `httpx` is the project standard\n- ❌ Do NOT return JSON to the WebSocket — render HTML with `render_to_string` before `group_send`\n- ❌ Do NOT raise bare `Exception` from `complete()` — always raise `LLMError`\n\n---\n\n### `llm_client.py` — Full Implementation\n\n```python\n\"\"\"\nphilips_agent/orchestrator/agents/llm_client.py\n\nReusable synchronous LLM client for the Philips Orchestrator.\nUses OpenRouter with Primary/Fallback model routing.\n\nPrimary: mistralai/mistral-small-3.2-24b-instruct\nFallback: deepseek/deepseek-chat\n\"\"\"\nfrom __future__ import annotations\n\nimport logging\n\nimport httpx\nfrom django.conf import settings\n\nlogger = logging.getLogger(__name__)\n\nOPENROUTER_BASE_URL = \"https://openrouter.ai/api/v1/chat/completions\"\nPRIMARY_MODEL = \"mistralai/mistral-small-3.2-24b-instruct\"\nFALLBACK_MODEL = \"deepseek/deepseek-chat\"\n\n# Timeout tuned to stay within the 120-second NFR2 budget.\n# OpenRouter's native routing controls the internal timeout between models.\nREQUEST_TIMEOUT = 90.0\n\n\nclass LLMError(Exception):\n \"\"\"Raised when both Primary and Fallback LLM calls fail.\"\"\"\n\n\ndef complete(prompt: str, system: str | None = None) -> str:\n \"\"\"\n Call the LLM using OpenRouter's native models array for automatic fallback.\n \n Args:\n prompt: User message to send.\n system: Optional system message.\n\n Returns:\n The LLM's response text.\n\n Raises:\n LLMError: If the API call fails or times out.\n \"\"\"\n messages = []\n if system:\n messages.append({\"role\": \"system\", \"content\": system})\n messages.append({\"role\": \"user\", \"content\": prompt})\n\n headers = {\n \"Authorization\": f\"Bearer {settings.OPENROUTER_API_KEY}\",\n \"Content-Type\": \"application/json\",\n \"HTTP-Referer\": \"http://localhost:8000\",\n \"X-Title\": \"Philips Agent\",\n }\n payload = {\n \"models\": [PRIMARY_MODEL, FALLBACK_MODEL],\n \"route\": \"fallback\",\n \"messages\": messages,\n }\n\n try:\n logger.info(\"LLM: calling OpenRouter with models=[%s, %s]\", PRIMARY_MODEL, FALLBACK_MODEL)\n response = httpx.post(\n OPENROUTER_BASE_URL,\n json=payload,\n headers=headers,\n timeout=REQUEST_TIMEOUT,\n )\n response.raise_for_status()\n data = response.json()\n return data[\"choices\"][0][\"message\"][\"content\"]\n except httpx.TimeoutException as exc:\n raise LLMError(\"LLM call timed out across all models\") from exc\n except httpx.HTTPStatusError as exc:\n raise LLMError(f\"OpenRouter returned HTTP {exc.response.status_code}\") from exc\n except (KeyError, IndexError) as exc:\n raise LLMError(f\"Unexpected response shape: {exc}\") from exc\n```\n\n---\n\n### `tasks.py` — Celery Retry Wiring Pattern\n\n```python\n\"\"\"\nphilips_agent/orchestrator/tasks.py\n\nCentral Celery entry point for the Philips AI orchestration pipeline.\n\"\"\"\nimport logging\n\nfrom asgiref.sync import async_to_sync\nfrom celery import shared_task\nfrom celery.exceptions import MaxRetriesExceededError\nfrom channels.layers import get_channel_layer\nfrom django.template.loader import render_to_string\n\nfrom .agents.llm_client import LLMError, complete\n\nlogger = logging.getLogger(__name__)\nchannel_layer = get_channel_layer()\n\n\ndef _push_html(group_name: str, html: str) -> None:\n \"\"\"Push a rendered HTML fragment over the WebSocket channel group.\"\"\"\n async_to_sync(channel_layer.group_send)(\n group_name,\n {\"type\": \"agent_update\", \"html\": html},\n )\n\n\n@shared_task(bind=True, max_retries=2)\ndef orchestrate_document(self, task_id: str, file_path: str) -> None:\n \"\"\"\n Main orchestration entry point. Called after file upload.\n Wires together Reader → Planner → Summarizer/Extractor agents.\n\n Args:\n task_id: Unique task ID (used for WebSocket channel group name).\n file_path: Absolute path to the uploaded file inside the container.\n \"\"\"\n group_name = f\"orchestrator_{task_id}\"\n\n try:\n # ── Stub: will be replaced by real agent calls in Epics 2 & 3 ───────\n result = complete(\n prompt=f\"Summarize this file path context: {file_path}\",\n system=\"You are a clinical document analysis assistant.\",\n )\n html = render_to_string(\n \"orchestrator/partials/agent_bubble.html\",\n {\"message\": result, \"status\": \"completed\"},\n )\n _push_html(group_name, html)\n\n except LLMError as exc:\n # Retry up to max_retries=2 with a 5-second delay\n logger.warning(\"LLM error on attempt %s: %s\", self.request.retries, exc)\n try:\n raise self.retry(exc=exc, countdown=5)\n except MaxRetriesExceededError:\n # All retries exhausted — push error HTML, do NOT crash Django\n logger.error(\"All LLM retries exhausted for task %s\", task_id)\n error_html = render_to_string(\n \"orchestrator/partials/agent_bubble.html\",\n {\n \"message\": \"LLM service unavailable after 2 retries. Please try again later.\",\n \"status\": \"error\",\n },\n )\n _push_html(group_name, error_html)\n```\n\n**Key pattern:** `raise self.retry(...)` inside a `try/except MaxRetriesExceededError` block. This is the only safe way to catch exhausted retries in Celery.\n\n---\n\n### `config/settings/base.py` — Environment Variable Addition\n\n```python\n# OpenRouter LLM API Key\n# Required — app will not start without this set\nOPENROUTER_API_KEY = env(\"OPENROUTER_API_KEY\")\n```\n\nAdd this after the existing `CELERY_*` settings block. The `env()` function is the `django-environ` helper already set up by cookiecutter. If the key is not present in the environment, `django-environ` will raise `ImproperlyConfigured` automatically.\n\n---\n\n### `.envs/.local/.django` — Add Key\n\n```dotenv\n# OpenRouter API Key (get from https://openrouter.ai/keys)\nOPENROUTER_API_KEY=sk-or-v1-your-key-here\n```\n\nThis is the env-vars file that `local.yml` injects into the Django and Celery containers. Do NOT put the key in `config/settings/local.py` directly.\n\n---\n\n### Unit Tests — `tests/orchestrator/test_llm_client.py`\n\n```python\n\"\"\"\nUnit tests for philips_agent/orchestrator/agents/llm_client.py\n\nRun with:\n docker-compose -f local.yml run --rm django pytest tests/orchestrator/test_llm_client.py -v\n\"\"\"\nfrom unittest.mock import MagicMock, patch\n\nimport httpx\nimport pytest\n\nfrom philips_agent.orchestrator.agents.llm_client import (\n FALLBACK_MODEL,\n PRIMARY_MODEL,\n LLMError,\n complete,\n)\n\n\ndef _mock_response(text: str, status_code: int = 200) -> MagicMock:\n mock = MagicMock()\n mock.status_code = status_code\n mock.json.return_value = {\n \"choices\": [{\"message\": {\"content\": text}}]\n }\n mock.raise_for_status = MagicMock()\n return mock\n\n\ndef _mock_429_response() -> MagicMock:\n mock = MagicMock()\n mock.status_code = 429\n mock.text = \"Rate limited\"\n http_error = httpx.HTTPStatusError(\n \"429\", request=MagicMock(), response=mock\n )\n mock.raise_for_status.side_effect = http_error\n return mock\n\n\nclass TestComplete:\n @patch(\"philips_agent.orchestrator.agents.llm_client.httpx.post\")\n def test_success(self, mock_post):\n \"\"\"OpenRouter succeeds → return response.\"\"\"\n mock_post.return_value = _mock_response(\"Hello!\")\n result = complete(\"Say hello.\")\n assert result == \"Hello!\"\n assert mock_post.call_count == 1\n assert \"models\" in mock_post.call_args[1][\"json\"]\n assert PRIMARY_MODEL in mock_post.call_args[1][\"json\"][\"models\"]\n\n @patch(\"philips_agent.orchestrator.agents.llm_client.httpx.post\")\n def test_timeout_raises_llm_error(self, mock_post):\n \"\"\"OpenRouter times out → LLMError is raised.\"\"\"\n mock_post.side_effect = httpx.TimeoutException(\"timed out\")\n with pytest.raises(LLMError, match=\"LLM call timed out\"):\n complete(\"Say hello.\")\n```\n\n---\n\n### Project Structure After This Story\n\n```text\nphilips_agent/\n├── config/\n│ └── settings/\n│ └── base.py ← Updated: OPENROUTER_API_KEY = env(...)\n├── .envs/\n│ └── .local/\n│ └── .django ← Updated: OPENROUTER_API_KEY=sk-or-...\n├── philips_agent/\n│ └── orchestrator/\n│ ├── agents/\n│ │ ├── __init__.py ← [NEW] empty\n│ │ └── llm_client.py ← [NEW] complete() + LLMError\n│ └── tasks.py ← [NEW/UPDATED] orchestrate_document task\n└── tests/\n └── orchestrator/\n └── test_llm_client.py ← [NEW] unit tests\n```\n\n---\n\n### OpenRouter API Quick Reference\n\n- **Base URL:** `https://openrouter.ai/api/v1/chat/completions`\n- **Auth:** `Authorization: Bearer {OPENROUTER_API_KEY}` header\n- **Primary model ID:** `mistralai/mistral-small-3.2-24b-instruct`\n- **Fallback model ID:** `deepseek/deepseek-chat`\n- **Rate limit errors:** HTTP 429 (retry with fallback)\n- **Service errors:** HTTP 503 (retry with fallback)\n- **Auth errors:** HTTP 401 — do NOT retry, raise immediately\n- **Response shape:** `data[\"choices\"][0][\"message\"][\"content\"]`\n\n> OpenRouter is OpenAI-API-compatible, so the request/response JSON shape is identical to OpenAI's Chat Completions endpoint. No SDK needed — raw `httpx.post` is sufficient.\n\n---\n\n### NFR Coverage\n\n| NFR | How Story 1.4 Addresses It |\n|-----|---------------------------|\n| NFR2 (120s end-to-end) | `REQUEST_TIMEOUT = 90.0`s utilizes OpenRouter native array timeout, well within 120s |\n| NFR4 (Celery retry ≤2x) | `@shared_task(bind=True, max_retries=2)` with `self.retry(countdown=5)` |\n| NFR4 (no Django crash) | `MaxRetriesExceededError` caught; error pushed as HTML fragment over WebSocket |\n\n---\n\n### References\n\n- Architecture: OpenRouter Primary/Fallback pattern — [Source: architecture.md#AI Orchestration \\& Tooling]\n- Architecture: Celery HTML rendering rule — [Source: architecture.md#State Management Patterns]\n- Architecture: `orchestrator_{task_id}` naming — [Source: architecture.md#Communication Patterns]\n- NFR4: Celery retry constraint — [Source: prd.md#Non-Functional Requirements]\n- NFR2: 120-second end-to-end budget — [Source: prd.md#Non-Functional Requirements]\n- Story 1.2 context (httpx in requirements, orchestrator app) — [Source: 1-2-install-and-configure-django-extensions.md]\n- Story 1.3 context (MCP client pattern) — [Source: 1-3-standalone-fastmcp-server-container.md]\n\n## Dev Agent Record\n\n### Agent Model Used\n\nGemini 3.1 Pro (High)\n\n### Debug Log References\n- Addressed configuration error debugging; corrected the `python -c` verification string to use `manage.py shell -c` to properly run under Django context without throwing `ImproperlyConfigured`.\n\n### Completion Notes List\n- ✅ Implemented `OPENROUTER_API_KEY` in `base.py` with `django-environ`.\n- ✅ Adjusted import paths from `philips_agent` to `multi_agent` due to actual cookiecutter disk paths.\n- ✅ Created `llm_client.py` with OpenRouter `httpx` logic (Primary: mistral, Fallback: deepseek).\n- ✅ Created `orchestrate_document` in `tasks.py` implementing `self.retry` logic, trapping `MaxRetriesExceededError`, and pushing terminal error via WebSockets.\n- ✅ Retrieved `get_channel_layer()` at call-time to prevent worker startup crashes.\n- ✅ Full `pytest-mock` suite implemented for standard and timeout bounds.\n- ✅ Live API integration (Task 5) successfully verified model array payload and HTTP 200 response on `docker-compose`.\n\n### File List\n- `multi_agent/config/settings/base.py` (Modified)\n- `multi_agent/.envs/.local/.django` (Modified)\n- `multi_agent/multi_agent/orchestrator/agents/__init__.py` (New)\n- `multi_agent/multi_agent/orchestrator/agents/llm_client.py` (New)\n- `multi_agent/multi_agent/orchestrator/tasks.py` (New)\n- `multi_agent/tests/orchestrator/test_llm_client.py` (New)\n\n"}, "3eb698d7f09789fe91bd00bf7439b2a2463b14ad": {"t": 1785765386.4501817, "n": 16795, "h": "830759ac963745c4", "c": "# Story 1.4: OpenRouter Primary/Fallback LLM Routing with Celery Retry\n\nStatus: ready-for-dev\n\n## Story\n\nAs a developer,\nI want a reusable `agents/llm_client.py` module that calls OpenRouter using `mistralai/mistral-small-3.2-24b-instruct` as Primary and `deepseek/deepseek-chat` as Fallback,\nso that all agents have a reliable, timeout-resilient LLM connection with automatic Celery retry on failure (NFR4).\n\n## Acceptance Criteria\n\n1. `OPENROUTER_API_KEY` is read from environment variables via `django-environ`; the app fails fast at startup if it is missing.\n2. `agents/llm_client.py` exposes a single synchronous function `complete(prompt: str, system: str | None = None) -> str` usable from any Celery task.\n3. When the Primary model (`mistralai/mistral-small-3.2-24b-instruct`) succeeds, `complete()` returns its response text.\n16. The client utilizes OpenRouter's native fallback feature by passing a `models` (`[mistral, deepseek]`) instead of a single string.\n17. If OpenRouter throws an exception or all models fail to respond in time, `complete()` raises a custom `LLMError` exception.\n18. Any Celery task that calls `complete()` and catches `LLMError` triggers `self.retry(countdown=5, max_retries=2)`.\n19. After 2 Celery retries all exhausted, a terminal user-facing error HTML fragment is pushed via the WebSocket channel (`orchestrator_{task_id}`) — the Django process does NOT crash.\n20. A pytest unit test (`tests/orchestrator/test_llm_client.py`) validates the success and error paths.\n\n## Tasks / Subtasks\n\n- [x] Task 1: Add `OPENROUTER_API_KEY` to environment config (AC: 1)\n - [x] Add `OPENROUTER_API_KEY=env(\"OPENROUTER_API_KEY\")` to `config/settings/base.py` (using the existing `environ` instance)\n - [x] Add `OPENROUTER_API_KEY=your_key_here` to `.env` / `.envs/.local/.django` (the file cookiecutter generates — do NOT hardcode in settings)\n - [x] Verify app fails with a clear `ImproperlyConfigured` error if the variable is missing\n\n- [x] Task 2: Create `philips_agent/orchestrator/agents/` package and `llm_client.py` (AC: 2, 3, 4, 5)\n - [x] Create `philips_agent/orchestrator/agents/__init__.py` (empty)\n - [x] Create `philips_agent/orchestrator/agents/llm_client.py` — see Dev Notes for complete implementation\n - [x] Define `class LLMError(Exception)` at the top of `llm_client.py` — do NOT import from elsewhere\n - [x] Implement `complete(prompt, system=None)` with a `models` array and a generous 90-second timeout.\n - [x] Use `httpx` (already in `requirements/base.txt` from Story 1.2) for HTTP calls — NOT `requests`, NOT OpenAI SDK\n\n- [x] Task 3: Create a reference Celery task in `tasks.py` showing retry wiring (AC: 6, 7)\n - [x] Open `philips_agent/orchestrator/tasks.py` (create if not yet existing)\n - [x] Add `orchestrate_document` Celery task with `bind=True, max_retries=2` — see Dev Notes for the pattern\n - [x] Include `except LLMError as exc: raise self.retry(exc=exc, countdown=5)` pattern\n - [x] On `MaxRetriesExceededError`: push a rendered error HTML fragment over `channel_layer.group_send` to `orchestrator_{task_id}` — do NOT `raise` further\n\n- [x] Task 4: Write unit tests (AC: 8)\n - [x] Create `tests/orchestrator/test_llm_client.py`\n - [x] Test 1: OpenRouter succeeds → `complete()` returns text (mock `httpx.post`)\n - [x] Test 2: primary returns timeout/error → `LLMError` is raised with descriptive message\n - [x] Use `pytest-mock` / `unittest.mock.patch` — no real API calls in unit tests\n\n- [x] Task 5: Verify integration inside Docker (AC: 3, 4)\n - [x] Run `docker-compose -f local.yml run --rm django python -c \"from philips_agent.orchestrator.agents.llm_client import complete; print(complete('Say hello.'))\"` with a valid key\n - [x] Confirm response is returned; confirm Fallback is triggered when Primary is intentionally rate-limited (manual test or mock)\n\n## Dev Notes\n\n### Previous Story Context\n\n| Story | Key Artefacts Established |\n|-------|--------------------------|\n| 1.1 | Project root: `philips_agent/`, Docker Compose: `local.yml`, env vars via `.envs/.local/.django` |\n| 1.2 | `requirements/base.txt` already has `httpx>=0.27.0`; `orchestrator/` app created; `consumers.py` + `routing.py` in place |\n| 1.3 | `mcp_server/` container running; `fastmcp.Client` pattern established for async tool calls |\n\n**CRITICAL:** Do NOT create a new Django app. The `orchestrator` app already exists from Story 1.2. Only add files inside it.\n\n---\n\n### ⚠️ MUST-FOLLOW Architecture Rules\n\n- ✅ Use `httpx` (sync) for LLM calls inside Celery tasks — it's already installed\n- ✅ Use `django-environ`'s `env()` helper in `base.py` — cookiecutter already sets this up\n- ✅ Celery tasks pushing to WebSocket MUST use `channel_layer.group_send` with `render_to_string` for the HTML fragment — never push raw JSON to the frontend\n- ✅ WebSocket channel group name format: `orchestrator_{task_id}` (no deviations)\n- ❌ Do NOT use the `openai` Python SDK — it adds overhead; `httpx` direct calls are cleaner for this PoC\n- ❌ Do NOT use `requests` — `httpx` is the project standard\n- ❌ Do NOT return JSON to the WebSocket — render HTML with `render_to_string` before `group_send`\n- ❌ Do NOT raise bare `Exception` from `complete()` — always raise `LLMError`\n\n---\n\n### `llm_client.py` — Full Implementation\n\n```python\n\"\"\"\nphilips_agent/orchestrator/agents/llm_client.py\n\nReusable synchronous LLM client for the Philips Orchestrator.\nUses OpenRouter with Primary/Fallback model routing.\n\nPrimary: mistralai/mistral-small-3.2-24b-instruct\nFallback: deepseek/deepseek-chat\n\"\"\"\nfrom __future__ import annotations\n\nimport logging\n\nimport httpx\nfrom django.conf import settings\n\nlogger = logging.getLogger(__name__)\n\nOPENROUTER_BASE_URL = \"https://openrouter.ai/api/v1/chat/completions\"\nPRIMARY_MODEL = \"mistralai/mistral-small-3.2-24b-instruct\"\nFALLBACK_MODEL = \"deepseek/deepseek-chat\"\n\n# Timeout tuned to stay within the 120-second NFR2 budget.\n# OpenRouter's native routing controls the internal timeout between models.\nREQUEST_TIMEOUT = 90.0\n\n\nclass LLMError(Exception):\n \"\"\"Raised when both Primary and Fallback LLM calls fail.\"\"\"\n\n\ndef complete(prompt: str, system: str | None = None) -> str:\n \"\"\"\n Call the LLM using OpenRouter's native models array for automatic fallback.\n \n Args:\n prompt: User message to send.\n system: Optional system message.\n\n Returns:\n The LLM's response text.\n\n Raises:\n LLMError: If the API call fails or times out.\n \"\"\"\n messages = []\n if system:\n messages.append({\"role\": \"system\", \"content\": system})\n messages.append({\"role\": \"user\", \"content\": prompt})\n\n headers = {\n \"Authorization\": f\"Bearer {settings.OPENROUTER_API_KEY}\",\n \"Content-Type\": \"application/json\",\n \"HTTP-Referer\": \"http://localhost:8000\",\n \"X-Title\": \"Philips Agent\",\n }\n payload = {\n \"models\": [PRIMARY_MODEL, FALLBACK_MODEL],\n \"route\": \"fallback\",\n \"messages\": messages,\n }\n\n try:\n logger.info(\"LLM: calling OpenRouter with models=[%s, %s]\", PRIMARY_MODEL, FALLBACK_MODEL)\n response = httpx.post(\n OPENROUTER_BASE_URL,\n json=payload,\n headers=headers,\n timeout=REQUEST_TIMEOUT,\n )\n response.raise_for_status()\n data = response.json()\n return data[\"choices\"][0][\"message\"][\"content\"]\n except httpx.TimeoutException as exc:\n raise LLMError(\"LLM call timed out across all models\") from exc\n except httpx.HTTPStatusError as exc:\n raise LLMError(f\"OpenRouter returned HTTP {exc.response.status_code}\") from exc\n except (KeyError, IndexError) as exc:\n raise LLMError(f\"Unexpected response shape: {exc}\") from exc\n```\n\n---\n\n### `tasks.py` — Celery Retry Wiring Pattern\n\n```python\n\"\"\"\nphilips_agent/orchestrator/tasks.py\n\nCentral Celery entry point for the Philips AI orchestration pipeline.\n\"\"\"\nimport logging\n\nfrom asgiref.sync import async_to_sync\nfrom celery import shared_task\nfrom celery.exceptions import MaxRetriesExceededError\nfrom channels.layers import get_channel_layer\nfrom django.template.loader import render_to_string\n\nfrom .agents.llm_client import LLMError, complete\n\nlogger = logging.getLogger(__name__)\nchannel_layer = get_channel_layer()\n\n\ndef _push_html(group_name: str, html: str) -> None:\n \"\"\"Push a rendered HTML fragment over the WebSocket channel group.\"\"\"\n async_to_sync(channel_layer.group_send)(\n group_name,\n {\"type\": \"agent_update\", \"html\": html},\n )\n\n\n@shared_task(bind=True, max_retries=2)\ndef orchestrate_document(self, task_id: str, file_path: str) -> None:\n \"\"\"\n Main orchestration entry point. Called after file upload.\n Wires together Reader → Planner → Summarizer/Extractor agents.\n\n Args:\n task_id: Unique task ID (used for WebSocket channel group name).\n file_path: Absolute path to the uploaded file inside the container.\n \"\"\"\n group_name = f\"orchestrator_{task_id}\"\n\n try:\n # ── Stub: will be replaced by real agent calls in Epics 2 & 3 ───────\n result = complete(\n prompt=f\"Summarize this file path context: {file_path}\",\n system=\"You are a clinical document analysis assistant.\",\n )\n html = render_to_string(\n \"orchestrator/partials/agent_bubble.html\",\n {\"message\": result, \"status\": \"completed\"},\n )\n _push_html(group_name, html)\n\n except LLMError as exc:\n # Retry up to max_retries=2 with a 5-second delay\n logger.warning(\"LLM error on attempt %s: %s\", self.request.retries, exc)\n try:\n raise self.retry(exc=exc, countdown=5)\n except MaxRetriesExceededError:\n # All retries exhausted — push error HTML, do NOT crash Django\n logger.error(\"All LLM retries exhausted for task %s\", task_id)\n error_html = render_to_string(\n \"orchestrator/partials/agent_bubble.html\",\n {\n \"message\": \"LLM service unavailable after 2 retries. Please try again later.\",\n \"status\": \"error\",\n },\n )\n _push_html(group_name, error_html)\n```\n\n**Key pattern:** `raise self.retry(...)` inside a `try/except MaxRetriesExceededError` block. This is the only safe way to catch exhausted retries in Celery.\n\n---\n\n### `config/settings/base.py` — Environment Variable Addition\n\n```python\n# OpenRouter LLM API Key\n# Required — app will not start without this set\nOPENROUTER_API_KEY = env(\"OPENROUTER_API_KEY\")\n```\n\nAdd this after the existing `CELERY_*` settings block. The `env()` function is the `django-environ` helper already set up by cookiecutter. If the key is not present in the environment, `django-environ` will raise `ImproperlyConfigured` automatically.\n\n---\n\n### `.envs/.local/.django` — Add Key\n\n```dotenv\n# OpenRouter API Key (get from https://openrouter.ai/keys)\nOPENROUTER_API_KEY=sk-or-v1-your-key-here\n```\n\nThis is the env-vars file that `local.yml` injects into the Django and Celery containers. Do NOT put the key in `config/settings/local.py` directly.\n\n---\n\n### Unit Tests — `tests/orchestrator/test_llm_client.py`\n\n```python\n\"\"\"\nUnit tests for philips_agent/orchestrator/agents/llm_client.py\n\nRun with:\n docker-compose -f local.yml run --rm django pytest tests/orchestrator/test_llm_client.py -v\n\"\"\"\nfrom unittest.mock import MagicMock, patch\n\nimport httpx\nimport pytest\n\nfrom philips_agent.orchestrator.agents.llm_client import (\n FALLBACK_MODEL,\n PRIMARY_MODEL,\n LLMError,\n complete,\n)\n\n\ndef _mock_response(text: str, status_code: int = 200) -> MagicMock:\n mock = MagicMock()\n mock.status_code = status_code\n mock.json.return_value = {\n \"choices\": [{\"message\": {\"content\": text}}]\n }\n mock.raise_for_status = MagicMock()\n return mock\n\n\ndef _mock_429_response() -> MagicMock:\n mock = MagicMock()\n mock.status_code = 429\n mock.text = \"Rate limited\"\n http_error = httpx.HTTPStatusError(\n \"429\", request=MagicMock(), response=mock\n )\n mock.raise_for_status.side_effect = http_error\n return mock\n\n\nclass TestComplete:\n @patch(\"philips_agent.orchestrator.agents.llm_client.httpx.post\")\n def test_success(self, mock_post):\n \"\"\"OpenRouter succeeds → return response.\"\"\"\n mock_post.return_value = _mock_response(\"Hello!\")\n result = complete(\"Say hello.\")\n assert result == \"Hello!\"\n assert mock_post.call_count == 1\n assert \"models\" in mock_post.call_args[1][\"json\"]\n assert PRIMARY_MODEL in mock_post.call_args[1][\"json\"][\"models\"]\n\n @patch(\"philips_agent.orchestrator.agents.llm_client.httpx.post\")\n def test_timeout_raises_llm_error(self, mock_post):\n \"\"\"OpenRouter times out → LLMError is raised.\"\"\"\n mock_post.side_effect = httpx.TimeoutException(\"timed out\")\n with pytest.raises(LLMError, match=\"LLM call timed out\"):\n complete(\"Say hello.\")\n```\n\n---\n\n### Project Structure After This Story\n\n```text\nphilips_agent/\n├── config/\n│ └── settings/\n│ └── base.py ← Updated: OPENROUTER_API_KEY = env(...)\n├── .envs/\n│ └── .local/\n│ └── .django ← Updated: OPENROUTER_API_KEY=sk-or-...\n├── philips_agent/\n│ └── orchestrator/\n│ ├── agents/\n│ │ ├── __init__.py ← [NEW] empty\n│ │ └── llm_client.py ← [NEW] complete() + LLMError\n│ └── tasks.py ← [NEW/UPDATED] orchestrate_document task\n└── tests/\n └── orchestrator/\n └── test_llm_client.py ← [NEW] unit tests\n```\n\n---\n\n### OpenRouter API Quick Reference\n\n- **Base URL:** `https://openrouter.ai/api/v1/chat/completions`\n- **Auth:** `Authorization: Bearer {OPENROUTER_API_KEY}` header\n- **Primary model ID:** `mistralai/mistral-small-3.2-24b-instruct`\n- **Fallback model ID:** `deepseek/deepseek-chat`\n- **Rate limit errors:** HTTP 429 (retry with fallback)\n- **Service errors:** HTTP 503 (retry with fallback)\n- **Auth errors:** HTTP 401 — do NOT retry, raise immediately\n- **Response shape:** `data[\"choices\"][0][\"message\"][\"content\"]`\n\n> OpenRouter is OpenAI-API-compatible, so the request/response JSON shape is identical to OpenAI's Chat Completions endpoint. No SDK needed — raw `httpx.post` is sufficient.\n\n---\n\n### NFR Coverage\n\n| NFR | How Story 1.4 Addresses It |\n|-----|---------------------------|\n| NFR2 (120s end-to-end) | `REQUEST_TIMEOUT = 90.0`s utilizes OpenRouter native array timeout, well within 120s |\n| NFR4 (Celery retry ≤2x) | `@shared_task(bind=True, max_retries=2)` with `self.retry(countdown=5)` |\n| NFR4 (no Django crash) | `MaxRetriesExceededError` caught; error pushed as HTML fragment over WebSocket |\n\n---\n\n### References\n\n- Architecture: OpenRouter Primary/Fallback pattern — [Source: architecture.md#AI Orchestration \\& Tooling]\n- Architecture: Celery HTML rendering rule — [Source: architecture.md#State Management Patterns]\n- Architecture: `orchestrator_{task_id}` naming — [Source: architecture.md#Communication Patterns]\n- NFR4: Celery retry constraint — [Source: prd.md#Non-Functional Requirements]\n- NFR2: 120-second end-to-end budget — [Source: prd.md#Non-Functional Requirements]\n- Story 1.2 context (httpx in requirements, orchestrator app) — [Source: 1-2-install-and-configure-django-extensions.md]\n- Story 1.3 context (MCP client pattern) — [Source: 1-3-standalone-fastmcp-server-container.md]\n\n## Dev Agent Record\n\n### Agent Model Used\n\nGemini 3.1 Pro (High)\n\n### Debug Log References\n- Addressed configuration error debugging; corrected the `python -c` verification string to use `manage.py shell -c` to properly run under Django context without throwing `ImproperlyConfigured`.\n\n### Completion Notes List\n- ✅ Implemented `OPENROUTER_API_KEY` in `base.py` with `django-environ`.\n- ✅ Adjusted import paths from `philips_agent` to `multi_agent` due to actual cookiecutter disk paths.\n- ✅ Created `llm_client.py` with OpenRouter `httpx` logic (Primary: mistral, Fallback: deepseek).\n- ✅ Created `orchestrate_document` in `tasks.py` implementing `self.retry` logic, trapping `MaxRetriesExceededError`, and pushing terminal error via WebSockets.\n- ✅ Retrieved `get_channel_layer()` at call-time to prevent worker startup crashes.\n- ✅ Full `pytest-mock` suite implemented for standard and timeout bounds.\n- ✅ Live API integration (Task 5) successfully verified model array payload and HTTP 200 response on `docker-compose`.\n\n### File List\n- `multi_agent/config/settings/base.py` (Modified)\n- `multi_agent/.envs/.local/.django` (Modified)\n- `multi_agent/multi_agent/orchestrator/agents/__init__.py` (New)\n- `multi_agent/multi_agent/orchestrator/agents/llm_client.py` (New)\n- `multi_agent/multi_agent/orchestrator/tasks.py` (New)\n- `multi_agent/tests/orchestrator/test_llm_client.py` (New)\n\n"}}}} |