{"repo": "aruchvesu/cereveon", "files": {"docker-compose.yml": {"ec1475e17980d641a85def6a3ff6a74418102c40": {"t": 1785745266.668027, "n": 1133, "h": "a2b4724aef6266f0", "c": "services:\n  api:\n    build:\n      context: .\n      dockerfile: llm/Dockerfile.api\n    user: \"10001:10001\"\n    environment:\n      SECA_ENV: ${SECA_ENV:-dev}\n      SECA_API_KEY: ${SECA_API_KEY:-dev-key}\n      SECRET_KEY: ${SECRET_KEY:-}\n      CORS_ALLOWED_ORIGINS: ${CORS_ALLOWED_ORIGINS:-}\n      DATABASE_URL: ${DATABASE_URL:-sqlite:///data/seca.db}\n      # LLM provider: DeepSeek (https://api.deepseek.com).  Set\n      # COACH_DEEPSEEK_API_KEY in your shell or .env so the container\n      # inherits it.  COACH_DEEPSEEK_API_BASE / COACH_DEEPSEEK_MODEL\n      # default to https://api.deepseek.com / deepseek-chat and\n      # rarely need overriding.\n      COACH_DEEPSEEK_API_KEY: ${COACH_DEEPSEEK_API_KEY:-}\n      COACH_DEEPSEEK_API_BASE: ${COACH_DEEPSEEK_API_BASE:-https://api.deepseek.com}\n      COACH_DEEPSEEK_MODEL: ${COACH_DEEPSEEK_MODEL:-deepseek-chat}\n      STOCKFISH_PATH: ${STOCKFISH_PATH:-/usr/games/stockfish}\n      REDIS_URL: ${REDIS_URL:-}\n    ports:\n      - \"8000:8000\"\n    volumes:\n      - api_data:/app/data\n    extra_hosts:\n      - \"host.docker.internal:host-gateway\"\n    restart: unless-stopped\n\nvolumes:\n  api_data:\n"}, "98890e4b3ac04488399746936f144fbdc07d3271": {"t": 1785749408.9508665, "n": 1133, "h": "a2b4724aef6266f0", "c": "services:\n  api:\n    build:\n      context: .\n      dockerfile: llm/Dockerfile.api\n    user: \"10001:10001\"\n    environment:\n      SECA_ENV: ${SECA_ENV:-dev}\n      SECA_API_KEY: ${SECA_API_KEY:-dev-key}\n      SECRET_KEY: ${SECRET_KEY:-}\n      CORS_ALLOWED_ORIGINS: ${CORS_ALLOWED_ORIGINS:-}\n      DATABASE_URL: ${DATABASE_URL:-sqlite:///data/seca.db}\n      # LLM provider: DeepSeek (https://api.deepseek.com).  Set\n      # COACH_DEEPSEEK_API_KEY in your shell or .env so the container\n      # inherits it.  COACH_DEEPSEEK_API_BASE / COACH_DEEPSEEK_MODEL\n      # default to https://api.deepseek.com / deepseek-chat and\n      # rarely need overriding.\n      COACH_DEEPSEEK_API_KEY: ${COACH_DEEPSEEK_API_KEY:-}\n      COACH_DEEPSEEK_API_BASE: ${COACH_DEEPSEEK_API_BASE:-https://api.deepseek.com}\n      COACH_DEEPSEEK_MODEL: ${COACH_DEEPSEEK_MODEL:-deepseek-chat}\n      STOCKFISH_PATH: ${STOCKFISH_PATH:-/usr/games/stockfish}\n      REDIS_URL: ${REDIS_URL:-}\n    ports:\n      - \"8000:8000\"\n    volumes:\n      - api_data:/app/data\n    extra_hosts:\n      - \"host.docker.internal:host-gateway\"\n    restart: unless-stopped\n\nvolumes:\n  api_data:\n"}, "da949ca39283ce98c2f82be676880c8562d5226d": {"t": 1785749410.030366, "n": 778, "h": "a30fa9cf69056f37", "c": "services:\n  api:\n    build:\n      context: .\n      dockerfile: llm/Dockerfile.api\n    user: \"10001:10001\"\n    environment:\n      SECA_ENV: ${SECA_ENV:-dev}\n      SECA_API_KEY: ${SECA_API_KEY:-dev-key}\n      SECRET_KEY: ${SECRET_KEY:-}\n      CORS_ALLOWED_ORIGINS: ${CORS_ALLOWED_ORIGINS:-}\n      DATABASE_URL: ${DATABASE_URL:-sqlite:///data/seca.db}\n      COACH_OLLAMA_URL: ${COACH_OLLAMA_URL:-http://host.docker.internal:11434}\n      COACH_OLLAMA_MODEL: ${COACH_OLLAMA_MODEL:-qwen2.5:7b-instruct-q2_K}\n      STOCKFISH_PATH: ${STOCKFISH_PATH:-/usr/games/stockfish}\n      REDIS_URL: ${REDIS_URL:-}\n    ports:\n      - \"8000:8000\"\n    volumes:\n      - api_data:/app/data\n    extra_hosts:\n      - \"host.docker.internal:host-gateway\"\n    restart: unless-stopped\n\nvolumes:\n  api_data:\n"}}, "README.md": {"ec1475e17980d641a85def6a3ff6a74418102c40": {"t": 1785746578.2911987, "n": 35466, "h": "72a7e9670b55078e", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nWhen the LLM path fails — validator rejection after the in-pipeline repair\nbudget (`run_mode_2`'s ≤ `MAX_MODE_2_RETRIES` rewrite attempts is exhausted),\noutput-firewall block, or LLM unreachable — the request does not return a\npatched LLM string. It falls through to a deterministic fallback\n(`_build_reply_deterministic` in `llm/seca/coach/chat_pipeline.py`, with the\nparallel path in `llm/seca/coach/live_move_pipeline.py`) that builds the\nreply from scratch using only trusted inputs: the engine signal, the\n`SafeExplainer` output, and a deterministic context block. The LLM's text\nis discarded entirely; the fallback never appends to or edits it. By\nconstruction, the reply cannot contain forbidden phrases because none are\never introduced. This fallback is documented under\n[*Deterministic Fallback*](docs/ARCHITECTURE.md#deterministic-fallback) in\nthe architecture spec, and is forbidden from being widened to accept any\nLLM-derived content.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA is the thin adaptation layer under `llm/seca/`: a 5-action\ncontextual bandit plus a deterministic skill tracker that adapt to\nindividual players without retraining either base model (Stockfish or\nthe LLM). Every adaptation step is closed-form\n(`A ← A + xxᵀ`, `b ← b + r·x` for the bandit; deterministic deltas for\nrating / confidence / weakness / embedding state), and a hard runtime\nguard (`llm/seca/safety/freeze.py`) enforces that nothing heavier — no\ngradient steps, no neural retraining, no autonomous RL — ever runs in\nthe live process.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Reward-signal trust | **Engine-derived.** `/game/finish` recomputes accuracy + weaknesses from the submitted PGN via the engine pool before they reach the bandit, skill tracker, or storage. Falls back to client-supplied values only when the engine pool is unavailable; emits `ACC_FALLBACK` / `ACC_DIVERGENCE` telemetry. See [`docs/SECA.md`](docs/SECA.md#trust-property-of-the-reward-signal). |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The per-request\nverifier (`verify_runtime_safety`) is wired into `GET /seca/status`\nas of 2026-05-14: the endpoint reflects the live runtime, not just\nthe boot-time constant, so a forbidden brain module lazily loaded\nafter startup flips the next status response to `safe_mode: false`\nwithout crashing the process.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/explain` | API key | — | Deterministic SAFE_V1 explanation (no LLM) — see note below |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n> **`/explain` and `/seca/explain` are intentionally deterministic.** Both\n> routes run the engine signal through `SafeExplainer` and return prose\n> templated from the ESV bands — no LLM call, no RAG retrieval, no\n> contract validators (the templates are safe by construction). The\n> real Mode-2 LLM pipeline (RAG retrieval → prompt rendering → DeepSeek\n> → output validators → deterministic fallback) is reachable via\n> `/chat` and `/chat/stream`. Keeping the `/explain` routes\n> deterministic means they are free, fast, and CI-friendly; the Android\n> client routes coaching conversation through `/chat` and uses\n> `/explain` for cheap structured explanations only.\n\n### API schema versioning\n\nTwo response headers carry the version contract:\n\n| Header | Value | Meaning |\n|---|---|---|\n| `X-API-Version` | `2` (currently) | Server's *preferred* version — what new clients should target |\n| `X-API-Versions-Supported` | `1, 2` (comma-separated when multiple) | Every version the server will accept on inbound requests |\n\nThe Android client sends `X-API-Version` on coaching requests; the\nserver gates on it (**lenient on missing**, **strict on\nunsupported**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| listed in `X-API-Versions-Supported` | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming the supported range |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read both response\nheaders and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`; CORS `expose_headers` includes both response\nheaders so browser scripts can read them.\n\n#### Bumping the version\n\nTwo distinct flows, depending on whether the new version is\nbreaking or additive:\n\n- **Additive (back-compat).** Append the new version to\n  `API_VERSIONS_SUPPORTED` in `llm/server.py` (and update\n  `API_VERSION` to the new value if it becomes the preferred).\n  Existing clients keep working — they're still in the supported\n  list. No coordinated Android release required.\n- **Breaking (drop old version).** Same as above, then drop the\n  old version from `API_VERSIONS_SUPPORTED` **after** the Android\n  client has rolled out to a sufficient fraction of users. The\n  bump from supported set `(1, 2)` to `(2,)` is the breaking\n  release; the migration window (when both are supported) is the\n  grace period.\n\nIn either case, also update `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` when\nthe *preferred* version bumps so new Android builds advertise\nthe new version. Pinned by\n`llm/tests/test_api_version_header.py` (AVH_01..AVH_14) +\n`llm/tests/test_doc_constants_pinned.py::test_api_version_constant`.\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs a **single tier**, built and signed by the CI pipeline.\nFull runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Everything: public ingress at `cereveon.com` behind Caddy, engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nAuto-deploys from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback). The filename is\nhistorical; the workflow's `name:` field is `CI/CD`, which is\nauthoritative.\n\n> A former Fly.io Node edge (`llm/server.js`, an Ollama prototype) was\n> removed 2026-07-15 — it never fronted `cereveon.com` (that resolves to\n> Hetzner's Caddy directly) and served only an unauthenticated,\n> validator-bypassing `/coach`·`/explain`. See the runbook for the\n> `flyctl apps destroy chesscoach` teardown step.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point — also hosts /engine/eval\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "061f6032afb9ad9a40999c4f1908b655fcdabb21": {"t": 1785749410.1474435, "n": 35466, "h": "72a7e9670b55078e", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nWhen the LLM path fails — validator rejection after the in-pipeline repair\nbudget (`run_mode_2`'s ≤ `MAX_MODE_2_RETRIES` rewrite attempts is exhausted),\noutput-firewall block, or LLM unreachable — the request does not return a\npatched LLM string. It falls through to a deterministic fallback\n(`_build_reply_deterministic` in `llm/seca/coach/chat_pipeline.py`, with the\nparallel path in `llm/seca/coach/live_move_pipeline.py`) that builds the\nreply from scratch using only trusted inputs: the engine signal, the\n`SafeExplainer` output, and a deterministic context block. The LLM's text\nis discarded entirely; the fallback never appends to or edits it. By\nconstruction, the reply cannot contain forbidden phrases because none are\never introduced. This fallback is documented under\n[*Deterministic Fallback*](docs/ARCHITECTURE.md#deterministic-fallback) in\nthe architecture spec, and is forbidden from being widened to accept any\nLLM-derived content.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA is the thin adaptation layer under `llm/seca/`: a 5-action\ncontextual bandit plus a deterministic skill tracker that adapt to\nindividual players without retraining either base model (Stockfish or\nthe LLM). Every adaptation step is closed-form\n(`A ← A + xxᵀ`, `b ← b + r·x` for the bandit; deterministic deltas for\nrating / confidence / weakness / embedding state), and a hard runtime\nguard (`llm/seca/safety/freeze.py`) enforces that nothing heavier — no\ngradient steps, no neural retraining, no autonomous RL — ever runs in\nthe live process.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Reward-signal trust | **Engine-derived.** `/game/finish` recomputes accuracy + weaknesses from the submitted PGN via the engine pool before they reach the bandit, skill tracker, or storage. Falls back to client-supplied values only when the engine pool is unavailable; emits `ACC_FALLBACK` / `ACC_DIVERGENCE` telemetry. See [`docs/SECA.md`](docs/SECA.md#trust-property-of-the-reward-signal). |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The per-request\nverifier (`verify_runtime_safety`) is wired into `GET /seca/status`\nas of 2026-05-14: the endpoint reflects the live runtime, not just\nthe boot-time constant, so a forbidden brain module lazily loaded\nafter startup flips the next status response to `safe_mode: false`\nwithout crashing the process.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/explain` | API key | — | Deterministic SAFE_V1 explanation (no LLM) — see note below |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n> **`/explain` and `/seca/explain` are intentionally deterministic.** Both\n> routes run the engine signal through `SafeExplainer` and return prose\n> templated from the ESV bands — no LLM call, no RAG retrieval, no\n> contract validators (the templates are safe by construction). The\n> real Mode-2 LLM pipeline (RAG retrieval → prompt rendering → DeepSeek\n> → output validators → deterministic fallback) is reachable via\n> `/chat` and `/chat/stream`. Keeping the `/explain` routes\n> deterministic means they are free, fast, and CI-friendly; the Android\n> client routes coaching conversation through `/chat` and uses\n> `/explain` for cheap structured explanations only.\n\n### API schema versioning\n\nTwo response headers carry the version contract:\n\n| Header | Value | Meaning |\n|---|---|---|\n| `X-API-Version` | `2` (currently) | Server's *preferred* version — what new clients should target |\n| `X-API-Versions-Supported` | `1, 2` (comma-separated when multiple) | Every version the server will accept on inbound requests |\n\nThe Android client sends `X-API-Version` on coaching requests; the\nserver gates on it (**lenient on missing**, **strict on\nunsupported**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| listed in `X-API-Versions-Supported` | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming the supported range |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read both response\nheaders and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`; CORS `expose_headers` includes both response\nheaders so browser scripts can read them.\n\n#### Bumping the version\n\nTwo distinct flows, depending on whether the new version is\nbreaking or additive:\n\n- **Additive (back-compat).** Append the new version to\n  `API_VERSIONS_SUPPORTED` in `llm/server.py` (and update\n  `API_VERSION` to the new value if it becomes the preferred).\n  Existing clients keep working — they're still in the supported\n  list. No coordinated Android release required.\n- **Breaking (drop old version).** Same as above, then drop the\n  old version from `API_VERSIONS_SUPPORTED` **after** the Android\n  client has rolled out to a sufficient fraction of users. The\n  bump from supported set `(1, 2)` to `(2,)` is the breaking\n  release; the migration window (when both are supported) is the\n  grace period.\n\nIn either case, also update `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` when\nthe *preferred* version bumps so new Android builds advertise\nthe new version. Pinned by\n`llm/tests/test_api_version_header.py` (AVH_01..AVH_14) +\n`llm/tests/test_doc_constants_pinned.py::test_api_version_constant`.\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs a **single tier**, built and signed by the CI pipeline.\nFull runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Everything: public ingress at `cereveon.com` behind Caddy, engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nAuto-deploys from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback). The filename is\nhistorical; the workflow's `name:` field is `CI/CD`, which is\nauthoritative.\n\n> A former Fly.io Node edge (`llm/server.js`, an Ollama prototype) was\n> removed 2026-07-15 — it never fronted `cereveon.com` (that resolves to\n> Hetzner's Caddy directly) and served only an unauthenticated,\n> validator-bypassing `/coach`·`/explain`. See the runbook for the\n> `flyctl apps destroy chesscoach` teardown step.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point — also hosts /engine/eval\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "52e2d0a1a31a6c76a4288e1061a68fd64aa594ef": {"t": 1785749411.431472, "n": 35623, "h": "599bcbedfde4a414", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nWhen the LLM path fails — validator rejection after the in-pipeline repair\nbudget (`run_mode_2`'s ≤ `MAX_MODE_2_RETRIES` rewrite attempts is exhausted),\noutput-firewall block, or LLM unreachable — the request does not return a\npatched LLM string. It falls through to a deterministic fallback\n(`_build_reply_deterministic` in `llm/seca/coach/chat_pipeline.py`, with the\nparallel path in `llm/seca/coach/live_move_pipeline.py`) that builds the\nreply from scratch using only trusted inputs: the engine signal, the\n`SafeExplainer` output, and a deterministic context block. The LLM's text\nis discarded entirely; the fallback never appends to or edits it. By\nconstruction, the reply cannot contain forbidden phrases because none are\never introduced. This fallback is documented under\n[*Deterministic Fallback*](docs/ARCHITECTURE.md#deterministic-fallback) in\nthe architecture spec, and is forbidden from being widened to accept any\nLLM-derived content.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA is the thin adaptation layer under `llm/seca/`: a 5-action\ncontextual bandit plus a deterministic skill tracker that adapt to\nindividual players without retraining either base model (Stockfish or\nthe LLM). Every adaptation step is closed-form\n(`A ← A + xxᵀ`, `b ← b + r·x` for the bandit; deterministic deltas for\nrating / confidence / weakness / embedding state), and a hard runtime\nguard (`llm/seca/safety/freeze.py`) enforces that nothing heavier — no\ngradient steps, no neural retraining, no autonomous RL — ever runs in\nthe live process.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Reward-signal trust | **Engine-derived.** `/game/finish` recomputes accuracy + weaknesses from the submitted PGN via the engine pool before they reach the bandit, skill tracker, or storage. Falls back to client-supplied values only when the engine pool is unavailable; emits `ACC_FALLBACK` / `ACC_DIVERGENCE` telemetry. See [`docs/SECA.md`](docs/SECA.md#trust-property-of-the-reward-signal). |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The per-request\nverifier (`verify_runtime_safety`) is wired into `GET /seca/status`\nas of 2026-05-14: the endpoint reflects the live runtime, not just\nthe boot-time constant, so a forbidden brain module lazily loaded\nafter startup flips the next status response to `safe_mode: false`\nwithout crashing the process.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/explain` | API key | — | Deterministic SAFE_V1 explanation (no LLM) — see note below |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n> **`/explain` and `/seca/explain` are intentionally deterministic.** Both\n> routes run the engine signal through `SafeExplainer` and return prose\n> templated from the ESV bands — no LLM call, no RAG retrieval, no\n> contract validators (the templates are safe by construction). The\n> real Mode-2 LLM pipeline (RAG retrieval → prompt rendering → DeepSeek\n> → output validators → deterministic fallback) is reachable via\n> `/chat` and `/chat/stream`. Keeping the `/explain` routes\n> deterministic means they are free, fast, and CI-friendly; the Android\n> client routes coaching conversation through `/chat` and uses\n> `/explain` for cheap structured explanations only.\n\n### API schema versioning\n\nTwo response headers carry the version contract:\n\n| Header | Value | Meaning |\n|---|---|---|\n| `X-API-Version` | `2` (currently) | Server's *preferred* version — what new clients should target |\n| `X-API-Versions-Supported` | `1, 2` (comma-separated when multiple) | Every version the server will accept on inbound requests |\n\nThe Android client sends `X-API-Version` on coaching requests; the\nserver gates on it (**lenient on missing**, **strict on\nunsupported**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| listed in `X-API-Versions-Supported` | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming the supported range |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read both response\nheaders and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`; CORS `expose_headers` includes both response\nheaders so browser scripts can read them.\n\n#### Bumping the version\n\nTwo distinct flows, depending on whether the new version is\nbreaking or additive:\n\n- **Additive (back-compat).** Append the new version to\n  `API_VERSIONS_SUPPORTED` in `llm/server.py` (and update\n  `API_VERSION` to the new value if it becomes the preferred).\n  Existing clients keep working — they're still in the supported\n  list. No coordinated Android release required.\n- **Breaking (drop old version).** Same as above, then drop the\n  old version from `API_VERSIONS_SUPPORTED` **after** the Android\n  client has rolled out to a sufficient fraction of users. The\n  bump from supported set `(1, 2)` to `(2,)` is the breaking\n  release; the migration window (when both are supported) is the\n  grace period.\n\nIn either case, also update `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` when\nthe *preferred* version bumps so new Android builds advertise\nthe new version. Pinned by\n`llm/tests/test_api_version_header.py` (AVH_01..AVH_14) +\n`llm/tests/test_doc_constants_pinned.py::test_api_version_constant`.\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point — also hosts /engine/eval\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "2da7672e21015a09c4cc2479f7b1387227fec750": {"t": 1785749412.6947799, "n": 35620, "h": "583ca67b0f8d325a", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nWhen the LLM path fails — validator rejection after the in-pipeline repair\nbudget (`run_mode_2`'s ≤ `MAX_MODE_2_RETRIES` rewrite attempts is exhausted),\noutput-firewall block, or LLM unreachable — the request does not return a\npatched LLM string. It falls through to a deterministic fallback\n(`_build_reply_deterministic` in `llm/seca/coach/chat_pipeline.py`, with the\nparallel path in `llm/seca/coach/live_move_pipeline.py`) that builds the\nreply from scratch using only trusted inputs: the engine signal, the\n`SafeExplainer` output, and a deterministic context block. The LLM's text\nis discarded entirely; the fallback never appends to or edits it. By\nconstruction, the reply cannot contain forbidden phrases because none are\never introduced. This fallback is documented under\n[*Deterministic Fallback*](docs/ARCHITECTURE.md#deterministic-fallback) in\nthe architecture spec, and is forbidden from being widened to accept any\nLLM-derived content.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA is the thin adaptation layer under `llm/seca/`: a 5-action\ncontextual bandit plus a deterministic skill tracker that adapt to\nindividual players without retraining either base model (Stockfish or\nthe LLM). Every adaptation step is closed-form\n(`A ← A + xxᵀ`, `b ← b + r·x` for the bandit; deterministic deltas for\nrating / confidence / weakness / embedding state), and a hard runtime\nguard (`llm/seca/safety/freeze.py`) enforces that nothing heavier — no\ngradient steps, no neural retraining, no autonomous RL — ever runs in\nthe live process.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Reward-signal trust | **Engine-derived.** `/game/finish` recomputes accuracy + weaknesses from the submitted PGN via the engine pool before they reach the bandit, skill tracker, or storage. Falls back to client-supplied values only when the engine pool is unavailable; emits `ACC_FALLBACK` / `ACC_DIVERGENCE` telemetry. See [`docs/SECA.md`](docs/SECA.md#trust-property-of-the-reward-signal). |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The per-request\nverifier (`verify_runtime_safety`) is wired into `GET /seca/status`\nas of 2026-05-14: the endpoint reflects the live runtime, not just\nthe boot-time constant, so a forbidden brain module lazily loaded\nafter startup flips the next status response to `safe_mode: false`\nwithout crashing the process.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/explain` | API key | — | Deterministic SAFE_V1 explanation (no LLM) — see note below |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n> **`/explain` and `/seca/explain` are intentionally deterministic.** Both\n> routes run the engine signal through `SafeExplainer` and return prose\n> templated from the ESV bands — no LLM call, no RAG retrieval, no\n> contract validators (the templates are safe by construction). The\n> real Mode-2 LLM pipeline (RAG retrieval → prompt rendering → DeepSeek\n> → output validators → deterministic fallback) is reachable via\n> `/chat` and `/chat/stream`. Keeping the `/explain` routes\n> deterministic means they are free, fast, and CI-friendly; the Android\n> client routes coaching conversation through `/chat` and uses\n> `/explain` for cheap structured explanations only.\n\n### API schema versioning\n\nTwo response headers carry the version contract:\n\n| Header | Value | Meaning |\n|---|---|---|\n| `X-API-Version` | `1` (currently) | Server's *preferred* version — what new clients should target |\n| `X-API-Versions-Supported` | `1` (comma-separated when multiple) | Every version the server will accept on inbound requests |\n\nThe Android client sends `X-API-Version` on coaching requests; the\nserver gates on it (**lenient on missing**, **strict on\nunsupported**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| listed in `X-API-Versions-Supported` | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming the supported range |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read both response\nheaders and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`; CORS `expose_headers` includes both response\nheaders so browser scripts can read them.\n\n#### Bumping the version\n\nTwo distinct flows, depending on whether the new version is\nbreaking or additive:\n\n- **Additive (back-compat).** Append the new version to\n  `API_VERSIONS_SUPPORTED` in `llm/server.py` (and update\n  `API_VERSION` to the new value if it becomes the preferred).\n  Existing clients keep working — they're still in the supported\n  list. No coordinated Android release required.\n- **Breaking (drop old version).** Same as above, then drop the\n  old version from `API_VERSIONS_SUPPORTED` **after** the Android\n  client has rolled out to a sufficient fraction of users. The\n  bump from supported set `(1, 2)` to `(2,)` is the breaking\n  release; the migration window (when both are supported) is the\n  grace period.\n\nIn either case, also update `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` when\nthe *preferred* version bumps so new Android builds advertise\nthe new version. Pinned by\n`llm/tests/test_api_version_header.py` (AVH_01..AVH_14) +\n`llm/tests/test_doc_constants_pinned.py::test_api_version_constant`.\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point — also hosts /engine/eval\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "3e98e8c780e73cfe5728a6f473e2584f67c61cad": {"t": 1785749413.9885974, "n": 35823, "h": "277546e7387e91ae", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nWhen the LLM path fails — validator rejection after the in-pipeline repair\nbudget (`run_mode_2`'s ≤ `MAX_MODE_2_RETRIES` rewrite attempts is exhausted),\noutput-firewall block, or LLM unreachable — the request does not return a\npatched LLM string. It falls through to a deterministic fallback\n(`_build_reply_deterministic` in `llm/seca/coach/chat_pipeline.py`, with the\nparallel path in `llm/seca/coach/live_move_pipeline.py`) that builds the\nreply from scratch using only trusted inputs: the engine signal, the\n`SafeExplainer` output, and a deterministic context block. The LLM's text\nis discarded entirely; the fallback never appends to or edits it. By\nconstruction, the reply cannot contain forbidden phrases because none are\never introduced. This fallback is documented under\n[*Deterministic Fallback*](docs/ARCHITECTURE.md#deterministic-fallback) in\nthe architecture spec, and is forbidden from being widened to accept any\nLLM-derived content.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA is the thin adaptation layer under `llm/seca/`: a 5-action\ncontextual bandit plus a deterministic skill tracker that adapt to\nindividual players without retraining either base model (Stockfish or\nthe LLM). Every adaptation step is closed-form\n(`A ← A + xxᵀ`, `b ← b + r·x` for the bandit; deterministic deltas for\nrating / confidence / weakness / embedding state), and a hard runtime\nguard (`llm/seca/safety/freeze.py`) enforces that nothing heavier — no\ngradient steps, no neural retraining, no autonomous RL — ever runs in\nthe live process.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Reward-signal trust | **Engine-derived.** `/game/finish` recomputes accuracy + weaknesses from the submitted PGN via the engine pool before they reach the bandit, skill tracker, or storage. Falls back to client-supplied values only when the engine pool is unavailable; emits `ACC_FALLBACK` / `ACC_DIVERGENCE` telemetry. See [`docs/SECA.md`](docs/SECA.md#trust-property-of-the-reward-signal). |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The per-request\nverifier (`verify_runtime_safety`) is wired into `GET /seca/status`\nas of 2026-05-14: the endpoint reflects the live runtime, not just\nthe boot-time constant, so a forbidden brain module lazily loaded\nafter startup flips the next status response to `safe_mode: false`\nwithout crashing the process.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/move` | JWT | 30/min | Request opponent move |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/explain` | API key | — | Deterministic SAFE_V1 explanation (no LLM) — see note below |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| POST | `/adaptation/mode` | JWT | — | Set adaptation mode for a player |\n| GET | `/adaptation/mode` | JWT | — | Read current adaptation mode |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n> **`/explain` and `/seca/explain` are intentionally deterministic.** Both\n> routes run the engine signal through `SafeExplainer` and return prose\n> templated from the ESV bands — no LLM call, no RAG retrieval, no\n> contract validators (the templates are safe by construction). The\n> real Mode-2 LLM pipeline (RAG retrieval → prompt rendering → DeepSeek\n> → output validators → deterministic fallback) is reachable via\n> `/chat` and `/chat/stream`. Keeping the `/explain` routes\n> deterministic means they are free, fast, and CI-friendly; the Android\n> client routes coaching conversation through `/chat` and uses\n> `/explain` for cheap structured explanations only.\n\n### API schema versioning\n\nTwo response headers carry the version contract:\n\n| Header | Value | Meaning |\n|---|---|---|\n| `X-API-Version` | `1` (currently) | Server's *preferred* version — what new clients should target |\n| `X-API-Versions-Supported` | `1` (comma-separated when multiple) | Every version the server will accept on inbound requests |\n\nThe Android client sends `X-API-Version` on coaching requests; the\nserver gates on it (**lenient on missing**, **strict on\nunsupported**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| listed in `X-API-Versions-Supported` | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming the supported range |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read both response\nheaders and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`; CORS `expose_headers` includes both response\nheaders so browser scripts can read them.\n\n#### Bumping the version\n\nTwo distinct flows, depending on whether the new version is\nbreaking or additive:\n\n- **Additive (back-compat).** Append the new version to\n  `API_VERSIONS_SUPPORTED` in `llm/server.py` (and update\n  `API_VERSION` to the new value if it becomes the preferred).\n  Existing clients keep working — they're still in the supported\n  list. No coordinated Android release required.\n- **Breaking (drop old version).** Same as above, then drop the\n  old version from `API_VERSIONS_SUPPORTED` **after** the Android\n  client has rolled out to a sufficient fraction of users. The\n  bump from supported set `(1, 2)` to `(2,)` is the breaking\n  release; the migration window (when both are supported) is the\n  grace period.\n\nIn either case, also update `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` when\nthe *preferred* version bumps so new Android builds advertise\nthe new version. Pinned by\n`llm/tests/test_api_version_header.py` (AVH_01..AVH_14) +\n`llm/tests/test_doc_constants_pinned.py::test_api_version_constant`.\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point — also hosts /engine/eval\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "26eaa8bf332457eddccd5aca1e7e4d47529d4a66": {"t": 1785749415.4198155, "n": 35983, "h": "8c1cc5cdb1ef0824", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nWhen the LLM path fails — validator rejection after the in-pipeline repair\nbudget (`run_mode_2`'s ≤ `MAX_MODE_2_RETRIES` rewrite attempts is exhausted),\noutput-firewall block, or LLM unreachable — the request does not return a\npatched LLM string. It falls through to a deterministic fallback\n(`_build_reply_deterministic` in `llm/seca/coach/chat_pipeline.py`, with the\nparallel path in `llm/seca/coach/live_move_pipeline.py`) that builds the\nreply from scratch using only trusted inputs: the engine signal, the\n`SafeExplainer` output, and a deterministic context block. The LLM's text\nis discarded entirely; the fallback never appends to or edits it. By\nconstruction, the reply cannot contain forbidden phrases because none are\never introduced. This fallback is documented under\n[*Deterministic Fallback*](docs/ARCHITECTURE.md#deterministic-fallback) in\nthe architecture spec, and is forbidden from being widened to accept any\nLLM-derived content.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA is the thin adaptation layer under `llm/seca/`: a 5-action\ncontextual bandit plus a deterministic skill tracker that adapt to\nindividual players without retraining either base model (Stockfish or\nthe LLM). Every adaptation step is closed-form\n(`A ← A + xxᵀ`, `b ← b + r·x` for the bandit; deterministic deltas for\nrating / confidence / weakness / embedding state), and a hard runtime\nguard (`llm/seca/safety/freeze.py`) enforces that nothing heavier — no\ngradient steps, no neural retraining, no autonomous RL — ever runs in\nthe live process.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Reward-signal trust | **Engine-derived.** `/game/finish` recomputes accuracy + weaknesses from the submitted PGN via the engine pool before they reach the bandit, skill tracker, or storage. Falls back to client-supplied values only when the engine pool is unavailable; emits `ACC_FALLBACK` / `ACC_DIVERGENCE` telemetry. See [`docs/SECA.md`](docs/SECA.md#trust-property-of-the-reward-signal). |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The per-request\nverifier (`verify_runtime_safety`) is wired into `GET /seca/status`\nas of 2026-05-14: the endpoint reflects the live runtime, not just\nthe boot-time constant, so a forbidden brain module lazily loaded\nafter startup flips the next status response to `safe_mode: false`\nwithout crashing the process.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/move` | JWT | 30/min | Request opponent move |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/analyze` | API key | 30/min | Engine signal only (no LLM) |\n| POST | `/explain` | API key | — | Deterministic SAFE_V1 explanation (no LLM) — see note below |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| POST | `/explanation_outcome` | API key | 20/min | Post-explanation learning outcome |\n| POST | `/adaptation/mode` | JWT | — | Set adaptation mode for a player |\n| GET | `/adaptation/mode` | JWT | — | Read current adaptation mode |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n> **`/explain` and `/seca/explain` are intentionally deterministic.** Both\n> routes run the engine signal through `SafeExplainer` and return prose\n> templated from the ESV bands — no LLM call, no RAG retrieval, no\n> contract validators (the templates are safe by construction). The\n> real Mode-2 LLM pipeline (RAG retrieval → prompt rendering → DeepSeek\n> → output validators → deterministic fallback) is reachable via\n> `/chat` and `/chat/stream`. Keeping the `/explain` routes\n> deterministic means they are free, fast, and CI-friendly; the Android\n> client routes coaching conversation through `/chat` and uses\n> `/explain` for cheap structured explanations only.\n\n### API schema versioning\n\nTwo response headers carry the version contract:\n\n| Header | Value | Meaning |\n|---|---|---|\n| `X-API-Version` | `1` (currently) | Server's *preferred* version — what new clients should target |\n| `X-API-Versions-Supported` | `1` (comma-separated when multiple) | Every version the server will accept on inbound requests |\n\nThe Android client sends `X-API-Version` on coaching requests; the\nserver gates on it (**lenient on missing**, **strict on\nunsupported**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| listed in `X-API-Versions-Supported` | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming the supported range |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read both response\nheaders and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`; CORS `expose_headers` includes both response\nheaders so browser scripts can read them.\n\n#### Bumping the version\n\nTwo distinct flows, depending on whether the new version is\nbreaking or additive:\n\n- **Additive (back-compat).** Append the new version to\n  `API_VERSIONS_SUPPORTED` in `llm/server.py` (and update\n  `API_VERSION` to the new value if it becomes the preferred).\n  Existing clients keep working — they're still in the supported\n  list. No coordinated Android release required.\n- **Breaking (drop old version).** Same as above, then drop the\n  old version from `API_VERSIONS_SUPPORTED` **after** the Android\n  client has rolled out to a sufficient fraction of users. The\n  bump from supported set `(1, 2)` to `(2,)` is the breaking\n  release; the migration window (when both are supported) is the\n  grace period.\n\nIn either case, also update `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` when\nthe *preferred* version bumps so new Android builds advertise\nthe new version. Pinned by\n`llm/tests/test_api_version_header.py` (AVH_01..AVH_14) +\n`llm/tests/test_doc_constants_pinned.py::test_api_version_constant`.\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point — also hosts /engine/eval\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "1d1c2542f7d679434f73974378cc55324685c2fd": {"t": 1785749416.6660771, "n": 34849, "h": "81700e1120851a5e", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nWhen the LLM path fails — validator rejection after the in-pipeline repair\nbudget (`run_mode_2`'s ≤ `MAX_MODE_2_RETRIES` rewrite attempts is exhausted),\noutput-firewall block, or LLM unreachable — the request does not return a\npatched LLM string. It falls through to a deterministic fallback\n(`_build_reply_deterministic` in `llm/seca/coach/chat_pipeline.py`, with the\nparallel path in `llm/seca/coach/live_move_pipeline.py`) that builds the\nreply from scratch using only trusted inputs: the engine signal, the\n`SafeExplainer` output, and a deterministic context block. The LLM's text\nis discarded entirely; the fallback never appends to or edits it. By\nconstruction, the reply cannot contain forbidden phrases because none are\never introduced. This fallback is documented under\n[*Deterministic Fallback*](docs/ARCHITECTURE.md#deterministic-fallback) in\nthe architecture spec, and is forbidden from being widened to accept any\nLLM-derived content.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA is the thin adaptation layer under `llm/seca/`: a 5-action\ncontextual bandit plus a deterministic skill tracker that adapt to\nindividual players without retraining either base model (Stockfish or\nthe LLM). Every adaptation step is closed-form\n(`A ← A + xxᵀ`, `b ← b + r·x` for the bandit; deterministic deltas for\nrating / confidence / weakness / embedding state), and a hard runtime\nguard (`llm/seca/safety/freeze.py`) enforces that nothing heavier — no\ngradient steps, no neural retraining, no autonomous RL — ever runs in\nthe live process.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Reward-signal trust | **Engine-derived.** `/game/finish` recomputes accuracy + weaknesses from the submitted PGN via the engine pool before they reach the bandit, skill tracker, or storage. Falls back to client-supplied values only when the engine pool is unavailable; emits `ACC_FALLBACK` / `ACC_DIVERGENCE` telemetry. See [`docs/SECA.md`](docs/SECA.md#trust-property-of-the-reward-signal). |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The per-request\nverifier (`verify_runtime_safety`) is wired into `GET /seca/status`\nas of 2026-05-14: the endpoint reflects the live runtime, not just\nthe boot-time constant, so a forbidden brain module lazily loaded\nafter startup flips the next status response to `safe_mode: false`\nwithout crashing the process.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/move` | JWT | 30/min | Request opponent move |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/analyze` | API key | 30/min | Engine signal only (no LLM) |\n| POST | `/explain` | API key | — | Deterministic SAFE_V1 explanation (no LLM) — see note below |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| POST | `/explanation_outcome` | API key | 20/min | Post-explanation learning outcome |\n| POST | `/adaptation/mode` | JWT | — | Set adaptation mode for a player |\n| GET | `/adaptation/mode` | JWT | — | Read current adaptation mode |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n> **`/explain` and `/seca/explain` are intentionally deterministic.** Both\n> routes run the engine signal through `SafeExplainer` and return prose\n> templated from the ESV bands — no LLM call, no RAG retrieval, no\n> contract validators (the templates are safe by construction). The\n> real Mode-2 LLM pipeline (RAG retrieval → prompt rendering → DeepSeek\n> → output validators → deterministic fallback) is reachable via\n> `/chat` and `/chat/stream`. Keeping the `/explain` routes\n> deterministic means they are free, fast, and CI-friendly; the Android\n> client routes coaching conversation through `/chat` and uses\n> `/explain` for cheap structured explanations only.\n\n### API schema versioning\n\nEvery response carries an `X-API-Version` header pinned at `1`. The\nAndroid client sends the same header on coaching requests; the server\ngates on it (Phase 1 — **lenient on missing**, **strict on mismatch**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| `1` (matches `API_VERSION`) | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming both versions |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read the server's current\nversion and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`.\n\nBumping the version requires a **coordinated server + Android release** —\nbump `API_VERSION` in `llm/server.py` and `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` in the same\nPR. Pinned by `llm/tests/test_api_version_header.py` (AVH_01..AVH_10).\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point — also hosts /engine/eval\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "706476de69898b71a20899b616949c7a6cbb4cbb": {"t": 1785749417.935605, "n": 34995, "h": "1f726536b230e2de", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nWhen the LLM path fails — validator rejection after the in-pipeline repair\nbudget (`run_mode_2`'s ≤ `MAX_MODE_2_RETRIES` rewrite attempts is exhausted),\noutput-firewall block, or LLM unreachable — the request does not return a\npatched LLM string. It falls through to a deterministic fallback\n(`_build_reply_deterministic` in `llm/seca/coach/chat_pipeline.py`, with the\nparallel path in `llm/seca/coach/live_move_pipeline.py`) that builds the\nreply from scratch using only trusted inputs: the engine signal, the\n`SafeExplainer` output, and a deterministic context block. The LLM's text\nis discarded entirely; the fallback never appends to or edits it. By\nconstruction, the reply cannot contain forbidden phrases because none are\never introduced. This fallback is documented under\n[*Deterministic Fallback*](docs/ARCHITECTURE.md#deterministic-fallback) in\nthe architecture spec, and is forbidden from being widened to accept any\nLLM-derived content.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA is the thin adaptation layer under `llm/seca/`: a 5-action\ncontextual bandit plus a deterministic skill tracker that adapt to\nindividual players without retraining either base model (Stockfish or\nthe LLM). Every adaptation step is closed-form\n(`A ← A + xxᵀ`, `b ← b + r·x` for the bandit; deterministic deltas for\nrating / confidence / weakness / embedding state), and a hard runtime\nguard (`llm/seca/safety/freeze.py`) enforces that nothing heavier — no\ngradient steps, no neural retraining, no autonomous RL — ever runs in\nthe live process.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Reward-signal trust | **Engine-derived.** `/game/finish` recomputes accuracy + weaknesses from the submitted PGN via the engine pool before they reach the bandit, skill tracker, or storage. Falls back to client-supplied values only when the engine pool is unavailable; emits `ACC_FALLBACK` / `ACC_DIVERGENCE` telemetry. See [`docs/SECA.md`](docs/SECA.md#trust-property-of-the-reward-signal). |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The per-request\nverifier (`verify_runtime_safety`) is wired into `GET /seca/status`\nas of 2026-05-14: the endpoint reflects the live runtime, not just\nthe boot-time constant, so a forbidden brain module lazily loaded\nafter startup flips the next status response to `safe_mode: false`\nwithout crashing the process.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/move` | JWT | 30/min | Request opponent move |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/analyze` | API key | 30/min | Engine signal only (no LLM) |\n| POST | `/explain` | API key | — | Deterministic SAFE_V1 explanation (no LLM) — see note below |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| POST | `/explanation_outcome` | API key | 20/min | Post-explanation learning outcome |\n| POST | `/adaptation/mode` | JWT | — | Set adaptation mode for a player |\n| GET | `/adaptation/mode` | JWT | — | Read current adaptation mode |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n> **`/explain` and `/seca/explain` are intentionally deterministic.** Both\n> routes run the engine signal through `SafeExplainer` and return prose\n> templated from the ESV bands — no LLM call, no RAG retrieval, no\n> contract validators (the templates are safe by construction). The\n> real Mode-2 LLM pipeline (RAG retrieval → prompt rendering → DeepSeek\n> → output validators → deterministic fallback) is reachable via\n> `/chat` and `/chat/stream`. Keeping the `/explain` routes\n> deterministic means they are free, fast, and CI-friendly; the Android\n> client routes coaching conversation through `/chat` and uses\n> `/explain` for cheap structured explanations only.\n\n### API schema versioning\n\nEvery response carries an `X-API-Version` header pinned at `1`. The\nAndroid client sends the same header on coaching requests; the server\ngates on it (Phase 1 — **lenient on missing**, **strict on mismatch**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| `1` (matches `API_VERSION`) | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming both versions |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read the server's current\nversion and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`.\n\nBumping the version requires a **coordinated server + Android release** —\nbump `API_VERSION` in `llm/server.py` and `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` in the same\nPR. Pinned by `llm/tests/test_api_version_header.py` (AVH_01..AVH_10).\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point — also hosts /engine/eval\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Telemetry exfiltration | Schema is `{timestamp, score, case_type, model, mode, attempt}` — no prompt text, FEN, or PII; never read at runtime |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "a58878e253a52973f71d11a0079d396981b2bb66": {"t": 1785749419.3433404, "n": 34291, "h": "36c1266d785cb7de", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nWhen the LLM path fails — validator rejection after the in-pipeline repair\nbudget (`run_mode_2`'s ≤ `MAX_MODE_2_RETRIES` rewrite attempts is exhausted),\noutput-firewall block, or LLM unreachable — the request does not return a\npatched LLM string. It falls through to a deterministic fallback\n(`_build_reply_deterministic` in `llm/seca/coach/chat_pipeline.py`, with the\nparallel path in `llm/seca/coach/live_move_pipeline.py`) that builds the\nreply from scratch using only trusted inputs: the engine signal, the\n`SafeExplainer` output, and a deterministic context block. The LLM's text\nis discarded entirely; the fallback never appends to or edits it. By\nconstruction, the reply cannot contain forbidden phrases because none are\never introduced. This fallback is documented under\n[*Deterministic Fallback*](docs/ARCHITECTURE.md#deterministic-fallback) in\nthe architecture spec, and is forbidden from being widened to accept any\nLLM-derived content.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA is the thin adaptation layer under `llm/seca/`: a 5-action\ncontextual bandit plus a deterministic skill tracker that adapt to\nindividual players without retraining either base model (Stockfish or\nthe LLM). Every adaptation step is closed-form\n(`A ← A + xxᵀ`, `b ← b + r·x` for the bandit; deterministic deltas for\nrating / confidence / weakness / embedding state), and a hard runtime\nguard (`llm/seca/safety/freeze.py`) enforces that nothing heavier — no\ngradient steps, no neural retraining, no autonomous RL — ever runs in\nthe live process.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Reward-signal trust | **Engine-derived.** `/game/finish` recomputes accuracy + weaknesses from the submitted PGN via the engine pool before they reach the bandit, skill tracker, or storage. Falls back to client-supplied values only when the engine pool is unavailable; emits `ACC_FALLBACK` / `ACC_DIVERGENCE` telemetry. See [`docs/SECA.md`](docs/SECA.md#trust-property-of-the-reward-signal). |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The per-request\nverifier (`verify_runtime_safety`) is wired into `GET /seca/status`\nas of 2026-05-14: the endpoint reflects the live runtime, not just\nthe boot-time constant, so a forbidden brain module lazily loaded\nafter startup flips the next status response to `safe_mode: false`\nwithout crashing the process.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/move` | JWT | 30/min | Request opponent move |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/analyze` | API key | 30/min | Engine signal only (no LLM) |\n| POST | `/explain` | API key | — | Full Mode-2 explanation |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| POST | `/explanation_outcome` | API key | 20/min | Post-explanation learning outcome |\n| POST | `/adaptation/mode` | JWT | — | Set adaptation mode for a player |\n| GET | `/adaptation/mode` | JWT | — | Read current adaptation mode |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n### API schema versioning\n\nEvery response carries an `X-API-Version` header pinned at `1`. The\nAndroid client sends the same header on coaching requests; the server\ngates on it (Phase 1 — **lenient on missing**, **strict on mismatch**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| `1` (matches `API_VERSION`) | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming both versions |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read the server's current\nversion and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`.\n\nBumping the version requires a **coordinated server + Android release** —\nbump `API_VERSION` in `llm/server.py` and `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` in the same\nPR. Pinned by `llm/tests/test_api_version_header.py` (AVH_01..AVH_10).\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point — also hosts /engine/eval\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Telemetry exfiltration | Schema is `{timestamp, score, case_type, model, mode, attempt}` — no prompt text, FEN, or PII; never read at runtime |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "e5dc10c8579ab829211dffb65ac0a373e13defbe": {"t": 1785749420.763219, "n": 34267, "h": "717199f132818679", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nWhen the LLM path fails — validator rejection after the in-pipeline repair\nbudget (`run_mode_2`'s ≤ `MAX_MODE_2_RETRIES` rewrite attempts is exhausted),\noutput-firewall block, or LLM unreachable — the request does not return a\npatched LLM string. It falls through to a deterministic fallback\n(`_build_reply_deterministic` in `llm/seca/coach/chat_pipeline.py`, with the\nparallel path in `llm/seca/coach/live_move_pipeline.py`) that builds the\nreply from scratch using only trusted inputs: the engine signal, the\n`SafeExplainer` output, and a deterministic context block. The LLM's text\nis discarded entirely; the fallback never appends to or edits it. By\nconstruction, the reply cannot contain forbidden phrases because none are\never introduced. This fallback is documented under\n[*Deterministic Fallback*](docs/ARCHITECTURE.md#deterministic-fallback) in\nthe architecture spec, and is forbidden from being widened to accept any\nLLM-derived content.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA is the thin adaptation layer under `llm/seca/`: a 5-action\ncontextual bandit plus a deterministic skill tracker that adapt to\nindividual players without retraining either base model (Stockfish or\nthe LLM). Every adaptation step is closed-form\n(`A ← A + xxᵀ`, `b ← b + r·x` for the bandit; deterministic deltas for\nrating / confidence / weakness / embedding state), and a hard runtime\nguard (`llm/seca/safety/freeze.py`) enforces that nothing heavier — no\ngradient steps, no neural retraining, no autonomous RL — ever runs in\nthe live process.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Reward-signal trust | **Engine-derived.** `/game/finish` recomputes accuracy + weaknesses from the submitted PGN via the engine pool before they reach the bandit, skill tracker, or storage. Falls back to client-supplied values only when the engine pool is unavailable; emits `ACC_FALLBACK` / `ACC_DIVERGENCE` telemetry. See [`docs/SECA.md`](docs/SECA.md#trust-property-of-the-reward-signal). |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. A per-request\nverifier with the same structural checks (`verify_runtime_safety`)\nexists in `freeze.py` for future wiring; `GET /seca/status` currently\nreturns the module-level `SAFE_MODE` flag only — the Android client's\nsafety gate reflects the boot-time state, not a per-request re-scan.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/move` | JWT | 30/min | Request opponent move |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/analyze` | API key | 30/min | Engine signal only (no LLM) |\n| POST | `/explain` | API key | — | Full Mode-2 explanation |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| POST | `/explanation_outcome` | API key | 20/min | Post-explanation learning outcome |\n| POST | `/adaptation/mode` | JWT | — | Set adaptation mode for a player |\n| GET | `/adaptation/mode` | JWT | — | Read current adaptation mode |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n### API schema versioning\n\nEvery response carries an `X-API-Version` header pinned at `1`. The\nAndroid client sends the same header on coaching requests; the server\ngates on it (Phase 1 — **lenient on missing**, **strict on mismatch**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| `1` (matches `API_VERSION`) | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming both versions |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read the server's current\nversion and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`.\n\nBumping the version requires a **coordinated server + Android release** —\nbump `API_VERSION` in `llm/server.py` and `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` in the same\nPR. Pinned by `llm/tests/test_api_version_header.py` (AVH_01..AVH_10).\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point — also hosts /engine/eval\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Telemetry exfiltration | Schema is `{timestamp, score, case_type, model, mode, attempt}` — no prompt text, FEN, or PII; never read at runtime |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "e7751a9b9aa6990e0805a6d7e718a5106e26d7a0": {"t": 1785749422.007883, "n": 34115, "h": "25e631f47df9218f", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nWhen the LLM path fails — validator rejection after the in-pipeline repair\nbudget (`run_mode_2`'s ≤ `MAX_MODE_2_RETRIES` rewrite attempts is exhausted),\noutput-firewall block, or LLM unreachable — the request does not return a\npatched LLM string. It falls through to a deterministic fallback\n(`_build_reply_deterministic` in `llm/seca/coach/chat_pipeline.py`, with the\nparallel path in `llm/seca/coach/live_move_pipeline.py`) that builds the\nreply from scratch using only trusted inputs: the engine signal, the\n`SafeExplainer` output, and a deterministic context block. The LLM's text\nis discarded entirely; the fallback never appends to or edits it. By\nconstruction, the reply cannot contain forbidden phrases because none are\never introduced. This fallback is documented under\n[*Deterministic Fallback*](docs/ARCHITECTURE.md#deterministic-fallback) in\nthe architecture spec, and is forbidden from being widened to accept any\nLLM-derived content.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA is the thin adaptation layer under `llm/seca/`: a 5-action\ncontextual bandit plus a deterministic skill tracker that adapt to\nindividual players without retraining either base model (Stockfish or\nthe LLM). Every adaptation step is closed-form\n(`A ← A + xxᵀ`, `b ← b + r·x` for the bandit; deterministic deltas for\nrating / confidence / weakness / embedding state), and a hard runtime\nguard (`llm/seca/safety/freeze.py`) enforces that nothing heavier — no\ngradient steps, no neural retraining, no autonomous RL — ever runs in\nthe live process.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Reward-signal trust | Client-supplied `accuracy` / `weaknesses` on `/game/finish` is currently trusted on faith; server-side PGN re-analysis is the planned mitigation. See [`docs/SECA.md`](docs/SECA.md#trust-property-of-the-reward-signal). |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. A per-request\nverifier with the same structural checks (`verify_runtime_safety`)\nexists in `freeze.py` for future wiring; `GET /seca/status` currently\nreturns the module-level `SAFE_MODE` flag only — the Android client's\nsafety gate reflects the boot-time state, not a per-request re-scan.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/move` | JWT | 30/min | Request opponent move |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/analyze` | API key | 30/min | Engine signal only (no LLM) |\n| POST | `/explain` | API key | — | Full Mode-2 explanation |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| POST | `/explanation_outcome` | API key | 20/min | Post-explanation learning outcome |\n| POST | `/adaptation/mode` | JWT | — | Set adaptation mode for a player |\n| GET | `/adaptation/mode` | JWT | — | Read current adaptation mode |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n### API schema versioning\n\nEvery response carries an `X-API-Version` header pinned at `1`. The\nAndroid client sends the same header on coaching requests; the server\ngates on it (Phase 1 — **lenient on missing**, **strict on mismatch**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| `1` (matches `API_VERSION`) | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming both versions |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read the server's current\nversion and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`.\n\nBumping the version requires a **coordinated server + Android release** —\nbump `API_VERSION` in `llm/server.py` and `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` in the same\nPR. Pinned by `llm/tests/test_api_version_header.py` (AVH_01..AVH_10).\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point — also hosts /engine/eval\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Telemetry exfiltration | Schema is `{timestamp, score, case_type, model, mode, attempt}` — no prompt text, FEN, or PII; never read at runtime |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "b23e9739421c628b2aaa6f5fdd9a05d1becc7587": {"t": 1785749423.304114, "n": 33715, "h": "0d68d819b5d282a7", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nWhen the LLM path fails — validator rejection after the in-pipeline repair\nbudget (`run_mode_2`'s ≤ `MAX_MODE_2_RETRIES` rewrite attempts is exhausted),\noutput-firewall block, or LLM unreachable — the request does not return a\npatched LLM string. It falls through to a deterministic fallback\n(`_build_reply_deterministic` in `llm/seca/coach/chat_pipeline.py`, with the\nparallel path in `llm/seca/coach/live_move_pipeline.py`) that builds the\nreply from scratch using only trusted inputs: the engine signal, the\n`SafeExplainer` output, and a deterministic context block. The LLM's text\nis discarded entirely; the fallback never appends to or edits it. By\nconstruction, the reply cannot contain forbidden phrases because none are\never introduced. This fallback is documented under\n[*Deterministic Fallback*](docs/ARCHITECTURE.md#deterministic-fallback) in\nthe architecture spec, and is forbidden from being widened to accept any\nLLM-derived content.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA (Self-Evolving Coaching Architecture) is the framework underneath\n`llm/seca/`. It defines a third path between *static AI* (strong but\nnon-adaptive) and *self-improving AI* (powerful but unstable): the\nunderlying intelligence — the chess engine and the language model — stays\nfixed, while a thin **decision layer** (contextual bandit + lightweight\nembeddings + deterministic skill trackers) adapts in real time.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Per-request safety verification | `verify_runtime_safety()` runs on every `GET /seca/status` |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The same structural\nchecks run on every `GET /seca/status` (request-time twin) so the Android\nclient's safety gate reflects current state, not a cached boot-time flag.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/move` | JWT | 30/min | Request opponent move |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/analyze` | API key | 30/min | Engine signal only (no LLM) |\n| POST | `/explain` | API key | — | Full Mode-2 explanation |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| POST | `/explanation_outcome` | API key | 20/min | Post-explanation learning outcome |\n| POST | `/adaptation/mode` | JWT | — | Set adaptation mode for a player |\n| GET | `/adaptation/mode` | JWT | — | Read current adaptation mode |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n### API schema versioning\n\nEvery response carries an `X-API-Version` header pinned at `1`. The\nAndroid client sends the same header on coaching requests; the server\ngates on it (Phase 1 — **lenient on missing**, **strict on mismatch**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| `1` (matches `API_VERSION`) | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming both versions |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read the server's current\nversion and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`.\n\nBumping the version requires a **coordinated server + Android release** —\nbump `API_VERSION` in `llm/server.py` and `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` in the same\nPR. Pinned by `llm/tests/test_api_version_header.py` (AVH_01..AVH_10).\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point — also hosts /engine/eval\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Telemetry exfiltration | Schema is `{timestamp, score, case_type, model, mode, attempt}` — no prompt text, FEN, or PII; never read at runtime |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "3cb5ddd5e589ce5a4b3b2e8cad60db86f1d3060e": {"t": 1785749424.5800662, "n": 33267, "h": "7da8b5bdd6bd757b", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nThere is exactly one place in the pipeline where deterministic content is\nappended to LLM-generated text: the fail-safe in `run_mode_2.py:354-375`,\nwhich guarantees that responses for `missing_data` and `forced_mate` cases\ncontain a fixed acknowledgement phrase even when the LLM fails the contract\nafter its repair budget. That fail-safe is documented in detail under\n[*Deterministic Phrase-Completion Fail-Safe*](docs/ARCHITECTURE.md) in the\narchitecture spec, and is forbidden from being widened.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA (Self-Evolving Coaching Architecture) is the framework underneath\n`llm/seca/`. It defines a third path between *static AI* (strong but\nnon-adaptive) and *self-improving AI* (powerful but unstable): the\nunderlying intelligence — the chess engine and the language model — stays\nfixed, while a thin **decision layer** (contextual bandit + lightweight\nembeddings + deterministic skill trackers) adapts in real time.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Per-request safety verification | `verify_runtime_safety()` runs on every `GET /seca/status` |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The same structural\nchecks run on every `GET /seca/status` (request-time twin) so the Android\nclient's safety gate reflects current state, not a cached boot-time flag.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/move` | JWT | 30/min | Request opponent move |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/analyze` | API key | 30/min | Engine signal only (no LLM) |\n| POST | `/explain` | API key | — | Full Mode-2 explanation |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| POST | `/explanation_outcome` | API key | 20/min | Post-explanation learning outcome |\n| POST | `/adaptation/mode` | JWT | — | Set adaptation mode for a player |\n| GET | `/adaptation/mode` | JWT | — | Read current adaptation mode |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n### API schema versioning\n\nEvery response carries an `X-API-Version` header pinned at `1`. The\nAndroid client sends the same header on coaching requests; the server\ngates on it (Phase 1 — **lenient on missing**, **strict on mismatch**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| `1` (matches `API_VERSION`) | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming both versions |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read the server's current\nversion and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`.\n\nBumping the version requires a **coordinated server + Android release** —\nbump `API_VERSION` in `llm/server.py` and `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` in the same\nPR. Pinned by `llm/tests/test_api_version_header.py` (AVH_01..AVH_10).\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point — also hosts /engine/eval\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Telemetry exfiltration | Schema is `{timestamp, score, case_type, model, mode, attempt}` — no prompt text, FEN, or PII; never read at runtime |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "58e4551966ad6d7136119f0cdd84d7a10e329afd": {"t": 1785749426.0163016, "n": 33328, "h": "a89dbc9c2968d50a", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nThere is exactly one place in the pipeline where deterministic content is\nappended to LLM-generated text: the fail-safe in `run_mode_2.py:354-375`,\nwhich guarantees that responses for `missing_data` and `forced_mate` cases\ncontain a fixed acknowledgement phrase even when the LLM fails the contract\nafter its repair budget. That fail-safe is documented in detail under\n[*Deterministic Phrase-Completion Fail-Safe*](docs/ARCHITECTURE.md) in the\narchitecture spec, and is forbidden from being widened.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA (Self-Evolving Coaching Architecture) is the framework underneath\n`llm/seca/`. It defines a third path between *static AI* (strong but\nnon-adaptive) and *self-improving AI* (powerful but unstable): the\nunderlying intelligence — the chess engine and the language model — stays\nfixed, while a thin **decision layer** (contextual bandit + lightweight\nembeddings + deterministic skill trackers) adapts in real time.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Per-request safety verification | `verify_runtime_safety()` runs on every `GET /seca/status` |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The same structural\nchecks run on every `GET /seca/status` (request-time twin) so the Android\nclient's safety gate reflects current state, not a cached boot-time flag.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/move` | JWT | 30/min | Request opponent move |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/analyze` | API key | 30/min | Engine signal only (no LLM) |\n| POST | `/explain` | API key | — | Full Mode-2 explanation |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| POST | `/explanation_outcome` | API key | 20/min | Post-explanation learning outcome |\n| POST | `/adaptation/mode` | JWT | — | Set adaptation mode for a player |\n| GET | `/adaptation/mode` | JWT | — | Read current adaptation mode |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n### API schema versioning\n\nEvery response carries an `X-API-Version` header pinned at `1`. The\nAndroid client sends the same header on coaching requests; the server\ngates on it (Phase 1 — **lenient on missing**, **strict on mismatch**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| `1` (matches `API_VERSION`) | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming both versions |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read the server's current\nversion and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`.\n\nBumping the version requires a **coordinated server + Android release** —\nbump `API_VERSION` in `llm/server.py` and `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` in the same\nPR. Pinned by `llm/tests/test_api_version_header.py` (AVH_01..AVH_10).\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_llm_regression.py`\n4. Real-LLM smoke test passes: `RUN_DEEPSEEK_TESTS=1 COACH_DEEPSEEK_API_KEY=... pytest llm/rag/tests/llm/test_deepseek_smoke.py`\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | DeepSeek API connectivity, output passes validators | local + tag pushes (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_deepseek_smoke.py` |\n| **D — Regression** | Repeated real-LLM runs, contract compliance over time | tag pushes + weekly cron (gated on `COACH_DEEPSEEK_API_KEY`) | `RUN_DEEPSEEK_TESTS=1 pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── elite_engine_service.py\n│   ├── engine_eval.py        # ESV extraction wrapper\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Telemetry exfiltration | Schema is `{timestamp, score, case_type, model, mode, attempt}` — no prompt text, FEN, or PII; never read at runtime |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "98890e4b3ac04488399746936f144fbdc07d3271": {"t": 1785749427.2760973, "n": 33228, "h": "f619c8ded5b1ffd1", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · DeepSeek API (LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a managed DeepSeek API\nclient for prose, and the Mode-2 explanation pipeline that gates everything\nbetween them. A small, deterministic adaptation layer (SECA) personalises\nopponent strength and teaching tone per player without ever retraining a\nmodel.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ DeepSeek API │ │ Postgres│ │ Redis (opt.) │\n                                          │ (managed LLM)│ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nThere is exactly one place in the pipeline where deterministic content is\nappended to LLM-generated text: the fail-safe in `run_mode_2.py:354-375`,\nwhich guarantees that responses for `missing_data` and `forced_mate` cases\ncontain a fixed acknowledgement phrase even when the LLM fails the contract\nafter its repair budget. That fail-safe is documented in detail under\n[*Deterministic Phrase-Completion Fail-Safe*](docs/ARCHITECTURE.md) in the\narchitecture spec, and is forbidden from being widened.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA (Self-Evolving Coaching Architecture) is the framework underneath\n`llm/seca/`. It defines a third path between *static AI* (strong but\nnon-adaptive) and *self-improving AI* (powerful but unstable): the\nunderlying intelligence — the chess engine and the language model — stays\nfixed, while a thin **decision layer** (contextual bandit + lightweight\nembeddings + deterministic skill trackers) adapts in real time.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Per-request safety verification | `verify_runtime_safety()` runs on every `GET /seca/status` |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The same structural\nchecks run on every `GET /seca/status` (request-time twin) so the Android\nclient's safety gate reflects current state, not a cached boot-time flag.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. LLM coaching is provided by the\n[DeepSeek API](https://platform.deepseek.com); set your key in `.env`:\n\n```bash\necho 'COACH_DEEPSEEK_API_KEY=sk-...' >> .env\n```\n\nWithout it, every `/chat` call falls back to the deterministic template\n(`chat_pipeline.py:557` — see the trust-boundary diagram). The API still\nserves; coaching just degrades to canned responses. `GET /llm/health`\nsurfaces the live LLM status so you can detect this in monitoring rather\nthan only in chat replies.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. The DeepSeek API key\nstill needs to be set in `.env` (or as a shell env var).\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/move` | JWT | 30/min | Request opponent move |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/analyze` | API key | 30/min | Engine signal only (no LLM) |\n| POST | `/explain` | API key | — | Full Mode-2 explanation |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| POST | `/explanation_outcome` | API key | 20/min | Post-explanation learning outcome |\n| POST | `/adaptation/mode` | JWT | — | Set adaptation mode for a player |\n| GET | `/adaptation/mode` | JWT | — | Read current adaptation mode |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n### API schema versioning\n\nEvery response carries an `X-API-Version` header pinned at `1`. The\nAndroid client sends the same header on coaching requests; the server\ngates on it (Phase 1 — **lenient on missing**, **strict on mismatch**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| `1` (matches `API_VERSION`) | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming both versions |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read the server's current\nversion and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`.\n\nBumping the version requires a **coordinated server + Android release** —\nbump `API_VERSION` in `llm/server.py` and `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` in the same\nPR. Pinned by `llm/tests/test_api_version_header.py` (AVH_01..AVH_10).\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_DEEPSEEK_API_KEY` | — | **Required for LLM coaching**. Sign up at [platform.deepseek.com](https://platform.deepseek.com), create a key, paste here. Without it the api still serves but every `/chat` call falls back to the deterministic template. |\n| `COACH_DEEPSEEK_API_BASE` | `https://api.deepseek.com` | OpenAI-compatible endpoint. Override only when pointing at a self-hosted gateway (LiteLLM, vLLM, etc.). |\n| `COACH_DEEPSEEK_MODEL` | `deepseek-chat` | DeepSeek-V3. Strong general-purpose; ~$0.14/M input + $0.28/M output. Alternative `deepseek-reasoner` (chain-of-thought) is ~4× cost — usually overkill for explain-the-position prose. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres + Redis. LLM coaching via DeepSeek API. |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass (`test_llm_regression.py` — currently disabled; needs follow-up to wire against DeepSeek)\n4. Real LLM smoke test passes (`test_ollama_smoke.py` — currently disabled; needs follow-up to wire against DeepSeek)\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | Real LLM connectivity, output passes validators | local only | `pytest llm/rag/tests/llm/test_ollama_smoke.py` *(legacy name; needs follow-up rewrite to hit DeepSeek)* |\n| **D — Regression** | Repeated real LLM runs, contract compliance over time | tag pushes only | `pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── elite_engine_service.py\n│   ├── engine_eval.py        # ESV extraction wrapper\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Fake adapter (real LLM via call_llm → DeepSeek)\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Telemetry exfiltration | Schema is `{timestamp, score, case_type, model, mode, attempt}` — no prompt text, FEN, or PII; never read at runtime |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "c18800e2e3c220e6d76d181589b10f070ed75a10": {"t": 1785749428.5728407, "n": 32421, "h": "1614b192cf9f16ff", "c": "# Cereveon\n\n**A chess coaching system that explains, but never invents.**\n\nCereveon decouples the three roles of a chess engine — picking moves, judging\npositions, and explaining what's happening — into independently verifiable\nlayers, then enforces by code that no layer ever crosses into another's job.\nThe opponent engine never explains. Stockfish never plays. The language model\nnever calculates.\n\nThe result is an AI tutor whose explanations are auditable end-to-end: every\nsentence the user reads is a function of a deterministic engine signal, a\nrule-based document retrieval, a fixed-order prompt template, and a contract\nvalidator that has the final say. The architecture exists to make\nhallucination structurally impossible, not to detect it after the fact.\n\n| | |\n|---|---|\n| **Status** | Source-available · production deployed |\n| **Backend** | Python 3.13 · FastAPI · Stockfish pool · Ollama (local LLM) |\n| **Client** | Android (Kotlin) · native C++ ~1800 Elo opponent via JNI |\n| **Tests** | 1 723 passing · coverage 94.4 % · 95 % floor on validators |\n| **License** | See [`docs/LICENSE.md`](docs/LICENSE.md) |\n\n---\n\n## Table of Contents\n\n1. [What this is](#what-this-is)\n2. [At a glance](#at-a-glance)\n3. [Architecture](#architecture)\n4. [SECA — adaptation without retraining](#seca--adaptation-without-retraining)\n5. [Quick start](#quick-start)\n6. [API](#api)\n7. [Configuration](#configuration)\n8. [Production deployment](#production-deployment)\n9. [Testing](#testing)\n10. [Repository structure](#repository-structure)\n11. [Security](#security)\n12. [Contributing](#contributing)\n13. [Further reading](#further-reading)\n\n---\n\n## What this is\n\nCereveon is a coaching application built around a deliberate constraint:\n**explanation is a separate concern from move generation, and must be\ntreated as untrusted output.** Every other design choice flows from that\npremise.\n\n| Cereveon **is** | Cereveon **is not** |\n|---|---|\n| A non-calculating chess explainer | A chess engine, or a competitor to one |\n| A tutor that grounds prose in engine truth | A move-recommendation service |\n| A safety-enforced LLM application | A general-purpose chat product |\n| Auditable end-to-end | Personalised in ways that bypass the contract |\n| Source-available | Open-source (see licence) |\n\nThe system runs an Android client against a FastAPI backend; the backend\nhouses a Stockfish process pool for evaluation, a local Ollama LLM for prose,\nand the Mode-2 explanation pipeline that gates everything between them. A\nsmall, deterministic adaptation layer (SECA) personalises opponent strength\nand teaching tone per player without ever retraining a model.\n\n---\n\n## At a glance\n\n```\n┌────────────────────┐     HTTPS / X-Api-Key + JWT     ┌──────────────────────┐\n│ Android client     │ ───────────────────────────────►│ FastAPI server       │\n│ • board UI         │                                  │ • Stockfish pool     │\n│ • SECA gate poll   │                                  │ • Mode-2 pipeline    │\n│ • C++ opponent     │                                  │ • SECA freeze guard  │\n│   (JNI, ~1800 Elo) │                                  │ • Auth (JWT + HMAC)  │\n└────────────────────┘                                  └────────┬─────────────┘\n                                                                 │\n                                                  ┌──────────────┼──────────────┐\n                                                  ▼              ▼              ▼\n                                          ┌──────────────┐ ┌─────────┐ ┌──────────────┐\n                                          │ Ollama       │ │ Postgres│ │ Redis (opt.) │\n                                          │ (local LLM)  │ │ (auth + │ │ (move L2)    │\n                                          │              │ │  events)│ │              │\n                                          └──────────────┘ └─────────┘ └──────────────┘\n```\n\nTwo engines, two distinct roles:\n\n- **Opponent engine** — bundled C++, ~1800 Elo, runs on-device through the\n  Android JNI bridge. Sole job: pick the opponent's move.\n- **Stockfish pool** — server-side, pooled subprocesses. Sole job: produce\n  the engine signal (ESV) consumed by the Mode-2 pipeline.\n\n---\n\n## Architecture\n\nThe full specification lives in [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md).\nThe summary below states the load-bearing rules; details belong in the spec.\n\n### Mode-2 data flow\n\n```\nStockfish JSON  (ground truth)\n       ↓\nEngine Signal Vector  (deterministic, lossy, no PV / depth / scores)\n       ↓\nRAG retrieval  (rule-based, no embeddings, no vector DB)\n       ↓\nMode-2 prompt rendering  (fixed injection order, golden-tested)\n       ↓\nLLM generation  (untrusted, only stochastic step)\n       ↓\nOutput validators  (hard gate, never bypassed)\n       ↓\nBounded retries  (≤ 2 quality retries, never to recover from a safety violation)\n       ↓\nFinal response\n```\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted) → validators → response](docs/architecture-diagram.svg)\n\nNo step may be skipped or reordered. Validators are the only place a final\nverdict on output is allowed.\n\n### Architectural invariants\n\n| Invariant | Enforced by |\n|---|---|\n| Opponent engine never explains | Layer isolation; on-device, never reaches LLM |\n| Stockfish never selects opponent moves | Separate pool, separate API surface |\n| LLM never calculates or suggests moves | Output validators (hard gate) |\n| ESV is the sole engine-derived input downstream | `extract_engine_signal` is the only source |\n| Prompt injection order is fixed | Golden snapshot tests on the rendered prompt |\n| Explanations only after a move is committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary in code |\n\n### Trust boundaries\n\n| Component | Trust | Deterministic |\n|---|---|---|\n| Stockfish JSON | Trusted | ✅ |\n| Engine Signal Vector (ESV) | Trusted | ✅ |\n| RAG document corpus | Trusted | ✅ |\n| Prompt renderer | Trusted | ✅ |\n| **LLM output** | **Untrusted** | ❌ |\n| Output validators + firewall | Trusted | ✅ |\n\nNon-determinism is isolated to LLM generation. Everything else is\nreproducible from inputs.\n\n### Output validators\n\n`llm/rag/validators/` and `llm/rag/safety/output_firewall.py` reject any\nLLM response that:\n\n- mentions the engine, depth, search nodes, or principal variations\n- suggests, names, or invents chess moves (algebraic notation, castling)\n- claims a forced mate the engine didn't see, or omits the inevitability\n  acknowledgement when the engine *did* see one\n- discloses the system prompt, claims an alternate identity, or leaks PII\n\nA failed validator is a hard stop. The bounded retry mechanism exists only\nto improve *quality* of an already-passing response — not to recover from a\nsafety violation.\n\nThere is exactly one place in the pipeline where deterministic content is\nappended to LLM-generated text: the fail-safe in `run_mode_2.py:354-375`,\nwhich guarantees that responses for `missing_data` and `forced_mate` cases\ncontain a fixed acknowledgement phrase even when the LLM fails the contract\nafter its repair budget. That fail-safe is documented in detail under\n[*Deterministic Phrase-Completion Fail-Safe*](docs/ARCHITECTURE.md) in the\narchitecture spec, and is forbidden from being widened.\n\n---\n\n## SECA — adaptation without retraining\n\nSECA (Self-Evolving Coaching Architecture) is the framework underneath\n`llm/seca/`. It defines a third path between *static AI* (strong but\nnon-adaptive) and *self-improving AI* (powerful but unstable): the\nunderlying intelligence — the chess engine and the language model — stays\nfixed, while a thin **decision layer** (contextual bandit + lightweight\nembeddings + deterministic skill trackers) adapts in real time.\n\nThe full specification is in [`docs/SECA.md`](docs/SECA.md). The\nload-bearing facts:\n\n| Property | Status in this build |\n|---|---|\n| Six-step loop (input → action → output → reward → update → repeat) | All steps live |\n| Bandit decision head | Closed-form LinUCB (`A ← A + xxᵀ`, `b ← b + r·x`); shadow warm-up by default, user-visible behind `SECA_USE_BANDIT_COACH=1` |\n| Online updates to base models (engine, LLM) | **Forbidden** by the freeze guard |\n| Background training tasks | **Forbidden** at startup |\n| Per-request safety verification | `verify_runtime_safety()` runs on every `GET /seca/status` |\n\n### Freeze guard\n\n`llm/seca/safety/freeze.py` enforces the no-retraining rule with three\nindependent checks at startup, plus a per-request structural twin:\n\n1. **Brain-tree allowlist** — anything under `llm.seca.brain.*` not on the\n   tiny allowlist (schema modules + `context_builder` + `experience_store`\n   + `decision`) is forbidden.\n2. **Forbidden module-name parts** — substring matches against historic\n   adaptive components (e.g. `brain.rl`, `brain.bandit.online`).\n3. **Forbidden source keywords** — substring matches against module\n   *source text*: `optimizer.step`, `loss.backward`, `.partial_fit(`,\n   `train(`, `bandit.update`, `bandit.save`, `import torch`, `nn.Module`.\n\nA violation `sys.exit(1)`s the process at startup. The same structural\nchecks run on every `GET /seca/status` (request-time twin) so the Android\nclient's safety gate reflects current state, not a cached boot-time flag.\n\n### Dormant-code policy\n\nEarlier revisions of this codebase carried a substantial volume of dormant\nRL/ML code. Five deletion sweeps removed it; what remains under `seca/` is\neither live or a deliberately-kept allowlisted stub. Reviving any of the\ndeleted research is a deliberate act — the relevant module has to be\nrewritten, the freeze allowlist updated, determinism guarantees\ndocumented, and tests added that pin the behaviour. The freeze guard's\nkeyword scan is the re-introduction tripwire.\n\n---\n\n## Quick start\n\n### Docker (recommended)\n\n```bash\ncp .env.example .env       # fill in values if you want non-defaults\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. Requires [Ollama](https://ollama.ai)\nrunning on the **host** (not in the container):\n\n```bash\nollama pull qwen2.5:7b-instruct-q2_K\nollama serve\n```\n\n`host.docker.internal` is mapped automatically on macOS, Windows, and\nLinux via `extra_hosts` in `docker-compose.yml`.\n\n### VS Code dev container\n\nOpen the repo and choose **\"Reopen in Container\"**. Provisions Python 3.13,\nNode.js 22, Stockfish, and all Python dependencies. Ollama still runs on\nthe host.\n\n### Bare-metal Python\n\n```bash\nsudo apt install stockfish      # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected from `PATH`, falling back to\n`/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows).\nOverride with `STOCKFISH_PATH`.\n\n### Android\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file\nautomatically. The app builds for `arm64-v8a` (physical devices, Apple\nSilicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n---\n\n## API\n\nThe backend exposes a FastAPI application at `llm/server.py`. The\nauthoritative schema reference is\n[`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md); the table below lists\nthe endpoint surface.\n\n### Authentication\n\nTwo layers, both enforced server-side:\n\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header. Required on coaching\n   endpoints. Bypassed in `SECA_ENV=dev` *only* when\n   `SECA_INSECURE_DEV=true`; defaults to enforced in dev too.\n2. **JWT session tokens** — issued by `/auth/register` and `/auth/login`\n   for player sessions. `Authorization: Bearer <token>`. Sliding refresh\n   via the `X-Auth-Token` rotation header.\n\nConstant-time comparison (`hmac.compare_digest`) is used for the API-key\ncheck. A missing `SECA_API_KEY` or short `SECRET_KEY` (< 32 chars) crashes\nthe server at startup when `SECA_ENV=prod` — by design.\n\n### Endpoint catalogue\n\n| Method | Path | Auth | Rate limit | Description |\n|---|---|---|---|---|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | Per-request SECA verification (safety gate) |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/move` | JWT | 30/min | Request opponent move |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/analyze` | API key | 30/min | Engine signal only (no LLM) |\n| POST | `/explain` | API key | — | Full Mode-2 explanation |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed chat |\n| POST | `/explanation_outcome` | API key | 20/min | Post-explanation learning outcome |\n| POST | `/adaptation/mode` | JWT | — | Set adaptation mode for a player |\n| GET | `/adaptation/mode` | JWT | — | Read current adaptation mode |\n| GET | `/next-training/{player_id}` | JWT | — | Curriculum next-task recommendation |\n| POST | `/game/start` | JWT | — | Open a game record |\n| POST | `/game/{id}/checkpoint` | JWT | — | Cross-device resume checkpoint |\n| GET | `/game/active` | JWT | — | Active game for this player |\n| GET | `/repertoire` | JWT | — | Player repertoire (ECO list) |\n| POST | `/repertoire` | JWT | — | Add an opening |\n| DELETE | `/repertoire/{eco}` | JWT | — | Remove an opening |\n| POST | `/repertoire/{eco}/drill-result` | JWT | — | Record a drill outcome |\n| POST | `/repertoire/{eco}/active` | JWT | — | Set the active opening |\n| POST | `/auth/register` | — | — | Create account |\n| POST | `/auth/login` | — | — | Issue JWT |\n| POST | `/auth/logout` | JWT | — | Invalidate session |\n| GET | `/auth/me` | JWT | — | Current player profile |\n| PATCH | `/auth/me` | JWT | — | Update profile |\n| POST | `/auth/change-password` | JWT | — | Rotate password |\n| POST | `/game/finish` | JWT | — | Close a game; runs SECA loop |\n| POST | `/game/coach-feedback` | JWT | — | Per-game coach feedback |\n| GET | `/game/history` | JWT | — | Recent games |\n\n### API schema versioning\n\nEvery response carries an `X-API-Version` header pinned at `1`. The\nAndroid client sends the same header on coaching requests; the server\ngates on it (Phase 1 — **lenient on missing**, **strict on mismatch**):\n\n| Client header | Server response |\n|---|---|\n| absent | request proceeds; INFO log records the missing-header request |\n| `1` (matches `API_VERSION`) | request proceeds silently |\n| anything else | HTTP 400 with `{\"detail\": \"...\"}` naming both versions |\n\nDiscovery routes (`/`, `/health`, `/seca/status`) never reject on\nmismatch so an out-of-date client can still read the server's current\nversion and surface an \"update the app\" UI. CORS preflights explicitly\nallow `X-API-Version`.\n\nBumping the version requires a **coordinated server + Android release** —\nbump `API_VERSION` in `llm/server.py` and `COACH_API_VERSION` in\n`android/app/src/main/java/ai/chesscoach/app/ApiVersion.kt` in the same\nPR. Pinned by `llm/tests/test_api_version_header.py` (AVH_01..AVH_10).\n\n### Security response headers\n\nEvery response carries:\n\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin\nrequests are blocked with a startup warning. Request bodies are capped at\n512 KB at the FastAPI middleware layer.\n\n---\n\n## Configuration\n\nAll configuration is via environment variables. `.env.example` is the\nauthoritative reference; the table below covers the variables operators\nneed most often.\n\n### Environment variables\n\n| Variable | Default | Description |\n|---|---|---|\n| `SECA_API_KEY` | `dev-key` | Auth key. Any value works in `dev`; required in `prod`. |\n| `SECA_ENV` | `dev` | `dev` or `prod`. |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`). |\n| `COACH_OLLAMA_URL` | `http://host.docker.internal:11434` | Ollama endpoint. |\n| `COACH_OLLAMA_MODEL` | `qwen2.5:7b-instruct-q2_K` | LLM model. **Pin the digest (`model:tag@sha256:...`) in production**, otherwise an upstream tag re-push silently changes behaviour under the LLM regression contract. Recipe in `.env.prod.example`. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path. |\n| `REDIS_URL` | *(unset)* | Redis URL for the L2 move cache; in-memory only when unset. |\n| `DATABASE_URL` | `sqlite:///data/seca.db` | SQLAlchemy DSN. PostgreSQL required for multi-worker deployments. |\n| `CORS_ALLOWED_ORIGINS` | *(empty — blocks all cross-origin)* | Comma-separated allowed origins. |\n| `TRUSTED_PROXIES` | prod: empty (warning logged); dev: `127.0.0.0/8, ::1` | Reverse-proxy IPs / CIDRs. **Required in prod** for per-client rate limiting; otherwise every request behind the reverse proxy keys on the same bucket. See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) > Trusted Proxies. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. **Never set in production** — see [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) § T6. |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes. |\n| `ENGINE_THREADS` | `1` | Threads per Stockfish process. |\n| `ENGINE_HASH_MB` | `128` | Hash table per process (MB). |\n| `ENGINE_SKILL_LEVEL` | `10` | UCI skill level. |\n| `ENGINE_DEFAULT_MOVETIME_MS` | `40` | Default search time. |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Predictive move pre-caching. |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Manual cache flush override. The cache key already fingerprints the engine config, so this is only needed when the Stockfish binary is replaced in place at the same path. |\n| `SECA_USE_BANDIT_COACH` | `false` | When `true`, the LinUCB bandit's selection becomes user-visible in `/game/finish`. Default: shadow warm-up only. |\n\n### Common issues\n\n| Symptom | Resolution |\n|---|---|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM; `NativeEngineProvider` returns null gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n| Rate limits feel \"stuck on one bucket\" | `TRUSTED_PROXIES` is unset in prod; the limiter keys on Caddy's IP for every request. Set the variable to the proxy CIDR. |\n\n---\n\n## Production deployment\n\nProduction runs in **two tiers**, both built and signed by the same CI\npipeline. Full runbook: [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md).\n\n| Tier | Image | Source | Port | Role |\n|---|---|---|---|---|\n| **Fly.io edge** | `cereveon` | root [`Dockerfile`](Dockerfile) → [`llm/server.js`](llm/server.js) (Node + Express) | 3000 | Public ingress, regional distribution, security middleware |\n| **Hetzner backend** | `cereveon-llm-api` | [`llm/Dockerfile.api`](llm/Dockerfile.api) (Python + Stockfish + full SECA stack) | 8000 | Heavy compute: engine pool, RAG, validators, auth, Postgres / Ollama / Redis |\n\nBoth tiers auto-deploy from\n[`.github/workflows/fly-deploy.yml`](.github/workflows/fly-deploy.yml) on\npush to `main`: the `deploy` job runs the zero-downtime rolling swap on\nHetzner (scale=2, health-gate, drain-or-rollback), then `fly-deploy`\nruns `flyctl deploy --image <ghcr-digest>` to update the edge. The\nfilename is historical; the workflow's `name:` field is `CI/CD`, which\nis authoritative.\n\nThe split is intentional and load-bearing: Fly provides regional\ndistribution and low-latency public ingress for a small Node edge that\nis cheap to deploy globally; Hetzner hosts the single heavy backend that\nthe edge proxies to.\n\n### Container hardening\n\n`docker-compose.prod.yml` ships two hardening tiers, applied per\nservice. The aggressive tier (`api`, `redis`) carries `read_only: true`,\n`tmpfs: [/tmp]`, `cap_drop: [ALL]`, and `no-new-privileges`. The\nconservative tier (`caddy`, `db`, `ollama`) carries only\n`no-new-privileges` pending staging validation of upstream-specific\nhardening recipes. Pinned by `llm/tests/test_container_hardening.py`\n(CH_01–CH_13). See [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) §8 for\nthe full rationale.\n\n### Release process\n\nReleases follow `vMAJOR.MINOR.PATCH`:\n\n- **MAJOR** — architectural or contract changes\n- **MINOR** — new features, RAG documents, golden cases\n- **PATCH** — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n\n1. Clean working tree (`git status`)\n2. CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass (`test_llm_regression.py`)\n4. Real LLM smoke test passes (`test_ollama_smoke.py`)\n5. Manual output sanity review (no engine mentions, no move suggestions)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for\nboth `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n---\n\n## Testing\n\nThe project uses six test categories. No layer is unprotected.\n\n| Category | Scope | CI? | Command |\n|---|---|---|---|\n| **A — Golden** | ESV mapping, RAG retrieval, prompt snapshots | ✅ | `pytest llm/rag/tests/golden/` |\n| **B — Contract** | Forbidden patterns, mate handling, missing data (Fake LLM) | ✅ | `pytest llm/rag/tests/contracts/` |\n| **C — Smoke** | Real LLM connectivity, output passes validators | local only | `pytest llm/rag/tests/llm/test_ollama_smoke.py` |\n| **D — Regression** | Repeated real LLM runs, contract compliance over time | tag pushes only | `pytest llm/rag/tests/llm/test_llm_regression.py` |\n| **E — Quality** | Length, sentence structure, non-triviality | advisory | `pytest llm/rag/tests/quality/` |\n| **F — Mutation** | mutmut against `llm/rag/validators/`: does the test fail when the validator is logically wrong? | local, on-demand | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is mandatory: it simulates contract violations to prove\nvalidator enforcement, and is not optional.\n\n### Quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set:\n\n- **Black** — `py313`, line length `100`. Scope: 22-file whitelist in\n  `llm/run_quality_gate.py:FORMAT_TARGETS`.\n- **Pylint** — default rule set MINUS the permissive softeners in\n  `pyproject.toml [tool.pylint.\"messages control\"] disable`\n  (`broad-exception-caught`, `line-too-long`, missing-docstring trio,\n  `subprocess-run-check`, `too-few-public-methods`, the `too-many-*`\n  trio). Every other category is enforced.\n- **Mypy** — 16-file `MYPY_TARGETS`. `python_version = \"3.13\"`,\n  `ignore_missing_imports = true`. Trust-boundary modules\n  (`llm.rag.validators.*`, `llm.rag.safety.*`, `llm.rag.contracts.*`)\n  carry stricter overrides: `disallow_untyped_defs`, `check_untyped_defs`,\n  `disallow_incomplete_defs`, `warn_return_any`.\n- **Coverage** — global ≥ 80 %, validators (`llm/rag/validators/*.py`)\n  and the post-LLM safety firewall (`llm/rag/safety/*.py`) ≥ 95 %.\n  Per-module floors enforced by `llm/check_coverage_thresholds.py`.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` and\n  `llm/requirements-ci.txt`. Any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. `CRITICAL`\n  unfixed → block; `HIGH/CRITICAL` → SARIF upload to the Security tab.\n\n### Hardening tripwires\n\n- **`python -O` regression** — validators on the production path use\n  explicit `if not <cond>: raise AssertionError(...)` rather than bare\n  `assert`. Pinned by\n  `llm/rag/tests/unit/test_validator_dash_o_hardening.py`\n  (`VAL_DASH_O_01`); see the policy block in `pyproject.toml`.\n- **Cross-tenant 404 collapse** — path-id endpoints return the same 404 +\n  `\"Not Found\"` body FastAPI emits for an unmounted URL, removing the\n  enumeration oracle. Pinned by\n  `test_security_authz.py::TestAut01CrossTenantNoLeak`.\n- **Stockfish crash recovery** — a crashed child is detected at release\n  time and replaced with a fresh `_spawn_engine()`; pool size survives\n  the crash. Pinned by `test_engine_pool_crash_recovery.py` (CR_01–CR_08).\n- **Proxy-aware rate limiting** — `TRUSTED_PROXIES` walks XFF\n  right-to-left; spoof attempts cannot escape an IP's bucket. Pinned by\n  `test_security_proxy_aware_limiter.py` (TPA_01–TPA_14).\n\n### Running tests\n\n```bash\npython llm/run_ci_suite.py                       # full CI suite (1 723 tests)\npython llm/run_quality_gate.py black             # formatting\npython llm/run_quality_gate.py pylint            # linting\npython llm/run_quality_gate.py mypy              # types\ncd android && ./gradlew test                     # Android host JVM tests\ncd android && ./gradlew connectedAndroidTest     # Android instrumented tests\n```\n\nReal-LLM regression tests (Category D) must run before every release,\nafter any system-prompt or RAG-document change, and after any model\nupdate — and never in CI.\n\n---\n\n## Repository structure\n\n```\n.\n├── android/                  # Kotlin client (UI, game orchestration, JNI)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI entry point\n│   ├── explain_pipeline.py   # Mode-2 outer pipeline + bounded retries\n│   ├── elite_engine_service.py\n│   ├── engine_eval.py        # ESV extraction wrapper\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic rule-based retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts (≥ 95 % coverage floor)\n│   │   ├── safety/           # Output firewall (≥ 95 % coverage floor)\n│   │   ├── llm/              # BaseLLM + Ollama / Fake adapters\n│   │   ├── llm/run_mode_2.py # Inner repair loop + REQUIRED-phrase fail-safe\n│   │   ├── deploy/embedded.py\n│   │   ├── documents/        # Static RAG corpus (no runtime submission)\n│   │   └── tests/            # Golden, contract, regression, quality, mutation\n│   ├── seca/\n│   │   ├── safety/freeze.py  # Startup + per-request safety enforcement\n│   │   ├── auth/             # JWT issuance + sliding refresh\n│   │   ├── brain/bandit/     # Allowlisted: context_builder, decision (LinUCB), experience_store\n│   │   ├── learning/         # Outcome tracking, skill update, player embedding\n│   │   ├── world_model/      # SafeWorldModel stub only\n│   │   ├── coach/            # Chat + live-move pipelines\n│   │   ├── curriculum/       # Live curriculum router + scheduler\n│   │   ├── adaptation/       # Per-session ELO drift, teaching policy\n│   │   ├── analytics/        # Event logging + training recommendations\n│   │   ├── analysis/         # Read-only historical roll-up\n│   │   ├── events/           # /game/finish event handling\n│   │   ├── storage/          # SQLAlchemy + raw-SQL split\n│   │   ├── engines/stockfish/ # Live engine pool\n│   │   ├── shared_limiter.py # Proxy-aware rate-limit key\n│   │   └── runtime/safe_mode.py\n│   └── tests/                # API contracts, security, hardening, integration\n├── docs/\n│   ├── ARCHITECTURE.md\n│   ├── SECA.md\n│   ├── API_CONTRACTS.md\n│   ├── TESTING.md\n│   ├── THREAT_MODEL.md\n│   ├── DEPLOYMENT.md\n│   ├── OPERATIONS.md\n│   ├── OPERATIONS_RETRIES.md\n│   ├── RELEASE.md\n│   └── LICENSE.md\n├── design/                   # React/Babel design canvas (not part of build)\n├── scripts/                  # Operational helpers (smoke test, mutation runner, …)\n├── docker-compose.yml        # Dev compose stack\n├── docker-compose.prod.yml   # Prod compose stack with container hardening\n└── pyproject.toml            # Black / Mypy / Pylint / pytest config\n```\n\n---\n\n## Security\n\nThe full threat model lives in\n[`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md). Key controls:\n\n| Threat | Control |\n|---|---|\n| Prompt injection via `user_query` | Pre-LLM sanitiser → structured prompt rendering → Mode-2 contracts → output firewall → bounded retries |\n| JWT replay / session theft | HMAC-signed tokens, ≥ 32-char secret, sliding refresh, HTTPS-only, per-IP rate limit |\n| Engine-pool DoS | Bounded pool, fast-fail queue timeout, movetime ceiling, L1+L2 cache, slowapi per-IP, transport-liveness probe on release |\n| Telemetry exfiltration | Schema is `{timestamp, score, case_type, model, mode, attempt}` — no prompt text, FEN, or PII; never read at runtime |\n| Malicious RAG document | No runtime submission surface; corpus is static in-tree, golden-pinned |\n| Operator misconfiguration | Two-flag bypass (`SECA_INSECURE_DEV`); hard startup guards on `SECA_API_KEY` and `SECRET_KEY` in prod; CORS defaults closed |\n\nCross-cutting: constant-time secret compare, security response headers,\n512 KB body cap, container hardening, per-request SECA verification.\n\nThe audit history of past findings — what was open, what's closed,\nwhere in the code — is tracked in\n`memory/project_security_depth_audit.md` (kept out of the public docs\ntree intentionally).\n\n---\n\n## Contributing\n\nThe architecture is the contract. Changes are categorised by what they\ntouch.\n\n### Allowed without architectural review\n\n- Add a RAG document to `llm/rag/documents/`\n- Add a golden test case\n- Tighten an existing validator\n- Improve explanation wording within existing contracts\n- Add a new `BaseLLM` adapter\n\n### Requires architectural review\n\n- New endpoint (extend `THREAT_MODEL.md` §3 with a fresh threat row)\n- Auth path other than JWT or `X-Api-Key`\n- Output firewall category change\n- New telemetry field\n- Change to the `extend-exclude` or quality-gate target lists in\n  `pyproject.toml` and `llm/run_quality_gate.py`\n- Change to the SECA freeze allowlist (`ALLOWED_BRAIN_MODULES`,\n  `FORBIDDEN_KEYWORDS`)\n- New env variable that affects production behaviour\n\n### Forbidden\n\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n- Skipping `--no-verify` on commits\n- Force-pushing to `main`\n\nThe repo's [`CLAUDE.md`](CLAUDE.md) lists the same rules in\nmachine-actionable form for AI assistants working in the tree.\n\n---\n\n## Further reading\n\n| Doc | Purpose |\n|---|---|\n| [`docs/ARCHITECTURE.md`](docs/ARCHITECTURE.md) | Formal system specification, trust boundaries, data flow |\n| [`docs/SECA.md`](docs/SECA.md) | Self-Evolving Coaching Architecture: framework, six-step loop, freeze guard, dormant-code policy |\n| [`docs/API_CONTRACTS.md`](docs/API_CONTRACTS.md) | Authoritative endpoint schemas |\n| [`docs/TESTING.md`](docs/TESTING.md) | Test categories, validator coverage matrix, quality gates |\n| [`docs/THREAT_MODEL.md`](docs/THREAT_MODEL.md) | Adversaries, threats, mitigations, accepted residuals |\n| [`docs/DEPLOYMENT.md`](docs/DEPLOYMENT.md) | Production runbook, two-tier topology, container hardening |\n| [`docs/OPERATIONS.md`](docs/OPERATIONS.md) | Runtime monitoring, telemetry, incident response |\n| [`docs/OPERATIONS_RETRIES.md`](docs/OPERATIONS_RETRIES.md) | Bounded retry policy and telemetry interpretation |\n| [`docs/RELEASE.md`](docs/RELEASE.md) | Mandatory release procedure and invariants |\n| [`docs/LICENSE.md`](docs/LICENSE.md) | Source-available licence terms |\n\n---\n\n## Design philosophy\n\nCereveon prioritises:\n\n1. **Correctness** — invariants enforced via code and tests, not convention\n2. **Determinism** — every layer except LLM generation is reproducible\n3. **Non-hallucination** — ESV normalisation + output validators make engine fabrication structurally impossible\n4. **Safety** — strict contracts on every LLM output; no output is always better than unsafe output\n5. **Maintainability** — loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, and explanation quality at the\nexpense of correctness.\n\n> *No output is always better than unsafe output. If the system refuses\n> to respond, that is a success condition, not a failure.*\n> — `docs/OPERATIONS.md` §12\n"}, "da949ca39283ce98c2f82be676880c8562d5226d": {"t": 1785749429.8141205, "n": 26806, "h": "4bd33b8697d4bbde", "c": "# Cereveon\n\nAI-powered chess coaching system enforcing strict architectural separation between move generation, position evaluation, and natural-language explanation.\n\n## System Overview\n\nCereveon is a source-available mono-repository (see [LICENSE.md](LICENSE.md)) containing four integrated layers:\n\n- **Android App** — UI, gameplay orchestration, and coaching display\n- **C++ Opponent Engine** — ~1800 Elo search via JNI bridge\n- **Stockfish Engine Pool** — pooled analysis instances providing position evaluation (distinct from the opponent engine)\n- **LLM Explanation System (Mode-2)** — RAG-grounded, contract-validated explanations backed by SECA safety enforcement\n\nThe system enforces non-negotiable role invariants: the opponent never explains, Stockfish never plays, the LLM never calculates.\n\n## Core Architecture\n\n### Architectural Roles\n\n```\nMoves are facts.\nEvaluations are judgments.\nExplanations are commentary.\n\nNo component is allowed to blur these roles.\n```\n\n### Design Invariants\n\n| Invariant | Enforcement |\n|-----------|-------------|\n| Opponent engine never explains | Layer isolation |\n| Stockfish never selects moves | Separate pool |\n| LLM never calculates or suggests moves | Output validators |\n| Explanations generated only after moves are committed | Pipeline ordering |\n| No decision-making component depends on LLM output | Trust boundary |\n\n## Mode-2 Pipeline\n\nThe explanation subsystem is designated **Mode-2**: a non-calculating chess explainer. No step may be skipped or reordered.\n\n![Cereveon Mode-2 pipeline — Android client → API → engine truth → ESV → RAG → prompt → LLM (untrusted, amber) → validators → response. Trust boundaries are dashed amber lines.](docs/architecture-diagram.svg)\n\nFor the formal specification of each layer and its invariants, see [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).\n\n```\nStockfish JSON (ground truth)\n        ↓\nEngine Signal Extraction (ESV)          ← deterministic, trust boundary\n        ↓\nRAG Retrieval                           ← deterministic, rule-based (no embeddings)\n        ↓\nPrompt Rendering (Mode-2)               ← fixed injection order, golden-tested\n        ↓\nLLM Generation                          ← untrusted, stochastic\n        ↓\nOutput Validation                       ← hard gate, never bypassed\n        ↓\nBounded Retry (≤ 2 retries)            ← quality only, not safety\n        ↓\nFinal Response\n```\n\n### Engine Signal Vector (ESV)\n\nThe ESV is the normalized, loss-limited representation of Stockfish output. It is the sole engine-derived input permitted downstream of the evaluator.\n\nProperties:\n- Extracted deterministically from raw Stockfish JSON\n- Coarsened to bands — no raw centipawn scores or numeric precision\n- No move lists, no principal variations, no search metadata\n- Downstream components receive the ESV and nothing else from the engine\n\n### RAG Retrieval\n\nDocument selection is deterministic and rule-based. There is no embedding similarity, no semantic search, and no vector database. Retrieval conditions are explicit: documents are selected based solely on ESV values. The document corpus covers tactics, pawn structure, endgame principles, positional concepts, and meta-coaching topics.\n\n**Trade-off (deterministic vs. semantic retrieval).** Rule-based retrieval is the design choice that makes the rest of this system auditable: every document selection is reproducible from the ESV, golden-testable, and inspectable line-by-line in `llm/rag/retriever/rule_matcher.py`. The cost is recall on positions whose ESV signature matches no rule in the document corpus — that case is real, not hypothetical, and the failure is observable: `retrieve()` returns an empty list, `render_mode_2_prompt` substitutes the placeholder string `(no retrieved context)` into the RAG section, and the LLM is forced to operate against the system prompt and engine signal alone without supporting positional concepts. We chose this fallback (degraded but not refused) deliberately — refusing to respond would punish the user for the corpus' gaps; the validators downstream (output firewall + Mode-2 contracts) still enforce the safety claims regardless of how thin the context block is. The mitigation for sustained empty-retrieval rates is corpus expansion, never a switch to semantic search.\n\n### LLM Layer\n\nThe LLM implements `BaseLLM.generate(prompt: str) -> str` — no additional methods are permitted. It may rephrase, explain, and contextualize. It may not reason beyond provided inputs, introduce new facts, or contradict the engine evaluation. The LLM is always treated as untrusted.\n\n### Output Validation\n\nAll LLM outputs pass contract validation before being returned. Validation is a hard gate — failure stops execution immediately and is never bypassed.\n\nEnforced contracts:\n- No engine tool mentions\n- No move suggestions or algebraic notation\n- No invented tactics\n- Correct forced-mate handling (inevitability emphasized, no long-term planning)\n- Explicit refusal when required engine data is missing\n\n### Bounded Retries\n\nRetries exist only to improve explanation *quality*, not to recover from safety violations. Hard limits:\n\n| Parameter | Value |\n|-----------|-------|\n| Maximum retries | 2 |\n| Total attempts | 3 |\n| Prompt changes between retries | Not allowed |\n| Temperature changes | Not allowed |\n| Validator bypass | Not allowed |\n\nA retry is triggered only when output passes validation but scores below the quality threshold. Any validation failure is a hard stop — no retry.\n\n## SECA Safety Enforcement\n\nThe **SECA** (Safety-Enforced Coaching Architecture) safety layer is enforced at server startup via `llm/seca/safety/freeze.py`. The `SafeWorldModel` is instantiated and passed to `enforce()` before any request is served.\n\nRuntime guarantees:\n- No online training\n- No bandit updates\n- No world model learning\n- No background adaptive loops\n- Deterministic runtime (non-determinism isolated to LLM generation only)\n\n`GET /seca/status` returns the current runtime safety flags. The Android client reads this at cold-start to confirm `safe_mode: true` before sending coaching requests.\n\n## Trust Boundaries\n\n| Component | Trust Level | Deterministic |\n|-----------|-------------|---------------|\n| Stockfish JSON | Trusted | Yes |\n| Engine Signal (ESV) | Trusted | Yes |\n| RAG Documents | Trusted | Yes |\n| Prompt Renderer | Trusted | Yes |\n| LLM Output | Untrusted | No |\n| Output Validators | Trusted | Yes |\n\nNon-determinism is strictly isolated to LLM generation.\n\n## API Reference\n\nThe backend exposes a FastAPI application (`llm/server.py`). All endpoints require `X-Api-Key` authentication unless noted. Rate limiting is applied per-IP via `slowapi`. Request bodies are capped at 512 KB.\n\n### Authentication\n\nAuthentication uses two layers:\n1. **API key** — `X-Api-Key: <SECA_API_KEY>` header on protected endpoints\n2. **JWT session tokens** — issued by `/auth/*` routes for player sessions (`Authorization: Bearer <token>`)\n\nIn `SECA_ENV=dev`, the API key check is bypassed. In `SECA_ENV=prod`, `SECA_API_KEY` must be set or the server refuses to start.\n\n### Endpoints\n\n| Method | Path | Auth | Rate limit | Description |\n|--------|------|------|-----------|-------------|\n| GET | `/` | — | — | Liveness probe |\n| GET | `/health` | — | — | Health check |\n| GET | `/seca/status` | — | — | SECA safety flags |\n| GET | `/debug/engine` | API key | — | Engine pool depth |\n| POST | `/move` | JWT | 30/min | Request opponent move |\n| POST | `/live/move` | JWT | 30/min | Real-time coaching hint on player move |\n| POST | `/analyze` | API key | 30/min | Engine signal only (no LLM) |\n| POST | `/explain` | API key | — | Full Mode-2 explanation |\n| POST | `/chat` | API key | 10/min | Multi-turn coaching conversation |\n| POST | `/chat/stream` | API key | 10/min | SSE-streamed coaching conversation |\n| POST | `/explanation_outcome` | API key | 20/min | Report post-explanation learning outcome |\n\n### `POST /move`\n\nRequests the opponent engine to select a move for a given position.\n\n```json\n{\n  \"fen\": \"<FEN or 'startpos'>\",\n  \"moves_uci\": [\"e2e4\", \"e7e5\"],\n  \"mode\": \"default | blitz | analysis | training\",\n  \"movetime_ms\": 40\n}\n```\n\nResponse includes `uci`, `san`, `opponent_elo`, `cache_hit`, and telemetry (`latency_ms`, `engine_time_ms`, `cache_hit_rate`, `queue_depth`).\n\nThe engine pool maintains a two-level move cache (in-memory L1, optional Redis L2). Cache hits bypass engine computation entirely. Predictive pre-caching runs asynchronously after each move to warm follow-up positions.\n\nOpponent Elo is computed dynamically via the adaptation layer based on the authenticated player's rating and confidence.\n\n### `POST /live/move`\n\nGenerates a real-time coaching hint immediately after a player move, before the opponent responds. Returns `hint`, `engine_signal`, `move_quality`, and `mode`.\n\n### `POST /analyze`\n\nReturns the ESV for a position without invoking the LLM.\n\n```json\n{\n  \"fen\": \"<FEN>\",\n  \"stockfish_json\": { ... },\n  \"user_query\": \"\"\n}\n```\n\n### `POST /explain`\n\nFull Mode-2 pipeline: ESV → RAG → prompt → LLM → validate → return.\n\n```json\n{\n  \"fen\": \"<FEN>\",\n  \"stockfish_json\": { ... },\n  \"user_query\": \"Why is this position difficult?\"\n}\n```\n\nResponse:\n```json\n{\n  \"explanation\": \"...\",\n  \"engine_signal\": { ... },\n  \"mode\": \"SAFE_V1\"\n}\n```\n\nUser queries are sanitized by `input_sanitizer.sanitize_user_query()` before reaching the LLM (injection protection). Schema validation also occurs at the HTTP boundary (FastAPI) as independent defence-in-depth.\n\n### `POST /chat` and `POST /chat/stream`\n\nLong-form coaching conversation over a full message history. `/chat/stream` returns Server-Sent Events:\n\n```\ndata: {\"type\": \"chunk\", \"text\": \"word \"}\n...\ndata: {\"type\": \"done\", \"engine_signal\": {...}, \"mode\": \"CHAT_V1\"}\n```\n\nChat history is capped at 50 turns. The pipeline always grounds responses in the engine evaluation — no free-form reasoning occurs.\n\n### Security Response Headers\n\nAll responses include:\n- `Strict-Transport-Security: max-age=31536000; includeSubDomains`\n- `X-Content-Type-Options: nosniff`\n- `X-Frame-Options: DENY`\n- `Referrer-Policy: strict-origin-when-cross-origin`\n\nCORS is configured via `CORS_ALLOWED_ORIGINS`. If unset, all cross-origin requests are blocked.\n\n## Stockfish Engine Pool\n\nThe pool is initialized at startup via `StockfishEnginePool`. Key parameters:\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `ENGINE_POOL_SIZE` | 8 | Concurrent Stockfish processes |\n| `ENGINE_THREADS` | 1 | Threads per process |\n| `ENGINE_HASH_MB` | 128 | Hash table per process (MB) |\n| `ENGINE_SKILL_LEVEL` | 10 | UCI skill level |\n| `ENGINE_DEFAULT_MOVETIME_MS` | 40 | Default search time |\n| `ENGINE_ANALYSIS_MOVETIME_MS` | 80 | Analysis mode search time |\n| `ENGINE_BLITZ_MOVETIME_MS` | 25 | Blitz mode search time |\n\nMove cache TTL is controlled by `MOVE_CACHE_TTL_SECONDS` (default 3600 s). Optional Redis backing via `REDIS_URL`.\n\nCache pre-warming runs at startup against configurable FEN positions (`ENGINE_PREWARM_FENS`) and modes (`ENGINE_PREWARM_MODES`). Default pre-warm set includes starting position and common early opening positions.\n\n## Adaptation Layer\n\n`compute_adaptation(player.rating, player.confidence)` returns per-request opponent Elo and teaching style parameters. The opponent Elo is derived from authenticated player data — not a static configuration value. Teaching style (explanation tone) adjusts based on the same inputs.\n\nThis is a **heuristic, subject to revision** — labelled here so it is not mistaken for a learned policy. The current formula (in `llm/seca/adaptation/{skill_profile,teaching_policy,opponent_policy}.py`):\n\n```\nr = clamp((rating − 400) / 2000, 0, 1)               # rating ∈ [400, 2400] → r ∈ [0, 1]\nexplanation_depth   = r\nconcept_complexity  = r ** 1.2\nopponent_strength   = r\nopponent_human_error = 1 − r\n\ntarget_elo        = int(600 + opponent_strength * 1800)        # ∈ [600, 2400]\nhuman_error_rate  = opponent_human_error * 0.25                # ∈ [0, 0.25]\nteaching_style    = simple        if explanation_depth < 0.3   # rating < 1000\n                  | intermediate  if explanation_depth < 0.7   # rating < 1800\n                  | advanced      otherwise\n```\n\nNote that `confidence` is in the function signature but is **not currently consumed** by the formula — it is a forward-compatible hook for a future variance-aware adaptation. Any change to incorporate it (or to retune the constants 400/2000/1800/0.25/0.3/0.7) is an explicit policy change, must be motivated in the commit, and requires updating `test_adaptive_engine_wiring.py`'s pinned ELO range.\n\n## Telemetry\n\nExplanation quality scores are recorded to `telemetry/quality_scores.jsonl` (append-only). Each record contains:\n\n```json\n{\"timestamp\": \"...\", \"score\": 8, \"case_type\": \"...\", \"model\": \"...\", \"mode\": \"...\", \"attempt\": 1}\n```\n\nNo prompt text, no FEN, no user data is stored. The file is never read back at runtime — telemetry is operational instrumentation only.\n\nHealthy quality distribution: majority 8–9, occasional 7, rare ≤ 6. Sustained downward drift or score clustering near threshold indicates model instability.\n\n## Repository Structure\n\n```\n├── android/                  # Android application (UI, game orchestration)\n├── engine/                   # C++ opponent engine (~1800 Elo, JNI bridge)\n├── llm/\n│   ├── server.py             # FastAPI application entry point\n│   ├── explain_pipeline.py   # Mode-2 pipeline with bounded retries\n│   ├── engine_pool.py        # Stockfish pool management\n│   ├── rag/\n│   │   ├── engine_signal/    # ESV extraction\n│   │   ├── retriever/        # Deterministic document retrieval\n│   │   ├── prompts/          # Mode-2 prompt templates (golden-tested)\n│   │   ├── validators/       # Output contracts\n│   │   └── tests/            # Golden, contract, regression, quality tests\n│   ├── seca/\n│   │   ├── safety/           # freeze.py — SECA enforcement at startup\n│   │   ├── auth/             # JWT authentication\n│   │   ├── curriculum/       # Skill-based curriculum scheduling\n│   │   ├── adaptation/       # Opponent Elo and teaching style adaptation\n│   │   ├── coach/            # Chat and live-move coaching pipelines\n│   │   ├── storage/          # PostgreSQL-backed event store and repo\n│   │   └── engines/          # Engine adapters\n│   └── tests/                # API contract and pipeline regression tests\n└── docs/\n    ├── ARCHITECTURE.md       # Formal system specification\n    ├── TESTING.md            # Test strategy and validator rules\n    ├── OPERATIONS.md         # Production operation guide\n    ├── OPERATIONS_RETRIES.md # Bounded retry policy\n    └── RELEASE.md            # Mandatory release procedure\n```\n\n## Testing\n\nThe project uses five test categories. All layers are covered — no layer is unprotected.\n\n| Category | Scope | CI | Command |\n|----------|-------|----|---------|\n| A — Golden tests | ESV mapping, RAG retrieval, prompt snapshots | Yes | `pytest llm/rag/tests/golden/` |\n| B — Contract tests | Forbidden patterns, mate handling, missing data (Fake LLM) | Yes | `pytest llm/rag/tests/contracts/` |\n| C — Smoke test | Real LLM connectivity, output passes validators | No (local only) | `pytest llm/rag/tests/llm/test_ollama_smoke.py` |\n| D — Regression tests | Repeated real LLM runs, contract compliance over time | No (on change events) | `pytest llm/rag/tests/llm/test_llm_regression.py` |\n| E — Quality heuristics | Length, sentence structure, non-triviality | No (advisory) | `pytest llm/rag/tests/quality/` |\n| F — Validator mutation | mutmut against `llm/rag/validators/` — does the test fail when the validator is logically wrong? | No (local, on-demand) | `bash scripts/run_mutation_tests.sh` |\n\nThe **Fake LLM** is a mandatory test component that simulates contract violations to prove validator enforcement. It is not optional.\n\n### Code-quality gates (precise, not vague)\n\n\"Mypy passes\" and \"Pylint passes\" are commitments only as strict as the\nconfig behind them. The actual scope and rule set in this repo:\n\n- **Black** — formatter. Target: `py313`, line length `100`. Scope: 22-file\n  whitelist defined in `llm/run_quality_gate.py:FORMAT_TARGETS` (the\n  stable Python surface — dormant SECA research subtrees are deliberately\n  excluded; see `pyproject.toml [tool.black] extend-exclude` for the list\n  and rationale).\n- **Pylint** — `pylint --score=n` on `PYLINT_TARGETS` (20 files). Default\n  rule set MINUS the permissive softeners enumerated in `pyproject.toml`\n  `[tool.pylint.\"messages control\"] disable`: `broad-exception-caught`,\n  `line-too-long` (covered by Black), `missing-class-docstring`,\n  `missing-function-docstring`, `missing-module-docstring`,\n  `subprocess-run-check`, `too-few-public-methods`, `too-many-arguments`,\n  `too-many-instance-attributes`, `too-many-locals`. Every other Pylint\n  category is enforced.\n- **Mypy** — `mypy` on `MYPY_TARGETS` (16 files). Config from\n  `pyproject.toml [tool.mypy]`: `python_version = \"3.13\"`,\n  `ignore_missing_imports = true` (external libraries without type stubs\n  are silenced — internal modules are NOT silenced),\n  `warn_unused_configs = true`. **Not** `--strict`: the typed utility\n  surface annotates aggressively but does not require every function in\n  the codebase to be annotated. Mypy errors on annotated functions are\n  fail-on, untyped function bodies are not deep-checked\n  (`--check-untyped-defs` is not enabled).\n- **Coverage** — global ≥ 80% via `pytest --cov-fail-under=80`, AND\n  per-module floors enforced by `llm/check_coverage_thresholds.py` after\n  the pytest run: validators (`llm/rag/validators/*.py`) and the\n  post-LLM safety firewall (`llm/rag/safety/*.py`) MUST stay ≥ 95%.\n  Three legacy modules (`contextual_bandit`, `curriculum/spacing`,\n  `skills/trainer`) carry documented exemptions at their current\n  coverage; each is a TODO to lift.\n- **`pip-audit`** — strict mode against `llm/requirements.txt` AND\n  `llm/requirements-ci.txt` (closed CI/runtime split documented in the\n  workflow); any unfixed CVE blocks the merge.\n- **Trivy** — image scan on the published GHCR images. CRITICAL\n  unfixed → block; HIGH/CRITICAL → SARIF upload to the Security tab.\n\nTo extend any of these (e.g. enable `mypy --strict` on a specific\npackage, or drop a Pylint disable), update the relevant config in\n`pyproject.toml` and the target list in `llm/run_quality_gate.py` —\nboth surfaces must agree, and the change is a contract update worth a\ndeliberate commit message.\n\n### Running Tests\n\n```bash\npython llm/run_ci_suite.py                                         # full CI suite\npython -m pytest -q llm/rag/tests/golden/test_retriever.py\npython -m pytest -q llm/rag/tests/golden/test_prompt_snapshot.py\npython -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\npython -m pytest -q llm/tests/test_api_contract_validation.py\npython -m pytest -q llm/tests/test_coaching_pipeline_regression.py\npython llm/run_quality_gate.py black                               # formatting\npython llm/run_quality_gate.py pylint                              # linting\npython llm/run_quality_gate.py mypy                                # types\n```\n\nLLM regression tests (Category D) must be run before every release, after any system prompt or RAG document change, and after any model update. They must never run in CI.\n\n## Developer Setup\n\n### Quick Start (Docker)\n\n```bash\ncp .env.example .env      # edit values if needed\ndocker compose up\n```\n\nAPI at `http://localhost:8000`. Requires [Ollama](https://ollama.ai) on the host:\n\n```bash\nollama pull qwen2.5:7b-instruct-q2_K\nollama serve\n```\n\n`host.docker.internal` is mapped automatically on macOS, Windows, and Linux via `extra_hosts` in `docker-compose.yml`.\n\n### Dev Container (VS Code)\n\nOpen the repo and select **\"Reopen in Container\"**. Installs Python 3.13, Node.js 22, Stockfish, and all Python dependencies automatically. Ollama must run on the host.\n\n### Bare-Metal Python\n\n```bash\nsudo apt install stockfish   # or: brew install stockfish\npip install -r llm/requirements.txt\ncp .env.example .env\npython -m uvicorn llm.server:app --host 0.0.0.0 --port 8000\n```\n\nStockfish is auto-detected via `PATH`, then falls back to `/usr/games/stockfish` (Linux) or `engines/stockfish.exe` (Windows). Override with `STOCKFISH_PATH` in `.env`.\n\n### Android Setup\n\n`android/local.properties` is gitignored and machine-specific:\n\n```bash\n# macOS / Linux / WSL\n./scripts/setup-android.sh\n\n# Windows (PowerShell)\n\"sdk.dir=$($env:LOCALAPPDATA -replace '\\\\','/')/Android/Sdk\" > android\\local.properties\n```\n\nOr open `android/` in Android Studio — it generates the file automatically.\n\nBuilds target `arm64-v8a` (physical devices, Apple Silicon emulators) and `x86_64` (Intel/AMD AVDs).\n\n### Environment Variables\n\n| Variable | Default | Description |\n|----------|---------|-------------|\n| `SECA_API_KEY` | `dev-key` | Auth key (any value works in `dev` mode; required in `prod`) |\n| `SECA_ENV` | `dev` | `dev` or `prod` |\n| `SECRET_KEY` | — | JWT signing secret (≥ 32 chars; required in `prod`) |\n| `COACH_OLLAMA_URL` | `http://host.docker.internal:11434` | Ollama endpoint |\n| `COACH_OLLAMA_MODEL` | `qwen2.5:7b-instruct-q2_K` | LLM model. Tags are fine for dev (Ollama resolves whatever digest is current), but **production should pin the digest** with `model:tag@sha256:...` so a re-pushed tag upstream cannot silently change behaviour under the LLM regression contract. Find the digest with `ollama show <tag> --modelfile \\| grep '^FROM'`; full recipe in `.env.prod.example`. |\n| `STOCKFISH_PATH` | auto-detected | Override Stockfish binary path |\n| `REDIS_URL` | *(unset)* | Redis for move cache L2; omit for in-memory only |\n| `ENGINE_CACHE_VERSION` | *(unset)* | Optional move-cache flush override. The L1+L2 cache key already incorporates a fingerprint of the engine config (`ENGINE_SKILL_LEVEL`, `ENGINE_THREADS`, `ENGINE_HASH_MB`, all movetime defaults, the Stockfish binary path), so changing any of those automatically invalidates stale entries. Set this to any string (e.g. `post-upgrade-2026-05`) to force a flush without changing those fields — useful when you replace the Stockfish binary in place at the same path. |\n| `SECA_INSECURE_DEV` | *(unset → `false`)* | Local-development opt-in for the no-`SECA_API_KEY` auth bypass. With `SECA_ENV=dev` and no key configured, requests to protected endpoints are rejected with HTTP 401 by default; set this to `true` to pass them through. **Never set in production** — see [docs/THREAT_MODEL.md](docs/THREAT_MODEL.md) § T6. |\n| `DATABASE_URL` | — | PostgreSQL DSN (required in `prod`) |\n| `CORS_ALLOWED_ORIGINS` | *(unset — blocks all cross-origin)* | Comma-separated allowed origins |\n| `ENGINE_POOL_SIZE` | `8` | Concurrent Stockfish processes |\n| `ENGINE_ASYNC_PREDICT_ENABLED` | `true` | Enable predictive move pre-caching |\n\nSee `.env.example` for the full reference.\n\n### Common Issues\n\n| Symptom | Fix |\n|---------|-----|\n| `UnsatisfiedLinkError: libchessengine.so` | Expected on host JVM — `NativeEngineProvider` returns `null` gracefully. Run on device/emulator via `connectedAndroidTest`. |\n| `FileNotFoundError: stockfish` | Install Stockfish or set `STOCKFISH_PATH` in `.env`. |\n| `host.docker.internal` unreachable on Linux | `docker-compose.yml` adds `host-gateway` automatically. For bare `docker run`, add `--add-host=host.docker.internal:host-gateway`. |\n| `sdk.dir` Gradle error | Run `./scripts/setup-android.sh` or open `android/` in Android Studio. |\n| Server refuses to start in prod | `SECA_API_KEY` and `SECRET_KEY` must be set when `SECA_ENV=prod`. |\n\n## CI/CD\n\nGitHub Actions (`fly-deploy.yml`) runs on pull requests and pushes to `main`, and on `v*.*.*` tag push.\n\nJobs:\n1. **actionlint** — workflow YAML validation\n2. **python-tests** — golden tests, contract tests (incl. validator violations corpus), API contract validation, pipeline regression, explain schema validation, full CI suite, Black/Pylint/Mypy gates (see *Code-quality gates* above for exact scope and strictness), per-module coverage floors (validators ≥ 95%, rest ≥ 80%), pip-audit, Trivy\n3. **android-build** — Gradle build and host JVM test suite\n4. **docker-build** — builds `cereveon` (Fly.io edge / Node) and `cereveon-llm-api` (Hetzner backend / Python) images\n5. **deploy** — triggered on `v*.*.*` tag; publishes to GHCR and deploys to production\n\nCI never runs real LLM inference, never requires Ollama, and never depends on telemetry.\n\n## Release Process\n\nReleases follow `vMAJOR.MINOR.PATCH` (monotonically increasing):\n- `MAJOR` — architectural or contract changes\n- `MINOR` — new features, RAG documents, golden cases\n- `PATCH` — bug fixes, wording improvements, no behavior change\n\nPre-release checklist (non-negotiable):\n1. Clean working tree (`git status`)\n2. All CI-safe tests pass (golden, contract, API contract, pipeline regression)\n3. LLM regression tests pass (`test_llm_regression.py`)\n4. Real LLM smoke test passes (`test_ollama_smoke.py`)\n5. Manual output sanity review (no engine mentions, no move suggestions, correct tone)\n\nPushing a `vX.Y.Z` tag publishes the GitHub Release and GHCR images for both `cereveon:vX.Y.Z` and `cereveon-llm-api:vX.Y.Z`.\n\n## Architecture Constraints\n\nThe following changes are explicitly **forbidden**:\n- Weakening output validators\n- Bypassing or replacing the ESV\n- Dynamic prompt mutation at runtime\n- LLM reasoning beyond provided inputs\n- Autonomous RL implementation\n- Disabling or skipping SECA enforcement\n\n**Allowed** without architectural review:\n- Adding RAG documents\n- Adding golden test cases\n- Improving explanation wording (within existing contracts)\n- Adding new `BaseLLM` adapters\n\n## Design Philosophy\n\nCereveon prioritizes:\n\n1. **Correctness** — Invariants enforced via code and test contracts, not convention\n2. **Determinism** — All layers except LLM generation are fully reproducible\n3. **Non-hallucination** — ESV normalization and output validators guarantee the LLM cannot invent engine facts\n4. **Safety** — Strict contracts on all LLM outputs; no output is always better than unsafe output\n5. **Maintainability** — Loose coupling with explicit trust boundaries at every layer\n\nOver convenience, feature velocity, or explanation quality at the expense of correctness.\n\n## Further Reading\n\n- `docs/ARCHITECTURE.md` — Formal system specification, trust boundaries, data flow\n- `docs/TESTING.md` — Test strategy, validator rules, all five test categories\n- `docs/THREAT_MODEL.md` — Adversaries, threats, mitigations, accepted residual risks\n- `docs/OPERATIONS.md` — Production operation, failure modes, incident response\n- `docs/OPERATIONS_RETRIES.md` — Bounded retry policy and telemetry interpretation\n- `docs/RELEASE.md` — Mandatory release procedure and invariants\n- `docs/LICENSE.md` — Rights and attribution\n"}}, ".github/workflows/fly-deploy.yml": {"ec1475e17980d641a85def6a3ff6a74418102c40": {"t": 1785747045.7104852, "n": 34398, "h": "1ef4eb5e430fb91d", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Deploys the single production tier built from this monorepo:\n#\n#   Hetzner heavy backend — `cereveon-llm-api` image (Python FastAPI +\n#   Stockfish + full SECA pipeline, port 8000), served at cereveon.com\n#   behind Caddy.  Updated by the `deploy` job below via SSH + docker\n#   compose.\n#\n# The former Fly.io edge (a Node/Express `cereveon` image running the\n# vestigial llm/server.js Ollama prototype) was removed 2026-07-15: it\n# never fronted cereveon.com (that has always resolved to Hetzner's\n# Caddy directly) and its only endpoints were an unauthenticated,\n# validator-bypassing /coach·/explain that 502'd on a non-existent\n# Ollama upstream.  See docs/DEPLOYMENT.md.\n#\n# The workflow's filename is preserved for URL stability across external\n# bookmarks; the workflow's `name:` field (CI/CD) is the authoritative\n# description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        # The flat ``llm/engine_eval.py`` / ``llm/elite_engine_service.py``\n        # modules were retired alongside host_app (PR #111) in the\n        # engine-library cleanup; live coverage moved to\n        # ``llm.seca.engines.stockfish.pool`` and is exercised by\n        # ``test_engine_pool_*.py`` plus the existing response-format\n        # / adapter-isolation regressions.\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_pool_evaluate_position.py \\\n            llm/tests/test_engine_pool_exhaustion.py \\\n            llm/tests/test_engine_pool_crash_recovery.py \\\n            llm/tests/test_fen_move_cache_key.py \\\n            llm/tests/test_stockfish_adapter_isolation.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v7\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        # PYSEC-2025-183 / CVE-2025-45768 is a DISPUTED advisory against\n        # pyjwt: it flags \"weak encryption\", but the upstream maintainer\n        # rejects the report because key length is application-controlled\n        # — see the NOTE field at\n        # https://api.osv.dev/v1/vulns/PYSEC-2025-183.  Our app already\n        # enforces SECRET_KEY length >= 32 chars in prod (validated at\n        # module load in llm/seca/auth/tokens.py; documented in\n        # .env.example and docs/THREAT_MODEL.md §T2), which is the\n        # strong-key property the advisory complains about.  No upstream\n        # fix version exists and one is unlikely to ship.  Revisit if\n        # pyjwt ever releases a version that explicitly addresses the\n        # advisory, or if our SECRET_KEY guarantees change.\n        #\n        # NOTE: PYSEC-2026-161 / CVE-2026-48710 (starlette \"BadHost\":\n        # missing Host-header validation poisons ``request.url.path`` and\n        # can bypass path-based security checks) was previously suppressed\n        # here while fastapi 0.128.1 capped ``starlette<0.51.0``.  That cap\n        # is gone — fastapi 0.136.3 requires only ``starlette>=0.46.0`` —\n        # so the runtime now ships starlette 1.2.1 (past the fixed 1.0.1)\n        # and the advisory is closed at the dependency level.  The\n        # ``--ignore-vuln`` was therefore removed; if a vulnerable\n        # starlette is ever re-pinned, this audit step fails (and so does\n        # llm/tests/test_security_host_header_poisoning.py).\n        run: python -m pip_audit -r llm/requirements.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        # Only the disputed pyjwt advisory remains suppressed here:\n        #   - PYSEC-2025-183 disputed pyjwt advisory + SECRET_KEY\n        #     mitigation apply identically (pyjwt is also pulled in\n        #     transitively by the CI dependency tree, e.g. test\n        #     fixtures that mint tokens against the auth router).\n        # PYSEC-2026-161 (starlette BadHost) is no longer suppressed:\n        # requirements-ci.txt installs the same starlette 1.2.1 as the\n        # runtime via ``-r requirements.txt``, which is past the 1.0.1 fix.\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v5\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v7\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v4\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v6\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v7\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v4\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published image\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile,site\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              # Apply synced Caddyfile + landing-page (site/) changes.  The SCP\n              # step above updates them on disk, but the running caddy container\n              # only picks up a new volume mount / Caddyfile on recreate.\n              # ``up -d caddy`` recreates caddy when its compose config changed\n              # (adding the ./site:/srv mount does) and is a no-op otherwise, so\n              # it will not blip ingress on ordinary api-only deploys.\n              #\n              # ``--no-deps`` is REQUIRED: caddy ``depends_on`` api, so without\n              # it ``compose up caddy`` also reconciles api and tries to pull the\n              # private GHCR api image -- which fails with \"error from registry:\n              # denied\" and silently aborts the caddy recreate (landing page then\n              # never goes live).  --no-deps limits the action to caddy, whose\n              # public image is already present locally.\n              docker compose -f docker-compose.prod.yml up -d --no-deps caddy || true\n              # A Caddyfile *content* change (e.g. a new route) does not alter\n              # caddy's compose config, so the ``up -d`` above is a no-op for it.\n              # Gracefully reload the freshly-synced Caddyfile so routing changes\n              # take effect with zero downtime (caddy keeps the running config if\n              # the new one fails to validate).\n              docker compose -f docker-compose.prod.yml exec -T caddy caddy reload --config /etc/caddy/Caddyfile --adapter caddyfile || true\n              # Prune stale unused images.  Without this, every merge to main\n              # leaves prior cereveon-llm-api digests on disk; on the 38 GB\n              # Hetzner box the / volume fills after ~30-50 merges and the\n              # next deploy's SCP step fails with the misleading\n              # \"error copy file to dest: ***\" signature (4 consecutive\n              # deploys stuck on 2026-05-15 before `docker image prune -af`\n              # unblocked it).  See feedback memory \"Hetzner production\n              # deploy gotchas\" #4 for the full incident.\n              #\n              # --filter \"until=72h\" keeps the previous deploy's image so a\n              # manual rollback skips the GHCR round-trip; -af removes\n              # everything else not currently in use (the running api's\n              # image is referenced and therefore safe).  Errors do not\n              # fail the deploy — `|| true` keeps an otherwise-healthy\n              # rollout from being marked failed by a transient docker\n              # hiccup during prune.\n              docker image prune -af --filter \"until=72h\" || true\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container image published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "33d9f0c2b645ab93b86793357bb33bc6a9a612e4": {"t": 1785749409.4009337, "n": 34398, "h": "1ef4eb5e430fb91d", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Deploys the single production tier built from this monorepo:\n#\n#   Hetzner heavy backend — `cereveon-llm-api` image (Python FastAPI +\n#   Stockfish + full SECA pipeline, port 8000), served at cereveon.com\n#   behind Caddy.  Updated by the `deploy` job below via SSH + docker\n#   compose.\n#\n# The former Fly.io edge (a Node/Express `cereveon` image running the\n# vestigial llm/server.js Ollama prototype) was removed 2026-07-15: it\n# never fronted cereveon.com (that has always resolved to Hetzner's\n# Caddy directly) and its only endpoints were an unauthenticated,\n# validator-bypassing /coach·/explain that 502'd on a non-existent\n# Ollama upstream.  See docs/DEPLOYMENT.md.\n#\n# The workflow's filename is preserved for URL stability across external\n# bookmarks; the workflow's `name:` field (CI/CD) is the authoritative\n# description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        # The flat ``llm/engine_eval.py`` / ``llm/elite_engine_service.py``\n        # modules were retired alongside host_app (PR #111) in the\n        # engine-library cleanup; live coverage moved to\n        # ``llm.seca.engines.stockfish.pool`` and is exercised by\n        # ``test_engine_pool_*.py`` plus the existing response-format\n        # / adapter-isolation regressions.\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_pool_evaluate_position.py \\\n            llm/tests/test_engine_pool_exhaustion.py \\\n            llm/tests/test_engine_pool_crash_recovery.py \\\n            llm/tests/test_fen_move_cache_key.py \\\n            llm/tests/test_stockfish_adapter_isolation.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v7\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        # PYSEC-2025-183 / CVE-2025-45768 is a DISPUTED advisory against\n        # pyjwt: it flags \"weak encryption\", but the upstream maintainer\n        # rejects the report because key length is application-controlled\n        # — see the NOTE field at\n        # https://api.osv.dev/v1/vulns/PYSEC-2025-183.  Our app already\n        # enforces SECRET_KEY length >= 32 chars in prod (validated at\n        # module load in llm/seca/auth/tokens.py; documented in\n        # .env.example and docs/THREAT_MODEL.md §T2), which is the\n        # strong-key property the advisory complains about.  No upstream\n        # fix version exists and one is unlikely to ship.  Revisit if\n        # pyjwt ever releases a version that explicitly addresses the\n        # advisory, or if our SECRET_KEY guarantees change.\n        #\n        # NOTE: PYSEC-2026-161 / CVE-2026-48710 (starlette \"BadHost\":\n        # missing Host-header validation poisons ``request.url.path`` and\n        # can bypass path-based security checks) was previously suppressed\n        # here while fastapi 0.128.1 capped ``starlette<0.51.0``.  That cap\n        # is gone — fastapi 0.136.3 requires only ``starlette>=0.46.0`` —\n        # so the runtime now ships starlette 1.2.1 (past the fixed 1.0.1)\n        # and the advisory is closed at the dependency level.  The\n        # ``--ignore-vuln`` was therefore removed; if a vulnerable\n        # starlette is ever re-pinned, this audit step fails (and so does\n        # llm/tests/test_security_host_header_poisoning.py).\n        run: python -m pip_audit -r llm/requirements.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        # Only the disputed pyjwt advisory remains suppressed here:\n        #   - PYSEC-2025-183 disputed pyjwt advisory + SECRET_KEY\n        #     mitigation apply identically (pyjwt is also pulled in\n        #     transitively by the CI dependency tree, e.g. test\n        #     fixtures that mint tokens against the auth router).\n        # PYSEC-2026-161 (starlette BadHost) is no longer suppressed:\n        # requirements-ci.txt installs the same starlette 1.2.1 as the\n        # runtime via ``-r requirements.txt``, which is past the 1.0.1 fix.\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v5\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v7\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v4\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v6\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v7\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v4\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published image\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile,site\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              # Apply synced Caddyfile + landing-page (site/) changes.  The SCP\n              # step above updates them on disk, but the running caddy container\n              # only picks up a new volume mount / Caddyfile on recreate.\n              # ``up -d caddy`` recreates caddy when its compose config changed\n              # (adding the ./site:/srv mount does) and is a no-op otherwise, so\n              # it will not blip ingress on ordinary api-only deploys.\n              #\n              # ``--no-deps`` is REQUIRED: caddy ``depends_on`` api, so without\n              # it ``compose up caddy`` also reconciles api and tries to pull the\n              # private GHCR api image -- which fails with \"error from registry:\n              # denied\" and silently aborts the caddy recreate (landing page then\n              # never goes live).  --no-deps limits the action to caddy, whose\n              # public image is already present locally.\n              docker compose -f docker-compose.prod.yml up -d --no-deps caddy || true\n              # A Caddyfile *content* change (e.g. a new route) does not alter\n              # caddy's compose config, so the ``up -d`` above is a no-op for it.\n              # Gracefully reload the freshly-synced Caddyfile so routing changes\n              # take effect with zero downtime (caddy keeps the running config if\n              # the new one fails to validate).\n              docker compose -f docker-compose.prod.yml exec -T caddy caddy reload --config /etc/caddy/Caddyfile --adapter caddyfile || true\n              # Prune stale unused images.  Without this, every merge to main\n              # leaves prior cereveon-llm-api digests on disk; on the 38 GB\n              # Hetzner box the / volume fills after ~30-50 merges and the\n              # next deploy's SCP step fails with the misleading\n              # \"error copy file to dest: ***\" signature (4 consecutive\n              # deploys stuck on 2026-05-15 before `docker image prune -af`\n              # unblocked it).  See feedback memory \"Hetzner production\n              # deploy gotchas\" #4 for the full incident.\n              #\n              # --filter \"until=72h\" keeps the previous deploy's image so a\n              # manual rollback skips the GHCR round-trip; -af removes\n              # everything else not currently in use (the running api's\n              # image is referenced and therefore safe).  Errors do not\n              # fail the deploy — `|| true` keeps an otherwise-healthy\n              # rollout from being marked failed by a transient docker\n              # hiccup during prune.\n              docker image prune -af --filter \"until=72h\" || true\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container image published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "a8f617be709c0c14f94df651ff3a4389623cac5e": {"t": 1785749410.7994523, "n": 33884, "h": "344cf65303812193", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Deploys the single production tier built from this monorepo:\n#\n#   Hetzner heavy backend — `cereveon-llm-api` image (Python FastAPI +\n#   Stockfish + full SECA pipeline, port 8000), served at cereveon.com\n#   behind Caddy.  Updated by the `deploy` job below via SSH + docker\n#   compose.\n#\n# The former Fly.io edge (a Node/Express `cereveon` image running the\n# vestigial llm/server.js Ollama prototype) was removed 2026-07-15: it\n# never fronted cereveon.com (that has always resolved to Hetzner's\n# Caddy directly) and its only endpoints were an unauthenticated,\n# validator-bypassing /coach·/explain that 502'd on a non-existent\n# Ollama upstream.  See docs/DEPLOYMENT.md.\n#\n# The workflow's filename is preserved for URL stability across external\n# bookmarks; the workflow's `name:` field (CI/CD) is the authoritative\n# description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        # The flat ``llm/engine_eval.py`` / ``llm/elite_engine_service.py``\n        # modules were retired alongside host_app (PR #111) in the\n        # engine-library cleanup; live coverage moved to\n        # ``llm.seca.engines.stockfish.pool`` and is exercised by\n        # ``test_engine_pool_*.py`` plus the existing response-format\n        # / adapter-isolation regressions.\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_pool_evaluate_position.py \\\n            llm/tests/test_engine_pool_exhaustion.py \\\n            llm/tests/test_engine_pool_crash_recovery.py \\\n            llm/tests/test_fen_move_cache_key.py \\\n            llm/tests/test_stockfish_adapter_isolation.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v7\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        # PYSEC-2025-183 / CVE-2025-45768 is a DISPUTED advisory against\n        # pyjwt: it flags \"weak encryption\", but the upstream maintainer\n        # rejects the report because key length is application-controlled\n        # — see the NOTE field at\n        # https://api.osv.dev/v1/vulns/PYSEC-2025-183.  Our app already\n        # enforces SECRET_KEY length >= 32 chars in prod (validated at\n        # module load in llm/seca/auth/tokens.py; documented in\n        # .env.example and docs/THREAT_MODEL.md §T2), which is the\n        # strong-key property the advisory complains about.  No upstream\n        # fix version exists and one is unlikely to ship.  Revisit if\n        # pyjwt ever releases a version that explicitly addresses the\n        # advisory, or if our SECRET_KEY guarantees change.\n        #\n        # NOTE: PYSEC-2026-161 / CVE-2026-48710 (starlette \"BadHost\":\n        # missing Host-header validation poisons ``request.url.path`` and\n        # can bypass path-based security checks) was previously suppressed\n        # here while fastapi 0.128.1 capped ``starlette<0.51.0``.  That cap\n        # is gone — fastapi 0.136.3 requires only ``starlette>=0.46.0`` —\n        # so the runtime now ships starlette 1.2.1 (past the fixed 1.0.1)\n        # and the advisory is closed at the dependency level.  The\n        # ``--ignore-vuln`` was therefore removed; if a vulnerable\n        # starlette is ever re-pinned, this audit step fails (and so does\n        # llm/tests/test_security_host_header_poisoning.py).\n        run: python -m pip_audit -r llm/requirements.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        # Only the disputed pyjwt advisory remains suppressed here:\n        #   - PYSEC-2025-183 disputed pyjwt advisory + SECRET_KEY\n        #     mitigation apply identically (pyjwt is also pulled in\n        #     transitively by the CI dependency tree, e.g. test\n        #     fixtures that mint tokens against the auth router).\n        # PYSEC-2026-161 (starlette BadHost) is no longer suppressed:\n        # requirements-ci.txt installs the same starlette 1.2.1 as the\n        # runtime via ``-r requirements.txt``, which is past the 1.0.1 fix.\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v5\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v7\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v4\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v6\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v7\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v4\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published image\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile,site\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              # Apply synced Caddyfile + landing-page (site/) changes.  The SCP\n              # step above updates them on disk, but the running caddy container\n              # only picks up a new volume mount / Caddyfile on recreate.\n              # ``up -d caddy`` recreates caddy when its compose config changed\n              # (adding the ./site:/srv mount does) and is a no-op otherwise, so\n              # it will not blip ingress on ordinary api-only deploys.\n              #\n              # ``--no-deps`` is REQUIRED: caddy ``depends_on`` api, so without\n              # it ``compose up caddy`` also reconciles api and tries to pull the\n              # private GHCR api image -- which fails with \"error from registry:\n              # denied\" and silently aborts the caddy recreate (landing page then\n              # never goes live).  --no-deps limits the action to caddy, whose\n              # public image is already present locally.\n              docker compose -f docker-compose.prod.yml up -d --no-deps caddy || true\n              # Prune stale unused images.  Without this, every merge to main\n              # leaves prior cereveon-llm-api digests on disk; on the 38 GB\n              # Hetzner box the / volume fills after ~30-50 merges and the\n              # next deploy's SCP step fails with the misleading\n              # \"error copy file to dest: ***\" signature (4 consecutive\n              # deploys stuck on 2026-05-15 before `docker image prune -af`\n              # unblocked it).  See feedback memory \"Hetzner production\n              # deploy gotchas\" #4 for the full incident.\n              #\n              # --filter \"until=72h\" keeps the previous deploy's image so a\n              # manual rollback skips the GHCR round-trip; -af removes\n              # everything else not currently in use (the running api's\n              # image is referenced and therefore safe).  Errors do not\n              # fail the deploy — `|| true` keeps an otherwise-healthy\n              # rollout from being marked failed by a transient docker\n              # hiccup during prune.\n              docker image prune -af --filter \"until=72h\" || true\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container image published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "0845b98bff58ef24db0a7de78b3b627b5ae5fb52": {"t": 1785749412.0593364, "n": 33397, "h": "a6aaad58c2af64e8", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Deploys the single production tier built from this monorepo:\n#\n#   Hetzner heavy backend — `cereveon-llm-api` image (Python FastAPI +\n#   Stockfish + full SECA pipeline, port 8000), served at cereveon.com\n#   behind Caddy.  Updated by the `deploy` job below via SSH + docker\n#   compose.\n#\n# The former Fly.io edge (a Node/Express `cereveon` image running the\n# vestigial llm/server.js Ollama prototype) was removed 2026-07-15: it\n# never fronted cereveon.com (that has always resolved to Hetzner's\n# Caddy directly) and its only endpoints were an unauthenticated,\n# validator-bypassing /coach·/explain that 502'd on a non-existent\n# Ollama upstream.  See docs/DEPLOYMENT.md.\n#\n# The workflow's filename is preserved for URL stability across external\n# bookmarks; the workflow's `name:` field (CI/CD) is the authoritative\n# description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        # The flat ``llm/engine_eval.py`` / ``llm/elite_engine_service.py``\n        # modules were retired alongside host_app (PR #111) in the\n        # engine-library cleanup; live coverage moved to\n        # ``llm.seca.engines.stockfish.pool`` and is exercised by\n        # ``test_engine_pool_*.py`` plus the existing response-format\n        # / adapter-isolation regressions.\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_pool_evaluate_position.py \\\n            llm/tests/test_engine_pool_exhaustion.py \\\n            llm/tests/test_engine_pool_crash_recovery.py \\\n            llm/tests/test_fen_move_cache_key.py \\\n            llm/tests/test_stockfish_adapter_isolation.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v7\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        # PYSEC-2025-183 / CVE-2025-45768 is a DISPUTED advisory against\n        # pyjwt: it flags \"weak encryption\", but the upstream maintainer\n        # rejects the report because key length is application-controlled\n        # — see the NOTE field at\n        # https://api.osv.dev/v1/vulns/PYSEC-2025-183.  Our app already\n        # enforces SECRET_KEY length >= 32 chars in prod (validated at\n        # module load in llm/seca/auth/tokens.py; documented in\n        # .env.example and docs/THREAT_MODEL.md §T2), which is the\n        # strong-key property the advisory complains about.  No upstream\n        # fix version exists and one is unlikely to ship.  Revisit if\n        # pyjwt ever releases a version that explicitly addresses the\n        # advisory, or if our SECRET_KEY guarantees change.\n        #\n        # NOTE: PYSEC-2026-161 / CVE-2026-48710 (starlette \"BadHost\":\n        # missing Host-header validation poisons ``request.url.path`` and\n        # can bypass path-based security checks) was previously suppressed\n        # here while fastapi 0.128.1 capped ``starlette<0.51.0``.  That cap\n        # is gone — fastapi 0.136.3 requires only ``starlette>=0.46.0`` —\n        # so the runtime now ships starlette 1.2.1 (past the fixed 1.0.1)\n        # and the advisory is closed at the dependency level.  The\n        # ``--ignore-vuln`` was therefore removed; if a vulnerable\n        # starlette is ever re-pinned, this audit step fails (and so does\n        # llm/tests/test_security_host_header_poisoning.py).\n        run: python -m pip_audit -r llm/requirements.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        # Only the disputed pyjwt advisory remains suppressed here:\n        #   - PYSEC-2025-183 disputed pyjwt advisory + SECRET_KEY\n        #     mitigation apply identically (pyjwt is also pulled in\n        #     transitively by the CI dependency tree, e.g. test\n        #     fixtures that mint tokens against the auth router).\n        # PYSEC-2026-161 (starlette BadHost) is no longer suppressed:\n        # requirements-ci.txt installs the same starlette 1.2.1 as the\n        # runtime via ``-r requirements.txt``, which is past the 1.0.1 fix.\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v5\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v7\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v4\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v6\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v7\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v4\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published image\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile,site\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              # Apply synced Caddyfile + landing-page (site/) changes.  The SCP\n              # step above updates them on disk, but the running caddy container\n              # only picks up a new volume mount / Caddyfile on recreate.\n              # ``up -d caddy`` recreates caddy when its compose config changed\n              # (adding the ./site:/srv mount does) and is a no-op otherwise, so\n              # it will not blip ingress on ordinary api-only deploys.\n              docker compose -f docker-compose.prod.yml up -d caddy || true\n              # Prune stale unused images.  Without this, every merge to main\n              # leaves prior cereveon-llm-api digests on disk; on the 38 GB\n              # Hetzner box the / volume fills after ~30-50 merges and the\n              # next deploy's SCP step fails with the misleading\n              # \"error copy file to dest: ***\" signature (4 consecutive\n              # deploys stuck on 2026-05-15 before `docker image prune -af`\n              # unblocked it).  See feedback memory \"Hetzner production\n              # deploy gotchas\" #4 for the full incident.\n              #\n              # --filter \"until=72h\" keeps the previous deploy's image so a\n              # manual rollback skips the GHCR round-trip; -af removes\n              # everything else not currently in use (the running api's\n              # image is referenced and therefore safe).  Errors do not\n              # fail the deploy — `|| true` keeps an otherwise-healthy\n              # rollout from being marked failed by a transient docker\n              # hiccup during prune.\n              docker image prune -af --filter \"until=72h\" || true\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container image published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "061f6032afb9ad9a40999c4f1908b655fcdabb21": {"t": 1785749413.3273373, "n": 32849, "h": "7a0c03bdbc45790c", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Deploys the single production tier built from this monorepo:\n#\n#   Hetzner heavy backend — `cereveon-llm-api` image (Python FastAPI +\n#   Stockfish + full SECA pipeline, port 8000), served at cereveon.com\n#   behind Caddy.  Updated by the `deploy` job below via SSH + docker\n#   compose.\n#\n# The former Fly.io edge (a Node/Express `cereveon` image running the\n# vestigial llm/server.js Ollama prototype) was removed 2026-07-15: it\n# never fronted cereveon.com (that has always resolved to Hetzner's\n# Caddy directly) and its only endpoints were an unauthenticated,\n# validator-bypassing /coach·/explain that 502'd on a non-existent\n# Ollama upstream.  See docs/DEPLOYMENT.md.\n#\n# The workflow's filename is preserved for URL stability across external\n# bookmarks; the workflow's `name:` field (CI/CD) is the authoritative\n# description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        # The flat ``llm/engine_eval.py`` / ``llm/elite_engine_service.py``\n        # modules were retired alongside host_app (PR #111) in the\n        # engine-library cleanup; live coverage moved to\n        # ``llm.seca.engines.stockfish.pool`` and is exercised by\n        # ``test_engine_pool_*.py`` plus the existing response-format\n        # / adapter-isolation regressions.\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_pool_evaluate_position.py \\\n            llm/tests/test_engine_pool_exhaustion.py \\\n            llm/tests/test_engine_pool_crash_recovery.py \\\n            llm/tests/test_fen_move_cache_key.py \\\n            llm/tests/test_stockfish_adapter_isolation.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v7\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        # PYSEC-2025-183 / CVE-2025-45768 is a DISPUTED advisory against\n        # pyjwt: it flags \"weak encryption\", but the upstream maintainer\n        # rejects the report because key length is application-controlled\n        # — see the NOTE field at\n        # https://api.osv.dev/v1/vulns/PYSEC-2025-183.  Our app already\n        # enforces SECRET_KEY length >= 32 chars in prod (validated at\n        # module load in llm/seca/auth/tokens.py; documented in\n        # .env.example and docs/THREAT_MODEL.md §T2), which is the\n        # strong-key property the advisory complains about.  No upstream\n        # fix version exists and one is unlikely to ship.  Revisit if\n        # pyjwt ever releases a version that explicitly addresses the\n        # advisory, or if our SECRET_KEY guarantees change.\n        #\n        # NOTE: PYSEC-2026-161 / CVE-2026-48710 (starlette \"BadHost\":\n        # missing Host-header validation poisons ``request.url.path`` and\n        # can bypass path-based security checks) was previously suppressed\n        # here while fastapi 0.128.1 capped ``starlette<0.51.0``.  That cap\n        # is gone — fastapi 0.136.3 requires only ``starlette>=0.46.0`` —\n        # so the runtime now ships starlette 1.2.1 (past the fixed 1.0.1)\n        # and the advisory is closed at the dependency level.  The\n        # ``--ignore-vuln`` was therefore removed; if a vulnerable\n        # starlette is ever re-pinned, this audit step fails (and so does\n        # llm/tests/test_security_host_header_poisoning.py).\n        run: python -m pip_audit -r llm/requirements.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        # Only the disputed pyjwt advisory remains suppressed here:\n        #   - PYSEC-2025-183 disputed pyjwt advisory + SECRET_KEY\n        #     mitigation apply identically (pyjwt is also pulled in\n        #     transitively by the CI dependency tree, e.g. test\n        #     fixtures that mint tokens against the auth router).\n        # PYSEC-2026-161 (starlette BadHost) is no longer suppressed:\n        # requirements-ci.txt installs the same starlette 1.2.1 as the\n        # runtime via ``-r requirements.txt``, which is past the 1.0.1 fix.\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v5\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v7\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v4\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v6\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v7\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v4\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published image\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              # Prune stale unused images.  Without this, every merge to main\n              # leaves prior cereveon-llm-api digests on disk; on the 38 GB\n              # Hetzner box the / volume fills after ~30-50 merges and the\n              # next deploy's SCP step fails with the misleading\n              # \"error copy file to dest: ***\" signature (4 consecutive\n              # deploys stuck on 2026-05-15 before `docker image prune -af`\n              # unblocked it).  See feedback memory \"Hetzner production\n              # deploy gotchas\" #4 for the full incident.\n              #\n              # --filter \"until=72h\" keeps the previous deploy's image so a\n              # manual rollback skips the GHCR round-trip; -af removes\n              # everything else not currently in use (the running api's\n              # image is referenced and therefore safe).  Errors do not\n              # fail the deploy — `|| true` keeps an otherwise-healthy\n              # rollout from being marked failed by a transient docker\n              # hiccup during prune.\n              docker image prune -af --filter \"until=72h\" || true\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container image published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "e3c32930195b12d26694f7613d459ef083055850": {"t": 1785749414.6413548, "n": 39627, "h": "c0e6fb4cdca3413f", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        # The flat ``llm/engine_eval.py`` / ``llm/elite_engine_service.py``\n        # modules were retired alongside host_app (PR #111) in the\n        # engine-library cleanup; live coverage moved to\n        # ``llm.seca.engines.stockfish.pool`` and is exercised by\n        # ``test_engine_pool_*.py`` plus the existing response-format\n        # / adapter-isolation regressions.\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_pool_evaluate_position.py \\\n            llm/tests/test_engine_pool_exhaustion.py \\\n            llm/tests/test_engine_pool_crash_recovery.py \\\n            llm/tests/test_fen_move_cache_key.py \\\n            llm/tests/test_stockfish_adapter_isolation.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v7\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        # PYSEC-2025-183 / CVE-2025-45768 is a DISPUTED advisory against\n        # pyjwt: it flags \"weak encryption\", but the upstream maintainer\n        # rejects the report because key length is application-controlled\n        # — see the NOTE field at\n        # https://api.osv.dev/v1/vulns/PYSEC-2025-183.  Our app already\n        # enforces SECRET_KEY length >= 32 chars in prod (validated at\n        # module load in llm/seca/auth/tokens.py; documented in\n        # .env.example and docs/THREAT_MODEL.md §T2), which is the\n        # strong-key property the advisory complains about.  No upstream\n        # fix version exists and one is unlikely to ship.  Revisit if\n        # pyjwt ever releases a version that explicitly addresses the\n        # advisory, or if our SECRET_KEY guarantees change.\n        #\n        # NOTE: PYSEC-2026-161 / CVE-2026-48710 (starlette \"BadHost\":\n        # missing Host-header validation poisons ``request.url.path`` and\n        # can bypass path-based security checks) was previously suppressed\n        # here while fastapi 0.128.1 capped ``starlette<0.51.0``.  That cap\n        # is gone — fastapi 0.136.3 requires only ``starlette>=0.46.0`` —\n        # so the runtime now ships starlette 1.2.1 (past the fixed 1.0.1)\n        # and the advisory is closed at the dependency level.  The\n        # ``--ignore-vuln`` was therefore removed; if a vulnerable\n        # starlette is ever re-pinned, this audit step fails (and so does\n        # llm/tests/test_security_host_header_poisoning.py).\n        run: python -m pip_audit -r llm/requirements.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        # Only the disputed pyjwt advisory remains suppressed here:\n        #   - PYSEC-2025-183 disputed pyjwt advisory + SECRET_KEY\n        #     mitigation apply identically (pyjwt is also pulled in\n        #     transitively by the CI dependency tree, e.g. test\n        #     fixtures that mint tokens against the auth router).\n        # PYSEC-2026-161 (starlette BadHost) is no longer suppressed:\n        # requirements-ci.txt installs the same starlette 1.2.1 as the\n        # runtime via ``-r requirements.txt``, which is past the 1.0.1 fix.\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v6\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v5\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v7\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v4\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v6\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v6\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v7\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v7\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v4\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v4\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              # Prune stale unused images.  Without this, every merge to main\n              # leaves prior cereveon-llm-api digests on disk; on the 38 GB\n              # Hetzner box the / volume fills after ~30-50 merges and the\n              # next deploy's SCP step fails with the misleading\n              # \"error copy file to dest: ***\" signature (4 consecutive\n              # deploys stuck on 2026-05-15 before `docker image prune -af`\n              # unblocked it).  See feedback memory \"Hetzner production\n              # deploy gotchas\" #4 for the full incident.\n              #\n              # --filter \"until=72h\" keeps the previous deploy's image so a\n              # manual rollback skips the GHCR round-trip; -af removes\n              # everything else not currently in use (the running api's\n              # image is referenced and therefore safe).  Errors do not\n              # fail the deploy — `|| true` keeps an otherwise-healthy\n              # rollout from being marked failed by a transient docker\n              # hiccup during prune.\n              docker image prune -af --filter \"until=72h\" || true\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "7267a690baef6e1d3cd63807ff30325322bdc956": {"t": 1785749416.2231808, "n": 40273, "h": "1bcdaf87597e48e3", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        # The flat ``llm/engine_eval.py`` / ``llm/elite_engine_service.py``\n        # modules were retired alongside host_app (PR #111) in the\n        # engine-library cleanup; live coverage moved to\n        # ``llm.seca.engines.stockfish.pool`` and is exercised by\n        # ``test_engine_pool_*.py`` plus the existing response-format\n        # / adapter-isolation regressions.\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_pool_evaluate_position.py \\\n            llm/tests/test_engine_pool_exhaustion.py \\\n            llm/tests/test_engine_pool_crash_recovery.py \\\n            llm/tests/test_fen_move_cache_key.py \\\n            llm/tests/test_stockfish_adapter_isolation.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v7\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        # PYSEC-2025-183 / CVE-2025-45768 is a DISPUTED advisory against\n        # pyjwt: it flags \"weak encryption\", but the upstream maintainer\n        # rejects the report because key length is application-controlled\n        # — see the NOTE field at\n        # https://api.osv.dev/v1/vulns/PYSEC-2025-183.  Our app already\n        # enforces SECRET_KEY length >= 32 chars in prod (validated at\n        # module load in llm/seca/auth/tokens.py; documented in\n        # .env.example and docs/THREAT_MODEL.md §T2), which is the\n        # strong-key property the advisory complains about.  No upstream\n        # fix version exists and one is unlikely to ship.  Revisit if\n        # pyjwt ever releases a version that explicitly addresses the\n        # advisory, or if our SECRET_KEY guarantees change.\n        #\n        # PYSEC-2026-161 is a REAL advisory against starlette 0.50.0:\n        # missing Host-header validation lets ``request.url.path`` be\n        # poisoned, which could bypass path-based security checks (fix\n        # is starlette 1.0.1).  Cannot bump in isolation —\n        # fastapi 0.128.1 pins ``starlette<0.51.0``, so closing the\n        # advisory requires a coordinated fastapi bump (tracked\n        # separately).  Mitigations in place until then:\n        #   1. Caddy reverse proxy uses a {$DOMAIN} site block (see\n        #      Caddyfile) which rejects requests with a Host header\n        #      that does not match the configured domain BEFORE they\n        #      reach uvicorn — the poisoned-Host attack vector is cut\n        #      at the edge.\n        #   2. Our auth path does NOT consume ``request.url.path``\n        #      for security decisions.  ``get_current_player``\n        #      validates the Bearer token and session_id only; routes\n        #      are matched by FastAPI's routing layer (operates on the\n        #      ACTUAL request path, not the poisoned reconstruction).\n        # Revisit when fastapi ships a release with\n        # ``starlette>=1.0.1`` (latest is 0.136.1 at the time of this\n        # commit — verify its starlette constraint before dropping\n        # this suppression).\n        run: python -m pip_audit -r llm/requirements.txt --strict --ignore-vuln PYSEC-2025-183 --ignore-vuln PYSEC-2026-161 --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        # Same suppression rationale as the runtime audit step above:\n        #   - PYSEC-2025-183 disputed pyjwt advisory + SECRET_KEY\n        #     mitigation apply identically (pyjwt is also pulled in\n        #     transitively by the CI dependency tree, e.g. test\n        #     fixtures that mint tokens against the auth router).\n        #   - PYSEC-2026-161 starlette Host-header advisory + Caddy\n        #     edge-mitigation apply identically (starlette is\n        #     transitively shared between runtime and CI deps).\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --ignore-vuln PYSEC-2025-183 --ignore-vuln PYSEC-2026-161 --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v6\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v5\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v7\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v4\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v6\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v6\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v7\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v7\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v4\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v4\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              # Prune stale unused images.  Without this, every merge to main\n              # leaves prior cereveon-llm-api digests on disk; on the 38 GB\n              # Hetzner box the / volume fills after ~30-50 merges and the\n              # next deploy's SCP step fails with the misleading\n              # \"error copy file to dest: ***\" signature (4 consecutive\n              # deploys stuck on 2026-05-15 before `docker image prune -af`\n              # unblocked it).  See feedback memory \"Hetzner production\n              # deploy gotchas\" #4 for the full incident.\n              #\n              # --filter \"until=72h\" keeps the previous deploy's image so a\n              # manual rollback skips the GHCR round-trip; -af removes\n              # everything else not currently in use (the running api's\n              # image is referenced and therefore safe).  Errors do not\n              # fail the deploy — `|| true` keeps an otherwise-healthy\n              # rollout from being marked failed by a transient docker\n              # hiccup during prune.\n              docker image prune -af --filter \"until=72h\" || true\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v6\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "3c142761edbe345c67b010a5776b335aaa1579f8": {"t": 1785749417.5323563, "n": 40273, "h": "d760e63ff2180968", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        # The flat ``llm/engine_eval.py`` / ``llm/elite_engine_service.py``\n        # modules were retired alongside host_app (PR #111) in the\n        # engine-library cleanup; live coverage moved to\n        # ``llm.seca.engines.stockfish.pool`` and is exercised by\n        # ``test_engine_pool_*.py`` plus the existing response-format\n        # / adapter-isolation regressions.\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_pool_evaluate_position.py \\\n            llm/tests/test_engine_pool_exhaustion.py \\\n            llm/tests/test_engine_pool_crash_recovery.py \\\n            llm/tests/test_fen_move_cache_key.py \\\n            llm/tests/test_stockfish_adapter_isolation.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v4\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        # PYSEC-2025-183 / CVE-2025-45768 is a DISPUTED advisory against\n        # pyjwt: it flags \"weak encryption\", but the upstream maintainer\n        # rejects the report because key length is application-controlled\n        # — see the NOTE field at\n        # https://api.osv.dev/v1/vulns/PYSEC-2025-183.  Our app already\n        # enforces SECRET_KEY length >= 32 chars in prod (validated at\n        # module load in llm/seca/auth/tokens.py; documented in\n        # .env.example and docs/THREAT_MODEL.md §T2), which is the\n        # strong-key property the advisory complains about.  No upstream\n        # fix version exists and one is unlikely to ship.  Revisit if\n        # pyjwt ever releases a version that explicitly addresses the\n        # advisory, or if our SECRET_KEY guarantees change.\n        #\n        # PYSEC-2026-161 is a REAL advisory against starlette 0.50.0:\n        # missing Host-header validation lets ``request.url.path`` be\n        # poisoned, which could bypass path-based security checks (fix\n        # is starlette 1.0.1).  Cannot bump in isolation —\n        # fastapi 0.128.1 pins ``starlette<0.51.0``, so closing the\n        # advisory requires a coordinated fastapi bump (tracked\n        # separately).  Mitigations in place until then:\n        #   1. Caddy reverse proxy uses a {$DOMAIN} site block (see\n        #      Caddyfile) which rejects requests with a Host header\n        #      that does not match the configured domain BEFORE they\n        #      reach uvicorn — the poisoned-Host attack vector is cut\n        #      at the edge.\n        #   2. Our auth path does NOT consume ``request.url.path``\n        #      for security decisions.  ``get_current_player``\n        #      validates the Bearer token and session_id only; routes\n        #      are matched by FastAPI's routing layer (operates on the\n        #      ACTUAL request path, not the poisoned reconstruction).\n        # Revisit when fastapi ships a release with\n        # ``starlette>=1.0.1`` (latest is 0.136.1 at the time of this\n        # commit — verify its starlette constraint before dropping\n        # this suppression).\n        run: python -m pip_audit -r llm/requirements.txt --strict --ignore-vuln PYSEC-2025-183 --ignore-vuln PYSEC-2026-161 --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        # Same suppression rationale as the runtime audit step above:\n        #   - PYSEC-2025-183 disputed pyjwt advisory + SECRET_KEY\n        #     mitigation apply identically (pyjwt is also pulled in\n        #     transitively by the CI dependency tree, e.g. test\n        #     fixtures that mint tokens against the auth router).\n        #   - PYSEC-2026-161 starlette Host-header advisory + Caddy\n        #     edge-mitigation apply identically (starlette is\n        #     transitively shared between runtime and CI deps).\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --ignore-vuln PYSEC-2025-183 --ignore-vuln PYSEC-2026-161 --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v4\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v4\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v4\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v4\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v6\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v6\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v7\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v7\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v4\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v4\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v4\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              # Prune stale unused images.  Without this, every merge to main\n              # leaves prior cereveon-llm-api digests on disk; on the 38 GB\n              # Hetzner box the / volume fills after ~30-50 merges and the\n              # next deploy's SCP step fails with the misleading\n              # \"error copy file to dest: ***\" signature (4 consecutive\n              # deploys stuck on 2026-05-15 before `docker image prune -af`\n              # unblocked it).  See feedback memory \"Hetzner production\n              # deploy gotchas\" #4 for the full incident.\n              #\n              # --filter \"until=72h\" keeps the previous deploy's image so a\n              # manual rollback skips the GHCR round-trip; -af removes\n              # everything else not currently in use (the running api's\n              # image is referenced and therefore safe).  Errors do not\n              # fail the deploy — `|| true` keeps an otherwise-healthy\n              # rollout from being marked failed by a transient docker\n              # hiccup during prune.\n              docker image prune -af --filter \"until=72h\" || true\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v6\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "883b3a16c60ede6802679c9c5879662f3f7e2d98": {"t": 1785749418.830172, "n": 40273, "h": "902e1ded6804dfb0", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        # The flat ``llm/engine_eval.py`` / ``llm/elite_engine_service.py``\n        # modules were retired alongside host_app (PR #111) in the\n        # engine-library cleanup; live coverage moved to\n        # ``llm.seca.engines.stockfish.pool`` and is exercised by\n        # ``test_engine_pool_*.py`` plus the existing response-format\n        # / adapter-isolation regressions.\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_pool_evaluate_position.py \\\n            llm/tests/test_engine_pool_exhaustion.py \\\n            llm/tests/test_engine_pool_crash_recovery.py \\\n            llm/tests/test_fen_move_cache_key.py \\\n            llm/tests/test_stockfish_adapter_isolation.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v4\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        # PYSEC-2025-183 / CVE-2025-45768 is a DISPUTED advisory against\n        # pyjwt: it flags \"weak encryption\", but the upstream maintainer\n        # rejects the report because key length is application-controlled\n        # — see the NOTE field at\n        # https://api.osv.dev/v1/vulns/PYSEC-2025-183.  Our app already\n        # enforces SECRET_KEY length >= 32 chars in prod (validated at\n        # module load in llm/seca/auth/tokens.py; documented in\n        # .env.example and docs/THREAT_MODEL.md §T2), which is the\n        # strong-key property the advisory complains about.  No upstream\n        # fix version exists and one is unlikely to ship.  Revisit if\n        # pyjwt ever releases a version that explicitly addresses the\n        # advisory, or if our SECRET_KEY guarantees change.\n        #\n        # PYSEC-2026-161 is a REAL advisory against starlette 0.50.0:\n        # missing Host-header validation lets ``request.url.path`` be\n        # poisoned, which could bypass path-based security checks (fix\n        # is starlette 1.0.1).  Cannot bump in isolation —\n        # fastapi 0.128.1 pins ``starlette<0.51.0``, so closing the\n        # advisory requires a coordinated fastapi bump (tracked\n        # separately).  Mitigations in place until then:\n        #   1. Caddy reverse proxy uses a {$DOMAIN} site block (see\n        #      Caddyfile) which rejects requests with a Host header\n        #      that does not match the configured domain BEFORE they\n        #      reach uvicorn — the poisoned-Host attack vector is cut\n        #      at the edge.\n        #   2. Our auth path does NOT consume ``request.url.path``\n        #      for security decisions.  ``get_current_player``\n        #      validates the Bearer token and session_id only; routes\n        #      are matched by FastAPI's routing layer (operates on the\n        #      ACTUAL request path, not the poisoned reconstruction).\n        # Revisit when fastapi ships a release with\n        # ``starlette>=1.0.1`` (latest is 0.136.1 at the time of this\n        # commit — verify its starlette constraint before dropping\n        # this suppression).\n        run: python -m pip_audit -r llm/requirements.txt --strict --ignore-vuln PYSEC-2025-183 --ignore-vuln PYSEC-2026-161 --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        # Same suppression rationale as the runtime audit step above:\n        #   - PYSEC-2025-183 disputed pyjwt advisory + SECRET_KEY\n        #     mitigation apply identically (pyjwt is also pulled in\n        #     transitively by the CI dependency tree, e.g. test\n        #     fixtures that mint tokens against the auth router).\n        #   - PYSEC-2026-161 starlette Host-header advisory + Caddy\n        #     edge-mitigation apply identically (starlette is\n        #     transitively shared between runtime and CI deps).\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --ignore-vuln PYSEC-2025-183 --ignore-vuln PYSEC-2026-161 --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v4\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v4\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v4\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v3\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              # Prune stale unused images.  Without this, every merge to main\n              # leaves prior cereveon-llm-api digests on disk; on the 38 GB\n              # Hetzner box the / volume fills after ~30-50 merges and the\n              # next deploy's SCP step fails with the misleading\n              # \"error copy file to dest: ***\" signature (4 consecutive\n              # deploys stuck on 2026-05-15 before `docker image prune -af`\n              # unblocked it).  See feedback memory \"Hetzner production\n              # deploy gotchas\" #4 for the full incident.\n              #\n              # --filter \"until=72h\" keeps the previous deploy's image so a\n              # manual rollback skips the GHCR round-trip; -af removes\n              # everything else not currently in use (the running api's\n              # image is referenced and therefore safe).  Errors do not\n              # fail the deploy — `|| true` keeps an otherwise-healthy\n              # rollout from being marked failed by a transient docker\n              # hiccup during prune.\n              docker image prune -af --filter \"until=72h\" || true\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "28897afbaf4ffbdb1606e9c22e0f1e09234637ee": {"t": 1785749420.1500287, "n": 38696, "h": "e17efed6e5e084e2", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        # The flat ``llm/engine_eval.py`` / ``llm/elite_engine_service.py``\n        # modules were retired alongside host_app (PR #111) in the\n        # engine-library cleanup; live coverage moved to\n        # ``llm.seca.engines.stockfish.pool`` and is exercised by\n        # ``test_engine_pool_*.py`` plus the existing response-format\n        # / adapter-isolation regressions.\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_pool_evaluate_position.py \\\n            llm/tests/test_engine_pool_exhaustion.py \\\n            llm/tests/test_engine_pool_crash_recovery.py \\\n            llm/tests/test_fen_move_cache_key.py \\\n            llm/tests/test_stockfish_adapter_isolation.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v4\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        # PYSEC-2025-183 / CVE-2025-45768 is a DISPUTED advisory against\n        # pyjwt: it flags \"weak encryption\", but the upstream maintainer\n        # rejects the report because key length is application-controlled\n        # — see the NOTE field at\n        # https://api.osv.dev/v1/vulns/PYSEC-2025-183.  Our app already\n        # enforces SECRET_KEY length >= 32 chars in prod (validated at\n        # module load in llm/seca/auth/tokens.py; documented in\n        # .env.example and docs/THREAT_MODEL.md §T2), which is the\n        # strong-key property the advisory complains about.  No upstream\n        # fix version exists and one is unlikely to ship.  Revisit if\n        # pyjwt ever releases a version that explicitly addresses the\n        # advisory, or if our SECRET_KEY guarantees change.\n        run: python -m pip_audit -r llm/requirements.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        # Same disputed PYSEC-2025-183 suppression rationale as the\n        # runtime audit step above — pyjwt is also pulled in\n        # transitively by the CI dependency tree (e.g. test fixtures\n        # that mint tokens against the auth router), and the disputed\n        # advisory + SECRET_KEY mitigation both apply identically here.\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v4\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v4\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v4\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v3\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              # Prune stale unused images.  Without this, every merge to main\n              # leaves prior cereveon-llm-api digests on disk; on the 38 GB\n              # Hetzner box the / volume fills after ~30-50 merges and the\n              # next deploy's SCP step fails with the misleading\n              # \"error copy file to dest: ***\" signature (4 consecutive\n              # deploys stuck on 2026-05-15 before `docker image prune -af`\n              # unblocked it).  See feedback memory \"Hetzner production\n              # deploy gotchas\" #4 for the full incident.\n              #\n              # --filter \"until=72h\" keeps the previous deploy's image so a\n              # manual rollback skips the GHCR round-trip; -af removes\n              # everything else not currently in use (the running api's\n              # image is referenced and therefore safe).  Errors do not\n              # fail the deploy — `|| true` keeps an otherwise-healthy\n              # rollout from being marked failed by a transient docker\n              # hiccup during prune.\n              docker image prune -af --filter \"until=72h\" || true\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "b6ef08a5fc6644396e3e83228f8ede40e101b658": {"t": 1785749421.4528053, "n": 38327, "h": "6046593ca7a9625c", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        # The flat ``llm/engine_eval.py`` / ``llm/elite_engine_service.py``\n        # modules were retired alongside host_app (PR #111) in the\n        # engine-library cleanup; live coverage moved to\n        # ``llm.seca.engines.stockfish.pool`` and is exercised by\n        # ``test_engine_pool_*.py`` plus the existing response-format\n        # / adapter-isolation regressions.\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_pool_evaluate_position.py \\\n            llm/tests/test_engine_pool_exhaustion.py \\\n            llm/tests/test_engine_pool_crash_recovery.py \\\n            llm/tests/test_fen_move_cache_key.py \\\n            llm/tests/test_stockfish_adapter_isolation.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v4\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        # PYSEC-2025-183 / CVE-2025-45768 is a DISPUTED advisory against\n        # pyjwt: it flags \"weak encryption\", but the upstream maintainer\n        # rejects the report because key length is application-controlled\n        # — see the NOTE field at\n        # https://api.osv.dev/v1/vulns/PYSEC-2025-183.  Our app already\n        # enforces SECRET_KEY length >= 32 chars in prod (validated at\n        # module load in llm/seca/auth/tokens.py; documented in\n        # .env.example and docs/THREAT_MODEL.md §T2), which is the\n        # strong-key property the advisory complains about.  No upstream\n        # fix version exists and one is unlikely to ship.  Revisit if\n        # pyjwt ever releases a version that explicitly addresses the\n        # advisory, or if our SECRET_KEY guarantees change.\n        run: python -m pip_audit -r llm/requirements.txt --strict --ignore-vuln PYSEC-2025-183 --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v4\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v4\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v4\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v3\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              # Prune stale unused images.  Without this, every merge to main\n              # leaves prior cereveon-llm-api digests on disk; on the 38 GB\n              # Hetzner box the / volume fills after ~30-50 merges and the\n              # next deploy's SCP step fails with the misleading\n              # \"error copy file to dest: ***\" signature (4 consecutive\n              # deploys stuck on 2026-05-15 before `docker image prune -af`\n              # unblocked it).  See feedback memory \"Hetzner production\n              # deploy gotchas\" #4 for the full incident.\n              #\n              # --filter \"until=72h\" keeps the previous deploy's image so a\n              # manual rollback skips the GHCR round-trip; -af removes\n              # everything else not currently in use (the running api's\n              # image is referenced and therefore safe).  Errors do not\n              # fail the deploy — `|| true` keeps an otherwise-healthy\n              # rollout from being marked failed by a transient docker\n              # hiccup during prune.\n              docker image prune -af --filter \"until=72h\" || true\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "4e18ff2f0485bc49f6cfcf0cf094706b4878b1d2": {"t": 1785749422.7783532, "n": 37497, "h": "2f72ccae3cbc49e5", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        # The flat ``llm/engine_eval.py`` / ``llm/elite_engine_service.py``\n        # modules were retired alongside host_app (PR #111) in the\n        # engine-library cleanup; live coverage moved to\n        # ``llm.seca.engines.stockfish.pool`` and is exercised by\n        # ``test_engine_pool_*.py`` plus the existing response-format\n        # / adapter-isolation regressions.\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_pool_evaluate_position.py \\\n            llm/tests/test_engine_pool_exhaustion.py \\\n            llm/tests/test_engine_pool_crash_recovery.py \\\n            llm/tests/test_fen_move_cache_key.py \\\n            llm/tests/test_stockfish_adapter_isolation.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v4\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        run: python -m pip_audit -r llm/requirements.txt --strict --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v4\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v4\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v4\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v3\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              # Prune stale unused images.  Without this, every merge to main\n              # leaves prior cereveon-llm-api digests on disk; on the 38 GB\n              # Hetzner box the / volume fills after ~30-50 merges and the\n              # next deploy's SCP step fails with the misleading\n              # \"error copy file to dest: ***\" signature (4 consecutive\n              # deploys stuck on 2026-05-15 before `docker image prune -af`\n              # unblocked it).  See feedback memory \"Hetzner production\n              # deploy gotchas\" #4 for the full incident.\n              #\n              # --filter \"until=72h\" keeps the previous deploy's image so a\n              # manual rollback skips the GHCR round-trip; -af removes\n              # everything else not currently in use (the running api's\n              # image is referenced and therefore safe).  Errors do not\n              # fail the deploy — `|| true` keeps an otherwise-healthy\n              # rollout from being marked failed by a transient docker\n              # hiccup during prune.\n              docker image prune -af --filter \"until=72h\" || true\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "3cb5ddd5e589ce5a4b3b2e8cad60db86f1d3060e": {"t": 1785749424.214637, "n": 36373, "h": "0bdf609741a80455", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        # The flat ``llm/engine_eval.py`` / ``llm/elite_engine_service.py``\n        # modules were retired alongside host_app (PR #111) in the\n        # engine-library cleanup; live coverage moved to\n        # ``llm.seca.engines.stockfish.pool`` and is exercised by\n        # ``test_engine_pool_*.py`` plus the existing response-format\n        # / adapter-isolation regressions.\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_pool_evaluate_position.py \\\n            llm/tests/test_engine_pool_exhaustion.py \\\n            llm/tests/test_engine_pool_crash_recovery.py \\\n            llm/tests/test_fen_move_cache_key.py \\\n            llm/tests/test_stockfish_adapter_isolation.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v4\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        run: python -m pip_audit -r llm/requirements.txt --strict --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v4\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v4\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v4\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v3\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "171d50fe3da4ba7b579a75aeb4ed689f28d4375f": {"t": 1785749425.9877498, "n": 36106, "h": "72f50bf0e7db7c93", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_eval_lru_cache.py \\\n            llm/tests/test_engine_eval_limits.py \\\n            llm/tests/test_engine_eval_benchmark.py \\\n            llm/tests/test_engine_eval_fallback_cache.py \\\n            llm/tests/test_stockfish_adapter_isolation.py \\\n            llm/tests/test_elite_engine_service.py \\\n            llm/tests/test_elite_engine_service_resolve_limits.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v4\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        run: python -m pip_audit -r llm/requirements.txt --strict --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v4\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v4\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v4\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v3\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.  The secret-presence check is surfaced via a\n    # job-level ``env:`` because GitHub Actions disallows\n    # ``secrets.X`` directly in step-level ``if:`` expressions (it\n    # would create a side-channel oracle for whether a secret exists);\n    # using ``env:`` is the documented workaround.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    env:\n      HAS_DEEPSEEK_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: env.HAS_DEEPSEEK_KEY != 'true'\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: env.HAS_DEEPSEEK_KEY == 'true'\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "58e4551966ad6d7136119f0cdd84d7a10e329afd": {"t": 1785749427.264733, "n": 35776, "h": "78bee74666b8c56c", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_eval_lru_cache.py \\\n            llm/tests/test_engine_eval_limits.py \\\n            llm/tests/test_engine_eval_benchmark.py \\\n            llm/tests/test_engine_eval_fallback_cache.py \\\n            llm/tests/test_stockfish_adapter_isolation.py \\\n            llm/tests/test_elite_engine_service.py \\\n            llm/tests/test_elite_engine_service_resolve_limits.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v4\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        run: python -m pip_audit -r llm/requirements.txt --strict --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v4\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v4\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v4\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_DEEPSEEK_API_KEY=stub' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v3\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (DeepSeek)\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    # Tag pushes only — never on per-push CI per TESTING.md Category D/C.\n    # Also runs on workflow_dispatch for ad-hoc pre-release verification.\n    # Both gated additionally on the COACH_DEEPSEEK_API_KEY secret being\n    # set: without it the job no-ops with a clear log line, so a fork or\n    # a repo without DeepSeek billing wired up doesn't spuriously fail\n    # the release path.\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Skip if COACH_DEEPSEEK_API_KEY is unset\n        if: ${{ secrets.COACH_DEEPSEEK_API_KEY == '' }}\n        run: |\n          echo \"::warning::COACH_DEEPSEEK_API_KEY secret is not set; skipping LLM regression and smoke tests.\"\n          echo \"To enable them, set COACH_DEEPSEEK_API_KEY in repo secrets (Settings → Secrets and variables → Actions).\"\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        if: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run DeepSeek smoke test (RELEASE.md Step 4)\n        if: ${{ secrets.COACH_DEEPSEEK_API_KEY != '' }}\n        env:\n          RUN_DEEPSEEK_TESTS: \"1\"\n          COACH_DEEPSEEK_API_KEY: ${{ secrets.COACH_DEEPSEEK_API_KEY }}\n        run: python -m pytest -q llm/rag/tests/llm/test_deepseek_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "484a48d812920d6940d6859e57311ec6f22eac5f": {"t": 1785749428.5418348, "n": 35617, "h": "65e5dd313f8efa59", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_eval_lru_cache.py \\\n            llm/tests/test_engine_eval_limits.py \\\n            llm/tests/test_engine_eval_benchmark.py \\\n            llm/tests/test_engine_eval_fallback_cache.py \\\n            llm/tests/test_stockfish_adapter_isolation.py \\\n            llm/tests/test_elite_engine_service.py \\\n            llm/tests/test_elite_engine_service_resolve_limits.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v4\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        run: python -m pip_audit -r llm/requirements.txt --strict --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v4\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v4\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v4\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_OLLAMA_MODEL=qwen2.5:7b-instruct-q2_K' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v3\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Stop-and-replace deploy (single-container).\n      #\n      # Why not rolling --scale api=2: the 4 GB Hetzner VPS cannot host two\n      # api containers in parallel.  Each api uses ~700 MB resident plus a\n      # transient peak when the Stockfish engine pool spawns at startup;\n      # combined with Postgres (200 MB), Redis (50 MB), and Caddy, the\n      # rolling pair pushes past the 3.7 GB limit.  In two consecutive\n      # deploys (PR #68 and PR #69) the rolling --scale api=2 strategy\n      # OOM-stalled — `docker inspect` on the new container hung under\n      # memory pressure, the SSH command_timeout (10 min) fired before the\n      # healthcheck loop completed, and the deploy was reported as failed.\n      # See feedback memory ‟Hetzner production deploy gotchas” for the\n      # full incident timeline.\n      #\n      # Trade-off: ~5-10 s of 502/503 from Caddy during the swap, in\n      # exchange for not OOMing the box.  Acceptable for our traffic\n      # profile (Android client retries idempotent GETs; chat POSTs are\n      # rare enough that the user-visible cost of one missed request is\n      # one offline-template fallback).  Bigger box (CX22, 8 GB) would\n      # restore the rolling-pair option — that's a separate followup.\n      #\n      # Sequence:\n      #  1. Record running container's image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --force-recreate api: stop old, remove, create new with new image.\n      #     Caddy briefly serves 502 until the new container's healthcheck\n      #     passes; total downtime usually 5-10 s.\n      #  4. Poll Docker's healthcheck on the new container (up to 90 s).\n      #  5a. Healthy → done.\n      #  5b. Unhealthy → re-recreate with the previous image.  Adds another\n      #      5-10 s of downtime but auto-restores the previous good state.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container's image for rollback ──────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Stop-and-replace: tear down old api, bring up new ──────────\n            #    --force-recreate stops the existing container, removes it,\n            #    and creates a new one with the new image (passed via the\n            #    GHCR_IMAGE env var that docker-compose.prod.yml reads).\n            #    Brief 502 from Caddy until the new container is healthy.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --force-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::No api container present after --force-recreate.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: deploy done ──────────────────────────────────────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              echo \"Deployment complete.\"\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: roll back to the previous image ────────────────\n            #    With stop-and-replace there is no surviving old container to\n            #    fall back on, so we re-recreate api with the previous image.\n            #    This adds another ~5-10 s of downtime, but auto-restores\n            #    the previous good state without operator intervention.\n            echo \"::error::New container unhealthy ($STATUS). Rolling back to ${PREV_IMAGE:-(none)}...\"\n            if [ -n \"${PREV_IMAGE:-}\" ]; then\n              GHCR_IMAGE=\"$PREV_IMAGE\" \\\n                docker compose -f docker-compose.prod.yml up -d --force-recreate api\n              echo \"Rolled back to $PREV_IMAGE.\"\n            else\n              echo \"::error::No previous image recorded; cannot auto-roll-back.  Investigate manually.\"\n            fi\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (Ollama)\n    runs-on: ubuntu-latest\n    timeout-minutes: 60\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Cache Ollama model weights\n        uses: actions/cache@v5\n        with:\n          path: ~/.ollama/models\n          key: ollama-qwen2.5-7b-instruct-q2_K-${{ runner.os }}\n\n      - name: Install Ollama\n        run: curl -fsSL https://ollama.com/install.sh | sh\n\n      - name: Start Ollama server and wait for ready\n        run: |\n          ollama serve &\n          for i in $(seq 1 30); do\n            sleep 2\n            if curl -sf http://127.0.0.1:11434/api/tags > /dev/null 2>&1; then\n              echo \"Ollama ready after $((i * 2))s\"\n              break\n            fi\n            if [ \"$i\" -eq 30 ]; then\n              echo \"::error::Ollama did not start within 60s\"\n              exit 1\n            fi\n          done\n\n      - name: Pull model (skipped when cache hit)\n        run: ollama pull qwen2.5:7b-instruct-q2_K\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        env:\n          RUN_OLLAMA_TESTS: \"1\"\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run Ollama smoke test (RELEASE.md Step 4)\n        env:\n          RUN_OLLAMA_TESTS: \"1\"\n        run: python -m pytest -q llm/rag/tests/llm/test_ollama_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "c3fbb2d05e35438710280f48d6d6d4af7090f99d": {"t": 1785749429.8182178, "n": 34181, "h": "2d1054b46e5e8c50", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_eval_lru_cache.py \\\n            llm/tests/test_engine_eval_limits.py \\\n            llm/tests/test_engine_eval_benchmark.py \\\n            llm/tests/test_engine_eval_fallback_cache.py \\\n            llm/tests/test_stockfish_adapter_isolation.py \\\n            llm/tests/test_elite_engine_service.py \\\n            llm/tests/test_elite_engine_service_resolve_limits.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v4\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        run: python -m pip_audit -r llm/requirements.txt --strict --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v4\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v4\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v4\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_OLLAMA_MODEL=qwen2.5:7b-instruct-q2_K' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v3\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Sync the deploy-time configs from the just-checked-out commit BEFORE\n      # the SSH step runs `docker compose up`.  Without this, the compose\n      # file and Caddyfile on /opt/chesscoach/ are whatever was last copied\n      # there manually — they don't update with the repo, so every deploy\n      # uses a stale topology.  Concrete failure mode: PR #65 dropped the\n      # `ollama` service from docker-compose.prod.yml in the repo, but the\n      # on-box file still declared it; subsequent `docker compose up` runs\n      # silently re-created the orphan ollama container, OOM-cascading the\n      # 4 GB box.  Now the deploy is repo-as-source-of-truth: any change\n      # to the compose file or Caddyfile is in effect on the next merge.\n      # The .env.prod file is intentionally NOT synced — it holds runtime\n      # secrets that never live in the repo.\n      - name: Sync compose file and Caddyfile to Hetzner\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/scp-action@v1.0.0\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          source: \"docker-compose.prod.yml,Caddyfile\"\n          target: \"/opt/chesscoach\"\n          overwrite: true\n\n      # Zero-downtime rolling deploy (blue/green-style via Docker Compose scale):\n      #\n      #  1. Record old container ID and image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --scale api=2 --no-recreate: start NEW container alongside old one.\n      #     Caddy proxies to the \"api\" service name; Docker's internal DNS\n      #     round-robins across all containers — both serve traffic immediately.\n      #  4. Poll Docker's own healthcheck on the NEW container (not round-robin).\n      #  5a. Healthy → gracefully drain old container (SIGTERM + 30 s), remove it.\n      #      Traffic shifts entirely to the new container. Zero downtime.\n      #  5b. Unhealthy → stop and remove new container. Old stays alive.\n      #      Automatic rollback, no manual intervention required.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container for rollback reference ────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Scale to 2: start new container alongside old ──────────────\n            #    --no-recreate keeps the running container untouched.\n            #    --scale api=2 adds a second instance with the new image.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --scale api=2 --no-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api \\\n              | grep -vxF \"${PREV_CONTAINER:-}\" | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::Could not identify new container after scale-up.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: gracefully drain old, traffic shifts to new ──────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              if [ -n \"${PREV_CONTAINER:-}\" ]; then\n                docker stop --time 30 \"$PREV_CONTAINER\"\n                docker rm \"$PREV_CONTAINER\"\n              fi\n              echo \"Deployment complete.\"\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: remove new, old container keeps serving ────────\n            echo \"::error::New container unhealthy ($STATUS). Rolling back...\"\n            docker stop \"$NEW_CONTAINER\" 2>/dev/null || true\n            docker rm  \"$NEW_CONTAINER\" 2>/dev/null || true\n            echo \"Rolled back — ${PREV_IMAGE:-original container} still running.\"\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (Ollama)\n    runs-on: ubuntu-latest\n    timeout-minutes: 60\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Cache Ollama model weights\n        uses: actions/cache@v5\n        with:\n          path: ~/.ollama/models\n          key: ollama-qwen2.5-7b-instruct-q2_K-${{ runner.os }}\n\n      - name: Install Ollama\n        run: curl -fsSL https://ollama.com/install.sh | sh\n\n      - name: Start Ollama server and wait for ready\n        run: |\n          ollama serve &\n          for i in $(seq 1 30); do\n            sleep 2\n            if curl -sf http://127.0.0.1:11434/api/tags > /dev/null 2>&1; then\n              echo \"Ollama ready after $((i * 2))s\"\n              break\n            fi\n            if [ \"$i\" -eq 30 ]; then\n              echo \"::error::Ollama did not start within 60s\"\n              exit 1\n            fi\n          done\n\n      - name: Pull model (skipped when cache hit)\n        run: ollama pull qwen2.5:7b-instruct-q2_K\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        env:\n          RUN_OLLAMA_TESTS: \"1\"\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run Ollama smoke test (RELEASE.md Step 4)\n        env:\n          RUN_OLLAMA_TESTS: \"1\"\n        run: python -m pytest -q llm/rag/tests/llm/test_ollama_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "927e295500edff32a69296f54bba5e0f55c42e8f": {"t": 1785749431.258329, "n": 32910, "h": "cc7a356adc12f9cc", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_eval_lru_cache.py \\\n            llm/tests/test_engine_eval_limits.py \\\n            llm/tests/test_engine_eval_benchmark.py \\\n            llm/tests/test_engine_eval_fallback_cache.py \\\n            llm/tests/test_stockfish_adapter_isolation.py \\\n            llm/tests/test_elite_engine_service.py \\\n            llm/tests/test_elite_engine_service_resolve_limits.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v4\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        run: python -m pip_audit -r llm/requirements.txt --strict --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v4\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v4\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v4\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_OLLAMA_MODEL=qwen2.5:7b-instruct-q2_K' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v3\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable=true\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Zero-downtime rolling deploy (blue/green-style via Docker Compose scale):\n      #\n      #  1. Record old container ID and image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --scale api=2 --no-recreate: start NEW container alongside old one.\n      #     Caddy proxies to the \"api\" service name; Docker's internal DNS\n      #     round-robins across all containers — both serve traffic immediately.\n      #  4. Poll Docker's own healthcheck on the NEW container (not round-robin).\n      #  5a. Healthy → gracefully drain old container (SIGTERM + 30 s), remove it.\n      #      Traffic shifts entirely to the new container. Zero downtime.\n      #  5b. Unhealthy → stop and remove new container. Old stays alive.\n      #      Automatic rollback, no manual intervention required.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container for rollback reference ────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Scale to 2: start new container alongside old ──────────────\n            #    --no-recreate keeps the running container untouched.\n            #    --scale api=2 adds a second instance with the new image.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --scale api=2 --no-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api \\\n              | grep -vxF \"${PREV_CONTAINER:-}\" | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::Could not identify new container after scale-up.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: gracefully drain old, traffic shifts to new ──────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              if [ -n \"${PREV_CONTAINER:-}\" ]; then\n                docker stop --time 30 \"$PREV_CONTAINER\"\n                docker rm \"$PREV_CONTAINER\"\n              fi\n              echo \"Deployment complete.\"\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: remove new, old container keeps serving ────────\n            echo \"::error::New container unhealthy ($STATUS). Rolling back...\"\n            docker stop \"$NEW_CONTAINER\" 2>/dev/null || true\n            docker rm  \"$NEW_CONTAINER\" 2>/dev/null || true\n            echo \"Rolled back — ${PREV_IMAGE:-original container} still running.\"\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (Ollama)\n    runs-on: ubuntu-latest\n    timeout-minutes: 60\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Cache Ollama model weights\n        uses: actions/cache@v5\n        with:\n          path: ~/.ollama/models\n          key: ollama-qwen2.5-7b-instruct-q2_K-${{ runner.os }}\n\n      - name: Install Ollama\n        run: curl -fsSL https://ollama.com/install.sh | sh\n\n      - name: Start Ollama server and wait for ready\n        run: |\n          ollama serve &\n          for i in $(seq 1 30); do\n            sleep 2\n            if curl -sf http://127.0.0.1:11434/api/tags > /dev/null 2>&1; then\n              echo \"Ollama ready after $((i * 2))s\"\n              break\n            fi\n            if [ \"$i\" -eq 30 ]; then\n              echo \"::error::Ollama did not start within 60s\"\n              exit 1\n            fi\n          done\n\n      - name: Pull model (skipped when cache hit)\n        run: ollama pull qwen2.5:7b-instruct-q2_K\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        env:\n          RUN_OLLAMA_TESTS: \"1\"\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run Ollama smoke test (RELEASE.md Step 4)\n        env:\n          RUN_OLLAMA_TESTS: \"1\"\n        run: python -m pytest -q llm/rag/tests/llm/test_ollama_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}, "da949ca39283ce98c2f82be676880c8562d5226d": {"t": 1785749432.5256383, "n": 32944, "h": "84ac9fa8f0b1ed25", "c": "# CI/CD pipeline for ChessCoach.\n#\n# Despite the historical filename, this workflow deploys to TWO production\n# tiers built from this monorepo:\n#\n#   1. Hetzner heavy backend  — `cereveon-llm-api` image (Python\n#      FastAPI + Stockfish + full SECA pipeline, port 8000).  Updated by\n#      the `deploy` job below via SSH + docker compose.\n#   2. Fly.io edge            — `cereveon` image (Node.js Express\n#      front, port 3000, global regional distribution).  Updated by the\n#      `fly-deploy` job below via flyctl, sequenced after Hetzner so the\n#      backend's new contract is live before the edge calls it.\n#\n# See docs/DEPLOYMENT.md for the topology + how each tier gets its\n# secrets.  The workflow's filename is preserved for URL stability across\n# external bookmarks; the workflow's `name:` field (CI/CD) is the\n# authoritative description.\nname: CI/CD\n\non:\n  pull_request:\n    branches: [main]\n  push:\n    branches: [main]\n    tags:\n      - \"v*.*.*\"\n  workflow_dispatch:\n\npermissions:\n  contents: read\n\nenv:\n  APP_IMAGE_NAME: cereveon\n  API_IMAGE_NAME: cereveon-llm-api\n\nconcurrency:\n  group: ci-cd-${{ github.workflow }}-${{ github.ref }}\n  cancel-in-progress: ${{ github.event_name == 'pull_request' }}\n\njobs:\n  workflow-lint:\n    name: actionlint\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Lint GitHub Actions workflows\n        uses: raven-actions/actionlint@v2\n\n  python-tests:\n    name: Python tests and coverage\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run golden tests (Category A — mandatory)\n        run: python -m pytest -q llm/rag/tests/golden/test_retriever.py llm/rag/tests/golden/test_prompt_snapshot.py\n\n      - name: Run LLM contract tests (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_fake_llm.py\n\n      - name: Run validator violations corpus (Category B — mandatory)\n        run: python -m pytest -q llm/rag/tests/contracts/test_violations_corpus.py\n\n      - name: Run API contract validation\n        run: python -m pytest -q llm/tests/test_api_contract_validation.py\n\n      - name: Run coaching pipeline regression tests\n        run: python -m pytest -q llm/tests/test_coaching_pipeline_regression.py\n\n      - name: Run explain schema validation tests\n        run: python -m pytest -q llm/tests/test_explain_schema_validation.py\n\n      - name: Run engine regression tests\n        run: |\n          python -m pytest -q --tb=short \\\n            llm/tests/test_engine_response_format.py \\\n            llm/tests/test_engine_eval_lru_cache.py \\\n            llm/tests/test_engine_eval_limits.py \\\n            llm/tests/test_engine_eval_benchmark.py \\\n            llm/tests/test_engine_eval_fallback_cache.py \\\n            llm/tests/test_stockfish_adapter_isolation.py \\\n            llm/tests/test_elite_engine_service.py \\\n            llm/tests/test_elite_engine_service_resolve_limits.py\n\n      - name: Run API security tests\n        run: python -m pytest -q --tb=short llm/tests/test_api_security.py\n\n      - name: Run regression pipeline\n        run: python llm/run_regression_suite.py\n\n      - name: Run pytest suite with coverage\n        run: python llm/run_ci_suite.py\n\n      - name: Upload test artifacts\n        if: always()\n        uses: actions/upload-artifact@v4\n        with:\n          name: python-test-artifacts\n          path: |\n            tmp_logs/coverage.xml\n            tmp_logs/pytest-ci.xml\n\n  python-quality:\n    name: Black, Pylint, and Mypy\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install quality dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Run Black\n        run: python llm/run_quality_gate.py black\n\n      - name: Run Pylint\n        run: python llm/run_quality_gate.py pylint\n\n      - name: Run Mypy\n        run: python llm/run_quality_gate.py mypy\n\n  dependency-security:\n    name: Python dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n\n      - name: Install pip-audit\n        run: |\n          python -m pip install --upgrade pip\n          pip install pip-audit==2.8.0\n\n      - name: Audit runtime dependencies\n        run: python -m pip_audit -r llm/requirements.txt --strict --progress-spinner off\n\n      - name: Audit CI/toolchain dependencies\n        run: python -m pip_audit -r llm/requirements-ci.txt --strict --progress-spinner off\n\n  node-security:\n    name: Node dependency audit\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Node.js\n        uses: actions/setup-node@v4\n        with:\n          node-version: \"20\"\n\n      - name: Generate package lock for audit\n        working-directory: llm\n        run: npm install --package-lock-only --ignore-scripts --no-audit --no-fund\n\n      - name: Audit Node dependencies\n        working-directory: llm\n        run: npm audit --omit=dev --audit-level=high\n\n  android-build:\n    name: Android build and unit tests\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    permissions:\n      contents: read\n    env:\n      GRADLE_USER_HOME: ${{ github.workspace }}/.gradle\n      ANDROID_USER_HOME: ${{ github.workspace }}/.android\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up JDK 17\n        uses: actions/setup-java@v4\n        with:\n          distribution: temurin\n          java-version: '17'\n          cache: gradle\n\n      - name: Set up Android SDK\n        uses: android-actions/setup-android@v4\n\n      - name: Install Android SDK platform 36\n        run: sdkmanager \"platforms;android-36\"\n\n      - name: Run Android host JVM unit tests\n        working-directory: android\n        run: ./gradlew test --no-daemon\n\n      - name: Generate and validate packaged manifests\n        working-directory: android\n        run: ./gradlew processDebugManifestForPackage processReleaseManifestForPackage --no-daemon\n\n      - name: Verify packaged manifest includes INTERNET permission\n        shell: bash\n        run: |\n          debug_manifest=\"android/app/build/intermediates/packaged_manifests/debug/processDebugManifestForPackage/AndroidManifest.xml\"\n          release_manifest=\"android/app/build/intermediates/packaged_manifests/release/processReleaseManifestForPackage/AndroidManifest.xml\"\n          for manifest in \"$debug_manifest\" \"$release_manifest\"; do\n            test -f \"$manifest\"\n            grep -q 'android.permission.INTERNET' \"$manifest\"\n          done\n\n      # Decode the base64-encoded keystore secret into a temp file.\n      # Skipped on PRs: signing is only needed for the artefact produced on push.\n      - name: Decode release keystore\n        if: github.event_name == 'push'\n        id: keystore\n        shell: bash\n        env:\n          KEYSTORE_B64: ${{ secrets.KEYSTORE_BASE64 }}\n        run: |\n          if [ -n \"$KEYSTORE_B64\" ]; then\n            echo \"$KEYSTORE_B64\" | base64 -d > \"$RUNNER_TEMP/release.jks\"\n            echo \"path=$RUNNER_TEMP/release.jks\" >> \"$GITHUB_OUTPUT\"\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Append signing properties to gradle.properties so Gradle picks them up\n      # via findProperty(). Secrets are passed as env vars (GitHub masks them in\n      # logs); the keystore path is a derived value, not a secret.\n      # Skipped entirely when the keystore was not decoded (no KEYSTORE_BASE64).\n      - name: Write signing properties\n        if: steps.keystore.outputs.available == 'true'\n        shell: bash\n        env:\n          SIGNING_KEY_ALIAS: ${{ secrets.KEY_ALIAS }}\n          SIGNING_KEY_PASSWORD: ${{ secrets.KEY_PASSWORD }}\n          SIGNING_STORE_PASSWORD: ${{ secrets.STORE_PASSWORD }}\n        run: |\n          {\n            echo \"MYAPP_UPLOAD_STORE_FILE=${{ steps.keystore.outputs.path }}\"\n            echo \"MYAPP_UPLOAD_KEY_ALIAS=$SIGNING_KEY_ALIAS\"\n            echo \"MYAPP_UPLOAD_KEY_PASSWORD=$SIGNING_KEY_PASSWORD\"\n            echo \"MYAPP_UPLOAD_STORE_PASSWORD=$SIGNING_STORE_PASSWORD\"\n          } >> android/gradle.properties\n\n      # COACH_API_BASE is a plain URL (visible in the APK — not a secret).\n      # PR builds use dev defaults (no secrets exposed to PR runners).\n      # Push builds inject COACH_API_KEY for the production artefact.\n      - name: Build release APK (PR)\n        if: github.event_name == 'pull_request'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Build release APK (push)\n        if: github.event_name == 'push'\n        working-directory: android\n        env:\n          COACH_API_BASE: ${{ vars.COACH_API_BASE }}\n          COACH_API_KEY: ${{ secrets.COACH_API_KEY }}\n        run: ./gradlew assembleRelease --no-daemon\n\n      - name: Upload release APK\n        if: github.event_name == 'push'\n        uses: actions/upload-artifact@v4\n        with:\n          name: release-apk-${{ github.sha }}\n          path: android/app/build/outputs/apk/release/app-release-unsigned.apk\n          if-no-files-found: warn\n\n  compose-validate:\n    name: Validate Compose files and Caddyfile\n    runs-on: ubuntu-latest\n    timeout-minutes: 10\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Validate docker-compose.yml syntax\n        run: docker compose config --quiet\n\n      - name: Validate docker-compose.prod.yml syntax\n        env:\n          DOMAIN: test.example.com\n          GHCR_IMAGE: ghcr.io/owner/test:latest\n        run: |\n          printf '%s\\n' \\\n            'POSTGRES_DB=chesscoach' \\\n            'POSTGRES_USER=chesscoach' \\\n            'POSTGRES_PASSWORD=stub' \\\n            'SECA_ENV=prod' \\\n            'SECA_API_KEY=stub' \\\n            'SECRET_KEY=stub' \\\n            'CORS_ALLOWED_ORIGINS=https://test.example.com' \\\n            'DATABASE_URL=sqlite:///data/seca.db' \\\n            'COACH_OLLAMA_MODEL=qwen2.5:7b-instruct-q2_K' \\\n            > .env.prod\n          docker compose -f docker-compose.prod.yml config --quiet\n\n      - name: Validate Caddyfile syntax\n        run: |\n          docker run --rm \\\n            -e DOMAIN=test.example.com \\\n            -v \"$GITHUB_WORKSPACE/Caddyfile:/etc/caddy/Caddyfile:ro\" \\\n            caddy:2-alpine \\\n            caddy adapt --config /etc/caddy/Caddyfile --adapter caddyfile > /dev/null\n\n  docker-images:\n    name: Build and publish Docker images\n    runs-on: ubuntu-latest\n    timeout-minutes: 30\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - android-build\n      - compose-validate\n    permissions:\n      contents: read\n      packages: write\n      id-token: write\n      attestations: write\n    outputs:\n      image_owner: ${{ steps.prep.outputs.owner }}\n      app_digest: ${{ steps.build-app.outputs.digest }}\n      api_digest: ${{ steps.build-api.outputs.digest }}\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Set up Docker Buildx\n        uses: docker/setup-buildx-action@v3\n\n      - name: Log in to GHCR\n        if: github.event_name == 'push'\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Docker metadata for app image\n        id: meta-app\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable={{is_default_branch}}\n\n      - name: Docker metadata for llm API image\n        id: meta-api\n        uses: docker/metadata-action@v5\n        with:\n          images: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          tags: |\n            type=sha,format=short,prefix=sha-\n            type=ref,event=branch\n            type=ref,event=pr,prefix=pr-\n            type=semver,pattern={{version}}\n            type=semver,pattern={{major}}.{{minor}}\n            type=raw,value=latest,enable={{is_default_branch}}\n\n      - name: Build app image\n        id: build-app\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./Dockerfile\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-app.outputs.tags }}\n          labels: ${{ steps.meta-app.outputs.labels }}\n          cache-from: type=gha,scope=app-image\n          cache-to: type=gha,mode=max,scope=app-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Build llm API image\n        id: build-api\n        uses: docker/build-push-action@v6\n        with:\n          context: .\n          file: ./llm/Dockerfile.api\n          push: ${{ github.event_name == 'push' }}\n          tags: ${{ steps.meta-api.outputs.tags }}\n          labels: ${{ steps.meta-api.outputs.labels }}\n          cache-from: type=gha,scope=llm-api-image\n          cache-to: type=gha,mode=max,scope=llm-api-image\n          provenance: false\n          sbom: false\n          build-args: BUILDDATE=${{ github.run_id }}\n\n      - name: Attest app image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-app.outputs.digest }}\n          push-to-registry: true\n\n      - name: Attest llm API image provenance\n        if: github.event_name == 'push'\n        uses: actions/attest-build-provenance@v2\n        with:\n          subject-name: ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}\n          subject-digest: ${{ steps.build-api.outputs.digest }}\n          push-to-registry: true\n\n      - name: Install Cosign\n        if: github.event_name == 'push'\n        uses: sigstore/cosign-installer@v3\n\n      - name: Sign published images\n        if: github.event_name == 'push'\n        shell: bash\n        run: |\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}@${{ steps.build-app.outputs.digest }}\"\n          cosign sign --yes \"ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}@${{ steps.build-api.outputs.digest }}\"\n\n  image-security:\n    name: Trivy image scan\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - docker-images\n    if: github.event_name == 'push'\n    permissions:\n      contents: read\n      packages: read\n      security-events: write\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Log in to GHCR\n        uses: docker/login-action@v3\n        with:\n          registry: ghcr.io\n          username: ${{ github.repository_owner }}\n          password: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Cache Trivy vulnerability DB\n        uses: actions/cache@v5\n        with:\n          path: ${{ runner.temp }}/trivy\n          key: trivy-db-${{ runner.os }}-${{ github.run_id }}\n          restore-keys: trivy-db-${{ runner.os }}-\n\n      - name: Scan published app image\n        id: trivy-app-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate app image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n          format: sarif\n          output: trivy-app-image.sarif\n          scanners: vuln\n          vuln-type: library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload app image scan results\n        if: hashFiles('trivy-app-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-app-image.sarif\n          category: trivy-app-image\n\n      - name: Scan published llm API image\n        id: trivy-api-image\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: table\n          scanners: vuln\n          vuln-type: os,library\n          severity: CRITICAL\n          ignore-unfixed: true\n          exit-code: \"1\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Generate llm API image SARIF\n        continue-on-error: true\n        uses: aquasecurity/trivy-action@v0.36.0\n        with:\n          scan-type: image\n          image-ref: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n          format: sarif\n          output: trivy-api-image.sarif\n          scanners: vuln\n          vuln-type: os,library\n          severity: HIGH,CRITICAL\n          ignore-unfixed: false\n          exit-code: \"0\"\n          trivy-config: trivy.yaml\n          trivyignores: .trivyignore\n          ignore-policy: trivy-ignore-policy.rego\n        env:\n          TRIVY_CACHE_DIR: ${{ runner.temp }}/trivy\n          TRIVY_SKIP_DB_UPDATE: \"true\"\n          TRIVY_TIMEOUT: \"15m\"\n          TRIVY_USERNAME: ${{ github.repository_owner }}\n          TRIVY_PASSWORD: ${{ secrets.GITHUB_TOKEN }}\n\n      - name: Upload llm API image scan results\n        if: hashFiles('trivy-api-image.sarif') != ''\n        uses: github/codeql-action/upload-sarif@v4\n        with:\n          sarif_file: trivy-api-image.sarif\n          category: trivy-api-image\n\n\n  deploy:\n    name: Deploy to Hetzner\n    runs-on: ubuntu-latest\n    timeout-minutes: 20\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    concurrency:\n      group: hetzner-production\n      cancel-in-progress: false\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Check Hetzner secrets availability\n        id: hetzner-check\n        shell: bash\n        env:\n          HETZNER_HOST: ${{ secrets.HETZNER_HOST }}\n          HETZNER_SSH_KEY: ${{ secrets.HETZNER_SSH_KEY }}\n        run: |\n          if [ -z \"${HETZNER_HOST}\" ] || [ -z \"${HETZNER_SSH_KEY}\" ]; then\n            echo \"::warning::HETZNER_HOST or HETZNER_SSH_KEY not configured. Skipping Hetzner deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      # Zero-downtime rolling deploy (blue/green-style via Docker Compose scale):\n      #\n      #  1. Record old container ID and image (rollback reference).\n      #  2. Pull the exact digest produced by the docker-images job.\n      #  3. --scale api=2 --no-recreate: start NEW container alongside old one.\n      #     Caddy proxies to the \"api\" service name; Docker's internal DNS\n      #     round-robins across all containers — both serve traffic immediately.\n      #  4. Poll Docker's own healthcheck on the NEW container (not round-robin).\n      #  5a. Healthy → gracefully drain old container (SIGTERM + 30 s), remove it.\n      #      Traffic shifts entirely to the new container. Zero downtime.\n      #  5b. Unhealthy → stop and remove new container. Old stays alive.\n      #      Automatic rollback, no manual intervention required.\n      - name: Deploy to Hetzner via SSH\n        if: steps.hetzner-check.outputs.available == 'true'\n        uses: appleboy/ssh-action@v1.2.5\n        env:\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.API_IMAGE_NAME }}@${{ needs.docker-images.outputs.api_digest }}\n        with:\n          host: ${{ secrets.HETZNER_HOST }}\n          username: deploy\n          key: ${{ secrets.HETZNER_SSH_KEY }}\n          envs: DEPLOY_IMAGE\n          script: |\n            set -euo pipefail\n            cd /opt/chesscoach\n\n            # ── 0. Pre-flight: verify required secrets are in .env.prod ───────\n            #    The server crashes at module load (not health-check time) when\n            #    SECRET_KEY is missing in prod, which causes a silent 90-second\n            #    wait before the deploy rolls back.  Fail fast with a clear\n            #    message instead.\n            for _var in SECA_API_KEY SECRET_KEY SECA_ENV; do\n              if ! grep -qE \"^${_var}=.+\" .env.prod 2>/dev/null; then\n                echo \"::error::Required variable ${_var} is missing or empty in /opt/chesscoach/.env.prod\"\n                echo \"  Add it:  echo '${_var}=<value>' >> /opt/chesscoach/.env.prod\"\n                echo \"  SECRET_KEY hint:  python3 -c \\\"import secrets; print(secrets.token_hex(32))\\\"\"\n                exit 1\n              fi\n            done\n\n            # ── 1. Record running container for rollback reference ────────────\n            PREV_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api 2>/dev/null | head -1 || true)\n            PREV_IMAGE=$([ -n \"$PREV_CONTAINER\" ] && \\\n              docker inspect --format='{{.Config.Image}}' \"$PREV_CONTAINER\" 2>/dev/null || true)\n            echo \"Previous: ${PREV_IMAGE:-none}\"\n            echo \"Deploying: $DEPLOY_IMAGE\"\n\n            # ── 2. Pull pinned, CI-signed image by digest ─────────────────────\n            docker pull \"$DEPLOY_IMAGE\"\n\n            # ── 3. Scale to 2: start new container alongside old ──────────────\n            #    --no-recreate keeps the running container untouched.\n            #    --scale api=2 adds a second instance with the new image.\n            GHCR_IMAGE=\"$DEPLOY_IMAGE\" \\\n              docker compose -f docker-compose.prod.yml up -d --scale api=2 --no-recreate api\n\n            # ── 4. Identify the new container ─────────────────────────────────\n            NEW_CONTAINER=$(docker compose -f docker-compose.prod.yml ps -q api \\\n              | grep -vxF \"${PREV_CONTAINER:-}\" | head -1 || true)\n            if [ -z \"$NEW_CONTAINER\" ]; then\n              echo \"::error::Could not identify new container after scale-up.\"\n              exit 1\n            fi\n            echo \"New container: $NEW_CONTAINER\"\n\n            # ── 5. Wait up to 90 s for Docker healthcheck on new container ────\n            for i in $(seq 1 18); do\n              STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"unknown\")\n              [ \"$STATUS\" = \"healthy\" ] && break\n              echo \"  Health.Status: $STATUS ($i/18)\"\n              sleep 5\n            done\n            STATUS=$(docker inspect --format='{{.State.Health.Status}}' \"$NEW_CONTAINER\" 2>/dev/null || echo \"failed\")\n\n            # ── 6a. Healthy: gracefully drain old, traffic shifts to new ──────\n            if [ \"$STATUS\" = \"healthy\" ]; then\n              if [ -n \"${PREV_CONTAINER:-}\" ]; then\n                docker stop --time 30 \"$PREV_CONTAINER\"\n                docker rm \"$PREV_CONTAINER\"\n              fi\n              echo \"Deployment complete.\"\n              exit 0\n            fi\n\n            # ── 6b. Unhealthy: remove new, old container keeps serving ────────\n            echo \"::error::New container unhealthy ($STATUS). Rolling back...\"\n            docker stop \"$NEW_CONTAINER\" 2>/dev/null || true\n            docker rm  \"$NEW_CONTAINER\" 2>/dev/null || true\n            echo \"Rolled back — ${PREV_IMAGE:-original container} still running.\"\n            exit 1\n\n  fly-deploy:\n    name: Deploy Fly edge\n    runs-on: ubuntu-latest\n    timeout-minutes: 15\n    needs:\n      - docker-images\n      - image-security\n      - deploy\n    if: github.event_name == 'push' && github.ref == 'refs/heads/main'\n    permissions:\n      contents: read\n    environment:\n      name: production\n    # Distinct from `hetzner-production` — Hetzner and Fly each get their own\n    # concurrency group so the Fly deploy is not serialised against the\n    # Hetzner deploy of an unrelated push.  Within Fly, deploys still\n    # serialise (cancel-in-progress: false).\n    concurrency:\n      group: fly-production\n      cancel-in-progress: false\n    steps:\n      - name: Check Fly secret availability\n        id: fly-check\n        shell: bash\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n        run: |\n          if [ -z \"${FLY_API_TOKEN}\" ]; then\n            echo \"::warning::FLY_API_TOKEN not configured. Skipping Fly deploy.\"\n            echo \"available=false\" >> \"$GITHUB_OUTPUT\"\n          else\n            echo \"available=true\" >> \"$GITHUB_OUTPUT\"\n          fi\n\n      - name: Checkout repository\n        if: steps.fly-check.outputs.available == 'true'\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up flyctl\n        if: steps.fly-check.outputs.available == 'true'\n        uses: superfly/flyctl-actions/setup-flyctl@1.6\n\n      # Deploy the digest-pinned edge image built and signed earlier in the\n      # pipeline.  flyctl reads fly.toml from the working directory for app\n      # config (vm size, region, http_service); --image overrides the\n      # registry reference so we never re-build on a Fly remote builder.\n      #\n      # The image lives in GHCR — Fly machines must be able to pull it.\n      # Either make ghcr.io/${owner}/cereveon public, or run\n      # `flyctl auth docker` on the Fly side once with a GHCR PAT.  See\n      # docs/DEPLOYMENT.md §\"Fly.io edge\" for the one-time setup.\n      - name: Deploy edge image to Fly.io\n        if: steps.fly-check.outputs.available == 'true'\n        env:\n          FLY_API_TOKEN: ${{ secrets.FLY_API_TOKEN }}\n          DEPLOY_IMAGE: ghcr.io/${{ needs.docker-images.outputs.image_owner }}/${{ env.APP_IMAGE_NAME }}@${{ needs.docker-images.outputs.app_digest }}\n        run: |\n          echo \"Deploying $DEPLOY_IMAGE to fly.io app 'chesscoach'\"\n          flyctl deploy --image \"$DEPLOY_IMAGE\" --app chesscoach\n\n  llm-regression:\n    name: LLM regression and smoke tests (Ollama)\n    runs-on: ubuntu-latest\n    timeout-minutes: 60\n    if: startsWith(github.ref, 'refs/tags/v') || github.event_name == 'workflow_dispatch'\n    permissions:\n      contents: read\n    steps:\n      - name: Checkout repository\n        uses: actions/checkout@v4\n        with:\n          persist-credentials: false\n\n      - name: Set up Python\n        uses: actions/setup-python@v5\n        with:\n          python-version: \"3.13\"\n          cache: pip\n          cache-dependency-path: |\n            llm/requirements.txt\n            llm/requirements-ci.txt\n            pyproject.toml\n\n      - name: Install CI dependencies\n        run: |\n          python -m pip install --upgrade pip\n          pip install -r llm/requirements-ci.txt\n\n      - name: Cache Ollama model weights\n        uses: actions/cache@v5\n        with:\n          path: ~/.ollama/models\n          key: ollama-qwen2.5-7b-instruct-q2_K-${{ runner.os }}\n\n      - name: Install Ollama\n        run: curl -fsSL https://ollama.com/install.sh | sh\n\n      - name: Start Ollama server and wait for ready\n        run: |\n          ollama serve &\n          for i in $(seq 1 30); do\n            sleep 2\n            if curl -sf http://127.0.0.1:11434/api/tags > /dev/null 2>&1; then\n              echo \"Ollama ready after $((i * 2))s\"\n              break\n            fi\n            if [ \"$i\" -eq 30 ]; then\n              echo \"::error::Ollama did not start within 60s\"\n              exit 1\n            fi\n          done\n\n      - name: Pull model (skipped when cache hit)\n        run: ollama pull qwen2.5:7b-instruct-q2_K\n\n      - name: Run LLM regression tests (RELEASE.md Step 3)\n        env:\n          RUN_OLLAMA_TESTS: \"1\"\n        run: python -m pytest -q llm/rag/tests/llm/test_llm_regression.py\n\n      - name: Run Ollama smoke test (RELEASE.md Step 4)\n        env:\n          RUN_OLLAMA_TESTS: \"1\"\n        run: python -m pytest -q llm/rag/tests/llm/test_ollama_smoke.py\n\n  release:\n    name: Create GitHub release\n    runs-on: ubuntu-latest\n    needs:\n      - workflow-lint\n      - python-tests\n      - python-quality\n      - dependency-security\n      - node-security\n      - docker-images\n      - image-security\n      - compose-validate\n      - llm-regression\n    if: github.event_name == 'push' && startsWith(github.ref, 'refs/tags/v')\n    permissions:\n      contents: write\n    steps:\n      - name: Normalize image owner\n        id: prep\n        shell: bash\n        run: echo \"owner=${GITHUB_REPOSITORY_OWNER,,}\" >> \"$GITHUB_OUTPUT\"\n\n      - name: Create release\n        uses: softprops/action-gh-release@v2\n        with:\n          generate_release_notes: true\n          body: |\n            Container images published for this release:\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.APP_IMAGE_NAME }}:${{ github.ref_name }}`\n            - `ghcr.io/${{ steps.prep.outputs.owner }}/${{ env.API_IMAGE_NAME }}:${{ github.ref_name }}`\n"}}, ".env.prod.example": {"ec1475e17980d641a85def6a3ff6a74418102c40": {"t": 1785747131.2514699, "n": 3740, "h": "0e1c369e11e28be1", "c": "# =============================================================================\n# .env.prod.example — Production server environment template\n# =============================================================================\n# Copy to .env.prod on your Hetzner server and fill in every CHANGE_ME value.\n#\n#   cp .env.prod.example .env.prod\n#   chmod 600 .env.prod\n#\n# Variables managed by Docker Compose or shell (not this file):\n#   REDIS_URL         — set in docker-compose.prod.yml environment: block\n#   DOMAIN            — exported in the shell before running compose\n#   GHCR_IMAGE        — exported in the shell / set by CI via DEPLOY_IMAGE\n# =============================================================================\n\n# ── Runtime mode (required) ──────────────────────────────────────────────────\nSECA_ENV=prod\n\n# ── API authentication (required) ────────────────────────────────────────────\n# Generate: openssl rand -hex 32\nSECA_API_KEY=CHANGE_ME\n\n# ── Session / JWT secret (required, >= 32 chars) ─────────────────────────────\n# Generate: python3 -c \"import secrets; print(secrets.token_hex(32))\"\nSECRET_KEY=CHANGE_ME\n\n# ── CORS (required) ──────────────────────────────────────────────────────────\n# Comma-separated list of allowed origins (no trailing slash).\nCORS_ALLOWED_ORIGINS=https://cereveon.com\n\n# ── Database (required) ──────────────────────────────────────────────────────\nDATABASE_URL=postgresql://chesscoach:CHANGE_ME@db:5432/chesscoach\nPOSTGRES_DB=chesscoach\nPOSTGRES_USER=chesscoach\nPOSTGRES_PASSWORD=CHANGE_ME\n\n# ── LLM / DeepSeek (required for coaching) ───────────────────────────────────\n# Sign up at https://platform.deepseek.com, create an API key, paste here.\n# Without it, every /chat call falls back to the deterministic template\n# (chat_pipeline.py:557 catches LLMConfigError and degrades gracefully).\nCOACH_DEEPSEEK_API_KEY=CHANGE_ME\n\n# Override only if you point at a self-hosted OpenAI-compatible gateway.\n# COACH_DEEPSEEK_API_BASE=https://api.deepseek.com\n\n# Default ``deepseek-chat`` is V3.  See .env.example for cost/latency\n# guidance.  ``deepseek-reasoner`` (chain-of-thought) costs ~4x for\n# similar quality on chess prose; not recommended.\n# COACH_DEEPSEEK_MODEL=deepseek-chat\n\n# ── Optional overrides ───────────────────────────────────────────────────────\n# STOCKFISH_PATH=/usr/games/stockfish\n# UVICORN_WORKERS=2\n\n# ── Billing — Google Play purchase verification ──────────────────────────────\n# Powers POST /billing/google/verify (llm/seca/billing/router.py), which\n# verifies a Play purchase token server-side and flips the player to Pro.\n# Leave these UNSET pre-launch: with any one missing the endpoint answers\n# 503 \"purchase verification not configured\" — safe and loud, never\n# fake-success.  To go live, create a Play Console service account with\n# the Android Publisher + financial-data grant, enable the Android\n# Publisher API, download its JSON key, and set all three.  The package\n# MUST match the shipped applicationId (com.cereveon.myapp).  The private\n# key keeps the literal \\n escapes from the JSON key file — the server\n# normalises them.  Full runbook: docs/BILLING_GO_LIVE.md.\n# GOOGLE_PLAY_PACKAGE_NAME=com.cereveon.myapp\n# GOOGLE_PLAY_SA_EMAIL=CHANGE_ME@CHANGE_ME.iam.gserviceaccount.com\n# GOOGLE_PLAY_SA_PRIVATE_KEY=CHANGE_ME\n\n# ── Entitlements — freemium enforcement ──────────────────────────────────────\n# Master switch for the free/Pro usage limits (llm/seca/entitlements).\n# The paywall only has a PURPOSE when this is on, so production runs it\n# ON.  Unset/empty/unrecognised = no metering (everyone effectively\n# unlimited).  Recognised enable values: true / 1 / yes.  Limits live in\n# llm/seca/entitlements/service.py::LIMITS.\nSECA_ENTITLEMENTS_ENFORCED=true\n"}, "6e3cbd20e251e6169cbfe1d6d917ffc25c237cd7": {"t": 1785749408.9177153, "n": 3740, "h": "0e1c369e11e28be1", "c": "# =============================================================================\n# .env.prod.example — Production server environment template\n# =============================================================================\n# Copy to .env.prod on your Hetzner server and fill in every CHANGE_ME value.\n#\n#   cp .env.prod.example .env.prod\n#   chmod 600 .env.prod\n#\n# Variables managed by Docker Compose or shell (not this file):\n#   REDIS_URL         — set in docker-compose.prod.yml environment: block\n#   DOMAIN            — exported in the shell before running compose\n#   GHCR_IMAGE        — exported in the shell / set by CI via DEPLOY_IMAGE\n# =============================================================================\n\n# ── Runtime mode (required) ──────────────────────────────────────────────────\nSECA_ENV=prod\n\n# ── API authentication (required) ────────────────────────────────────────────\n# Generate: openssl rand -hex 32\nSECA_API_KEY=CHANGE_ME\n\n# ── Session / JWT secret (required, >= 32 chars) ─────────────────────────────\n# Generate: python3 -c \"import secrets; print(secrets.token_hex(32))\"\nSECRET_KEY=CHANGE_ME\n\n# ── CORS (required) ──────────────────────────────────────────────────────────\n# Comma-separated list of allowed origins (no trailing slash).\nCORS_ALLOWED_ORIGINS=https://cereveon.com\n\n# ── Database (required) ──────────────────────────────────────────────────────\nDATABASE_URL=postgresql://chesscoach:CHANGE_ME@db:5432/chesscoach\nPOSTGRES_DB=chesscoach\nPOSTGRES_USER=chesscoach\nPOSTGRES_PASSWORD=CHANGE_ME\n\n# ── LLM / DeepSeek (required for coaching) ───────────────────────────────────\n# Sign up at https://platform.deepseek.com, create an API key, paste here.\n# Without it, every /chat call falls back to the deterministic template\n# (chat_pipeline.py:557 catches LLMConfigError and degrades gracefully).\nCOACH_DEEPSEEK_API_KEY=CHANGE_ME\n\n# Override only if you point at a self-hosted OpenAI-compatible gateway.\n# COACH_DEEPSEEK_API_BASE=https://api.deepseek.com\n\n# Default ``deepseek-chat`` is V3.  See .env.example for cost/latency\n# guidance.  ``deepseek-reasoner`` (chain-of-thought) costs ~4x for\n# similar quality on chess prose; not recommended.\n# COACH_DEEPSEEK_MODEL=deepseek-chat\n\n# ── Optional overrides ───────────────────────────────────────────────────────\n# STOCKFISH_PATH=/usr/games/stockfish\n# UVICORN_WORKERS=2\n\n# ── Billing — Google Play purchase verification ──────────────────────────────\n# Powers POST /billing/google/verify (llm/seca/billing/router.py), which\n# verifies a Play purchase token server-side and flips the player to Pro.\n# Leave these UNSET pre-launch: with any one missing the endpoint answers\n# 503 \"purchase verification not configured\" — safe and loud, never\n# fake-success.  To go live, create a Play Console service account with\n# the Android Publisher + financial-data grant, enable the Android\n# Publisher API, download its JSON key, and set all three.  The package\n# MUST match the shipped applicationId (com.cereveon.myapp).  The private\n# key keeps the literal \\n escapes from the JSON key file — the server\n# normalises them.  Full runbook: docs/BILLING_GO_LIVE.md.\n# GOOGLE_PLAY_PACKAGE_NAME=com.cereveon.myapp\n# GOOGLE_PLAY_SA_EMAIL=CHANGE_ME@CHANGE_ME.iam.gserviceaccount.com\n# GOOGLE_PLAY_SA_PRIVATE_KEY=CHANGE_ME\n\n# ── Entitlements — freemium enforcement ──────────────────────────────────────\n# Master switch for the free/Pro usage limits (llm/seca/entitlements).\n# The paywall only has a PURPOSE when this is on, so production runs it\n# ON.  Unset/empty/unrecognised = no metering (everyone effectively\n# unlimited).  Recognised enable values: true / 1 / yes.  Limits live in\n# llm/seca/entitlements/service.py::LIMITS.\nSECA_ENTITLEMENTS_ENFORCED=true\n"}, "98890e4b3ac04488399746936f144fbdc07d3271": {"t": 1785749410.1593215, "n": 2362, "h": "c03a36fd284fd4d0", "c": "# =============================================================================\n# .env.prod.example — Production server environment template\n# =============================================================================\n# Copy to .env.prod on your Hetzner server and fill in every CHANGE_ME value.\n#\n#   cp .env.prod.example .env.prod\n#   chmod 600 .env.prod\n#\n# Variables managed by Docker Compose or shell (not this file):\n#   REDIS_URL         — set in docker-compose.prod.yml environment: block\n#   DOMAIN            — exported in the shell before running compose\n#   GHCR_IMAGE        — exported in the shell / set by CI via DEPLOY_IMAGE\n# =============================================================================\n\n# ── Runtime mode (required) ──────────────────────────────────────────────────\nSECA_ENV=prod\n\n# ── API authentication (required) ────────────────────────────────────────────\n# Generate: openssl rand -hex 32\nSECA_API_KEY=CHANGE_ME\n\n# ── Session / JWT secret (required, >= 32 chars) ─────────────────────────────\n# Generate: python3 -c \"import secrets; print(secrets.token_hex(32))\"\nSECRET_KEY=CHANGE_ME\n\n# ── CORS (required) ──────────────────────────────────────────────────────────\n# Comma-separated list of allowed origins (no trailing slash).\nCORS_ALLOWED_ORIGINS=https://cereveon.com\n\n# ── Database (required) ──────────────────────────────────────────────────────\nDATABASE_URL=postgresql://chesscoach:CHANGE_ME@db:5432/chesscoach\nPOSTGRES_DB=chesscoach\nPOSTGRES_USER=chesscoach\nPOSTGRES_PASSWORD=CHANGE_ME\n\n# ── LLM / DeepSeek (required for coaching) ───────────────────────────────────\n# Sign up at https://platform.deepseek.com, create an API key, paste here.\n# Without it, every /chat call falls back to the deterministic template\n# (chat_pipeline.py:557 catches LLMConfigError and degrades gracefully).\nCOACH_DEEPSEEK_API_KEY=CHANGE_ME\n\n# Override only if you point at a self-hosted OpenAI-compatible gateway.\n# COACH_DEEPSEEK_API_BASE=https://api.deepseek.com\n\n# Default ``deepseek-chat`` is V3.  See .env.example for cost/latency\n# guidance.  ``deepseek-reasoner`` (chain-of-thought) costs ~4x for\n# similar quality on chess prose; not recommended.\n# COACH_DEEPSEEK_MODEL=deepseek-chat\n\n# ── Optional overrides ───────────────────────────────────────────────────────\n# STOCKFISH_PATH=/usr/games/stockfish\n# UVICORN_WORKERS=2\n"}, "da949ca39283ce98c2f82be676880c8562d5226d": {"t": 1785749411.3774128, "n": 2213, "h": "c3587a5487ffb028", "c": "# =============================================================================\n# .env.prod.example — Production server environment template\n# =============================================================================\n# Copy to .env.prod on your Hetzner server and fill in every CHANGE_ME value.\n#\n#   cp .env.prod.example .env.prod\n#   chmod 600 .env.prod\n#\n# Variables managed by Docker Compose or shell (not this file):\n#   COACH_OLLAMA_URL  — set in docker-compose.prod.yml environment: block\n#   REDIS_URL         — set in docker-compose.prod.yml environment: block\n#   DOMAIN            — exported in the shell before running compose\n#   GHCR_IMAGE        — exported in the shell / set by CI via DEPLOY_IMAGE\n# =============================================================================\n\n# ── Runtime mode (required) ──────────────────────────────────────────────────\nSECA_ENV=prod\n\n# ── API authentication (required) ────────────────────────────────────────────\n# Generate: openssl rand -hex 32\nSECA_API_KEY=CHANGE_ME\n\n# ── Session / JWT secret (required, >= 32 chars) ─────────────────────────────\n# Generate: python3 -c \"import secrets; print(secrets.token_hex(32))\"\nSECRET_KEY=CHANGE_ME\n\n# ── CORS (required) ──────────────────────────────────────────────────────────\n# Comma-separated list of allowed origins (no trailing slash).\nCORS_ALLOWED_ORIGINS=https://cereveon.com\n\n# ── Database (required) ──────────────────────────────────────────────────────\nDATABASE_URL=postgresql://chesscoach:CHANGE_ME@db:5432/chesscoach\nPOSTGRES_DB=chesscoach\nPOSTGRES_USER=chesscoach\nPOSTGRES_PASSWORD=CHANGE_ME\n\n# ── Optional overrides ───────────────────────────────────────────────────────\n# STOCKFISH_PATH=/usr/games/stockfish\n#\n# Production model pin. Tags can be re-pushed; pin the digest so a rotation\n# upstream cannot silently change behaviour under the LLM regression\n# contract. Find the current digest with:\n#\n#     docker compose -f docker-compose.prod.yml exec ollama \\\n#         ollama show qwen2.5:7b-instruct-q2_K --modelfile | grep '^FROM'\n#\n# Then paste the resulting `model:tag@sha256:<hex>` reference below.\n# COACH_OLLAMA_MODEL=qwen2.5:7b-instruct-q2_K@sha256:CHANGE_ME\n#\n# UVICORN_WORKERS=2\n"}}}}