1 line
47 KiB
JSON
1 line
47 KiB
JSON
{"repo": "alayra-systems-pvt-limited/alayra-nexus", "files": {"nexus-ui-plan.md": {"e17633ccf0f85776f9f0a472299bff7ff98e7813": {"t": 1785596021.6556487, "n": 46571, "h": "53d3f8cee535348e", "c": "# Alayra Nexus — Dashboard Plan (Phase 7)\n\n**Status:** LIVING DOCUMENT — this is the single source of truth for Phase 7 ordering.\n**Last reconciled:** 2026-07-14 (Session 47) · **Author:** Abbas · **Scope:** the Nexus operator dashboard.\n\n> **Why this file was rewritten.** The original (2026-07-11) plan ended with *\"Nothing in this plan has\n> been implemented\"* and numbered the phases one way; a product re-sequencing on 2026-07-12 numbered\n> them another way; and the code went a third way. Three sources drifted. This file now reconciles all\n> three and is authoritative. If a phase is not in §3, it is not scheduled.\n\nThe `kinetic-admin` app is **reference only** — a source of UX patterns — and is never modified.\n\n---\n\n## 0. THE BLOCKER: the new dashboard is not being served\n\n**`src/server.ts` serves `frontend/` (the old vanilla dashboard). `Dockerfile` copies `frontend/` into\nthe runtime image. `web/dist` is never served and never packaged.**\n\nEverything built in P7.1–P7.5 — Overview, the redesigned Nexus, model management, Analytics — is\ntherefore **invisible in every real deployment**, including the published container. It only appears\nwhen a developer runs the Vite dev server.\n\n**The cutover is blocked by a parity gap, not by laziness.** The old dashboard still owns two things\nthe new one does not:\n\n| Capability | Old `frontend/` | New `web/` |\n|---|---|---|\n| Settings (SSRF · guardrails · routing · cache · notifications · compliance · audit viewer) | ✅ working | ❌ placeholder |\n| Team keys | ✅ working | ❌ placeholder |\n| Models | ✅ own tab | ✅ *better* — folded into Nexus (P7.4b) |\n| Overview · Analytics · themes · pool/key/model editing | ❌ none | ✅ shipped |\n\nFlipping the static root today would take working configuration **away** from operators — a real\nregression. So the remaining phases are ordered to **close parity, then cut over.** The cutover is the\nmilestone that makes five phases of work real; nothing after it matters as much.\n\n---\n\n## 1. What is actually shipped (verified against git, 2026-07-14)\n\n| Phase | What landed | Commit |\n|---|---|---|\n| **P7.1** | Vite + Preact `web/` app; slate-glass tokens; **dark + light** theme; self-hosted fonts + charts (killed the CDN bug); component kit; app shell + router; 12-section IA | — |\n| **P7.2** | **Overview** landing: one `/admin/overview` aggregate, clickable stat cards, four 7-day charts, leaderboards, recent activity | — |\n| **P7.3** | **Nexus** (pools/keys/health + honest routing-rules view), **Models**, **Connect** | — |\n| **P7.4a** | Chart aliveness (per-metric colour, rich hover), Alayra logo wired, Nexus add-provider/add-key restored | `d5174af` |\n| **P7.4b** | **Models folded INTO Nexus** (tab removed); live model discovery (`fetch-models`); per-key RPM/TPM/max-users | `8f05a7f` |\n| **P7.4c** | Editable model details: capability-driven per-modality pricing (incl. realtime audio in/out) + bundled pricing catalog + auto-fill | `a102b6d` |\n| **P7.4d** | Edit provider / edit key dialogs; **Max Users enforced**; per-provider extra headers (Anthropic `anthropic-version`) | `bd12911` `4ff409b` `5c695e8` |\n| **P7.5** | **Analytics**: per-request outcome + latency + **cache-savings recording** (they were never written down), one aggregate endpoint, live Analytics page | `2d604ef` `7ca17db` `ed2c7b8` |\n\n**Sections live in `web/`:** Overview · Nexus · Connect · Analytics · Teams · Security · Caching · Logs · Settings.\n**Still placeholders:** Enterprise · Admin.\n\n---\n\n## 2. Information architecture — 12 sections\n\n**Models is gone** — folded into Nexus in P7.4b, where a pool now owns its own models, keys, limits\nand pricing. **Health was added in P7.12** (Abbas's call — server/provider health and future\nbenchmarks needed a home of their own; route `/status` because `/health` is the liveness probe).\n\nOverview · **Nexus** · Connect · Analytics · Teams · Enterprise · Security · Caching · **Health** ·\nLogs · Settings · Admin\n\n---\n\n## 3. Remaining phases — ordered by dependency, not preference\n\nThe first three phases exist to **reach parity so the cutover can happen**. Everything in them is\n\"surface a backend that is already built and tested\" — low risk, high visibility.\n\n### ~~P7.6 — Settings (sub-tabs) + Logs~~ ✅ **DONE** (`414baf5`)\nSettings is seven sub-tabs (Routing · Cache · Guardrails · Notifications · Network · Compliance ·\nAppearance), each loading and saving only what it owns; every control states its consequence. Logs is\nits own filterable, read-only section. **No backend change was needed** — every endpoint already\nexisted. Fixed two bugs found while building: the Toggle was silently dead (a `<label>` around a\n`<button>` re-dispatched the click, so every switch fired twice and no-oped), and panels re-seeded\nfrom a prop in an effect, which could clobber an in-progress edit and left \"unsaved changes\" stuck\nforever after a save.\n\n### ~~P7.6b — Dashboard polish + review fixes~~ ✅ **DONE**\nA finish pass over what P7.1–P7.6 shipped, no new features. Three visible defects fixed and verified\nlive in both themes: (1) **charts** rendered blurred and ~2× too tall — the SVG used a fixed\n`320×120` viewBox with `preserveAspectRatio=\"none\"`, so the browser stretched it ~1.9×; it now\nmeasures its container and renders 1:1 (a 120px-tall chart measures 120px, was 223px); (2) **table\nheaders** on right-aligned columns drifted left of their cells — a CSS specificity bug where\n`.table thead th` beat `.tRight`, fixed with `.table thead th.tRight`; (3) **dark-theme `<select>`\npopups** rendered white with black hover text because the `<option>` had no styling — now themed from\ntokens. Plus a review round on the edit dialogs: `PoolModels` remove no longer leaves buttons disabled\n(moved to `finally`, edit disabled during removal); `EditKeyDialog` stops treating a typed `0` as\n\"keep old\"; edit dialog Save buttons tied to their form via `form=`; `Modal` uses `aria-labelledby`;\nregistry id-collision suffix increments. Chart geometry locked by a regression test.\n\n### ~~P7.7 — Security + Caching~~ ✅ **DONE**\n**Security:** one home for 2FA/TOTP (enrol via manual secret + otpauth URI — no QR dependency —\nconfirm, one-time recovery codes, regenerate, disable), admin API tokens (mint owner/viewer,\nplaintext-once, revoke), the sign-in policy facts (session TTL / lockout), and a read-only network-egress\nsummary linking to Settings → Network (one editor, no duplicate). All backend already existed.\n**Caching:** the response-cache control **moved here** from Settings (so Settings went 7→6 tabs) and\nnow sits beside live stats (entries, 7-day hit rate + savings, scoped to a recent window for honesty)\nand a one-click purge behind an honest confirm. New backend: `GET /admin/cache/stats` +\n`POST /admin/cache/purge` (SCAN + UNLINK, non-blocking; purge is global by design — the namespace is\ninside the hashed key), plus `countKeys`/`deleteKeys` in a testable `lib/redisScan.ts`. The pre-existing\n`POST /admin/cache/flush` (registry-cache bust) was left untouched for parity.\n*Deferred to P7.13:* real viewer role-gating in the new dashboard. Today an owner-only action a viewer\nattempts is met with a plain \"your session is read-only\" message rather than being hidden — consistent\nwith how the rest of the redesigned console behaves, and cleanly fixed once the accounts primitive lands.\n\n### ~~P7.7c — CodeQL security-scan remediation~~ ✅ **DONE**\nTwelve CodeQL alerts from the P7.7 push, triaged against the code into three groups.\n**Fixed (7):** per-route rate limits (`lib/routeRateLimits.ts` tiers AUTH/ADMIN_WRITE/ADMIN_READ)\non the sensitive routes — login, TOTP confirm/disable, recovery-codes, SSO login/callback, key\nmetrics, team-key delete, cache flush — on top of the existing global abuse guard; a test proves the\nlimit engages (429). **Tightened (1):** the SSRF sink now fetches the validated `URL` object\n`assertSafeUrl` returns (the guard already blocked private/metadata/loopback and is tested).\n**Bonus:** recovery codes widened 40→64-bit. **Documented false positives (dismissed in GitHub UI):**\nsha256 on high-entropy tokens (session/API/team keys) and the constant-time-compare length-equaliser —\nbcrypt is the wrong tool there and would break the O(1) lookups; reasoning captured in code comments.\nNo schema change, nothing on the proxy hot path.\n\n**Follow-up (same day): CodeQL cannot see `@fastify/rate-limit` or the SSRF guard**, so the code fixes\nabove did not clear the alerts — it re-raised them at shifted line numbers, and manual dismissal is a\ntreadmill on an actively-edited codebase. Resolved durably by switching CodeQL from default to\n**advanced setup** (`.github/workflows/codeql.yml` + `.github/codeql/codeql-config.yml`) and filtering\nthe two rules that only ever false-positive here (`js/missing-rate-limiting`, `js/insufficient-password-hash`),\neach with a written justification. Every other query stays active — SSRF (`js/request-forgery`) included,\nwith its one finding dismissed by design. Requires a one-time GitHub toggle (disable default setup).\n\n### ~~P7.8 — Teams~~ ✅ **DONE** — *the last parity blocker cleared*\nTeams are now a first-class console section — the old dashboard only ever had a bare \"Team Keys\" list;\nthis is a genuine upgrade. Two sub-tabs: **Teams** (create/edit/delete, with budget cap + period,\nstatus, and per-period spend shown against the cap) and **Access keys** (create, assign to a team,\nreassign, copy, revoke). The BYOK fall-back flag is edited on the team form.\n\nThe headline fix: **`assignedTier` is wired into routing.** It used to be stored and silently ignored\n— a lie. A team's preferred tier now leads the model-first candidate ordering (`selectModels` takes a\n`preferredTier`, threaded from the team key through `discoverBestPool` on both the chat and non-chat\npaths), then the normal premium→standard→fast failover follows, so a preference never becomes an\noutage when that tier is momentarily exhausted. Unit-tested at the ordering layer.\n\nMembers + Org are deliberately deferred to P7.13 (they need the same accounts primitive as sub-admins;\nbuild it once).\n\n### ~~P7.9 — CUTOVER~~ ✅ **DONE** 🚩 *the milestone — the redesign now reaches users*\nThe gateway serves the redesigned dashboard (`web/dist`) instead of the old `frontend/`, which is\ndeleted. New this phase: a **SPA deep-link fallback** (`lib/spaFallback.ts` + a `setNotFoundHandler`\nin `server.ts`) so a refresh or bookmark on a client-side route (`/teams`, `/nexus`, `/admin` …)\nserves `index.html` and lets the client router resolve it — while API clients still get an honest\nJSON 404. The discriminator is the `Accept` header, not a route list, so it never rots as sections are\nadded. Static plugin now `wildcard: false`; Vite `base` switched `'./'`→`'/'` so assets resolve from\nthe root at any route depth. Docker builds `web/` in the build stage and copies only `web/dist` into\nthe runtime image. Verified: an `inject` integration test drives the real wiring (deep link → app,\nAPI path → JSON 404), and the built deep link serves 200 + absolute assets. **This is the phase that\nmakes P7.1–P7.8 real.** Small in code, largest in consequence.\n\n### ~~P7.9b — Sign-in screen~~ ✅ **DONE** — *the cutover's missing front door*\nThe cutover surfaced that the redesigned dashboard never rebuilt the old dashboard's login, so it was\nunreachable without a hand-set token. Backend auth (password → session token, TOTP, lockout) has\nexisted since Phase 6; this added only its screen. `web/src/pages/Login.tsx` (password, on-demand 2FA\ncode field, plain wrong-password / lockout messages); `App` gates on a session token and seeds it\nsynchronously so a signed-in reload never flashes login; a global 401 clears the token and fires\n`nx:unauthorized` → back to sign-in; the Topbar sign-out is wired. `api.ts` gained `login()` (off the\ngeneric path so a failed sign-in doesn't self-trigger the 401 handler) and `clearToken()`. Full basic\nidentity (sub-admins, first-run, recovery-key, device fingerprint) is still P7.13; this is just the\nfront door so the product is usable.\n\n### ~~P7.10 — Budgeting cascade + Teams restructure~~ ✅ **DONE**\nTeams became **three sub-tabs**: **Overview** (the team list + totals), **Access keys** (the global\nkey view, now filterable by team and free-text search), and **Team stats** — the new one. Team stats\nis a team selector (dropdown + search) and a period filter (today/7d/30d/90d) over a new read,\n`GET /admin/teams/:id/stats` (`services/teamStats.service.ts`): spend/requests/success-rate/tokens for\nthe viewing window, cost + request trends, busiest models (reusing Analytics' `ByModel`), and the\n**per-member breakdown** — one row per access key, click to expand into spend, share of team,\nrequests, tokens, cost/request and last-active. A **\"member\" is honestly a team's access key**: the\ngateway has no user identity, so a key *is* the seat, and an idle key is listed with zeros rather than\ndropped. **Share is a proportion, never an allocation** — the cap lives on the team and no per-member\ncap exists to report. Two windows coexist and are labelled: the period tabs pick the *viewing* window,\nwhile the budget card reports the team's *own* daily/weekly/monthly cycle, read exactly the way\nadmission reads it — blurring them would show a 7-day spend against a monthly cap.\n\nThe cascade's enforceable half: **`Team.overBudgetAction`** (migration `0013`, additive, default\n`'block'` = the historical behaviour, so no existing team changed). `block` = today's hard 429;\n`notify` = a soft cap that alerts but never blocks; `downgrade` = keep serving but pin routing to the\nfast/cheapest tier. Threaded through `checkTeamBudget` (which now returns `downgrade` alongside\n`allowed`) into **both** admission paths, where an over-budget downgraded team routes to `fast`\nregardless of its `assignedTier`. Unit-tested at the verdict layer and proven on the dispatch path\n(over-budget → served, not blocked, with `discoverBestPool` called with `fast`).\n\n*Deliberately not built, because the schema cannot back it honestly (see §5):* an **org-level or\npool-level budget parent** (no `Org` model exists, and pools are shared infrastructure, not spend\nowners), a **true per-tier % split** of one budget (needs two caps per team), **per-team email\nrecipients** (notification config is global), and **quarterly/annual** windows. Listed in the backlog\nrather than faked.\n\n### ~~P7.11 — Notifications bell + Branding~~ ✅ **DONE**\n**The finding that shaped the phase:** all five alert types were gated *at their call sites* by\n`notificationsArmed(event)` = `config.enabled && config.events[event]` — and `enabled` is **false by\ndefault**. So on a default install the alert message was never even built, and alerts were\nfire-and-forget email/webhook with no persistence at all. A bell wired to the old `notify()` would\nhave been permanently empty: a placeholder pretending to be a feature.\n\n**So recording was split from delivery.** Every raised alert is now recorded in the feed\n*regardless* of the email config (migration `0014`, new `Notification` table); `enabled` and the\nper-event flags now gate only whether an alert *also leaves the building*. `notify()` records first,\nthen decides on delivery; the four call sites no longer short-circuit. Delivery behaviour, coalescing\nand the release-on-failed-send are untouched — their tests pass unchanged. The feed takes its **own**\nRedis claim key (`nexus:notify:feed:`), separate from the send guard (`nexus:notify:sent:`), because\nthe send path *releases* its claim on a failed delivery to allow a retry — sharing one key would\nre-insert an entry already sitting in the bell. This reframes Settings → Notifications as *email*\nsettings, which lines up with the deferred note #2 below (that UI reorg is still deferred).\n\n**The bell** (`shell/NotificationsBell.tsx`) is a real dropdown: unread badge (counted over *all*\nunread, never just the page shown), entries with relative time, mark-one / mark-all read, close on\noutside-click or Escape, and a 60s poll (a socket would be overkill — alerts are coalesced to one per\nwindow per source). Selecting an alert marks it read and **jumps to the section that raised it** via\na pure `sectionFor()` map in `lib/notify.ts` (keyBanned/breakerOpened/tierExhausted → Nexus,\nadminLockout → Security, budgetThreshold → Teams) — an alert saying a key died is only useful if it\nlands you where you can replace it.\n\n**Branding** — company name + logo in the sidebar and on the sign-in screen. **No schema**: it is one\nsmall singleton blob, which is what `AppSettings` is for. The logo is stored as a **data URI**, never\na URL: this gateway self-hosts its assets (the Chart.js CDN was removed for exactly this reason), and\na remote logo would break air-gapped/strict-CSP deployments *and* leak a request to a third party on\nevery load of a public sign-in page. Validation is pure and tested (`lib/branding.ts`): PNG/JPEG/WEBP/SVG\nonly, ≤64KB, and a stored value that no longer validates is dropped rather than served. SVG is safe\nbecause the logo is only ever rendered through an `<img src=…>` (browsers treat that as secure static\nmode) — it must never be inlined into the DOM. The read is a **public `GET /branding`** (the login\nscreen has no session yet; a branded login page is public by definition); the write is\n`PUT /admin/branding`, owner-only and audited. A white-labelled sidebar keeps \"Alayra Nexus\" on the\nline beneath, so the console still says what it is.\n\n**Enhancements taken in this phase:** (1) **notification retention** — the feed is pruned by the\nexisting `runRetention()` job via a new `notificationRetentionDays` (default **30**, editable in\nSettings → Compliance, `0` = forever). Alerts are operational noise, not a record — the audit trail\ntestifies to what happened — so they default shorter than the 90-day audit/usage windows, and this is\nthe one growing table that would otherwise have had no ceiling. (2) Mark-all-read + unread-only\nfiltering. **Caught in live verification:** saving branding left the sidebar showing the old name\nuntil a page reload, because each reader holds its own fetch — fixed with a `nx:branding` event that\nevery `useBranding` listens for (the same idiom the session gate already uses for `nx:unauthorized`),\nand pinned by a regression test. Dead code removed: `notificationsArmed` had no callers left.\n\n### ~~P7.12 — Health section (Server health + the new Health IA home)~~ ✅ **DONE**\n**Abbas's IA call, adopted:** health metrics had no home in the 11-section IA — the phase said *what*\nbut never *where*. There is now a **Health section** (12th section, after Caching) with three tabs:\n**Server** (the real P7.12), **Providers** (read-only capacity summary reusing `getNexusOverview`,\nlinking to Nexus — one editor, no duplicate, same rule as Security→Network), and **Benchmarks**\n(honestly empty until built; it says what it will do rather than showing invented numbers).\n**Route is `/status`, NOT `/health`** — `GET /health` is the liveness probe, a JSON route the SPA\nfallback deliberately excludes, so a deep link to /health would render `{ok:true}` instead of the page.\n\n**The audit finding fixed:** `/health` was a hardcoded `{ok:true}` that never checked anything — a\nload balancer would call the gateway healthy with its database down. Two probes now, each answering\nthe question it is actually for: `/health` stays a cheap liveness ping (restarting cannot fix a dead\ndatabase; a liveness probe that checks dependencies turns every DB blip into a restart loop), and the\nnew **`GET /ready`** really probes Redis + Postgres and answers **503** with per-check detail when a\ndependency is down. Degraded-but-answering still says ready — pulling a slow gateway out of rotation\nturns a slowdown into an outage. Both exempt from the abuse rate limit.\n\n**What makes it real, not decoration:** a new in-memory **health sampler**\n(`services/healthSampler.service.ts` over the pure `lib/health.ts`) probes Redis (PING) and Postgres\n(SELECT 1) every 15s, reads the event-loop delay histogram (`monitorEventLoopDelay`, reset per tick)\nand process CPU/RSS/heap, and keeps **one hour in a ring buffer** (240 samples, a few KB). That\nbuffer feeds the latency **sparklines**, the **p50/p95/p99 chips**, and the **per-minute status\nstrip** (worst sample per minute; empty minutes render as grey gaps, and a fresh process says\n\"collecting — N of 240 samples\" instead of faking continuity). Redis detail parses `INFO`\n(memory/maxmemory, clients, ops/sec, hit rate, evictions, fragmentation); Postgres detail reads\n`pg_stat_activity`/`pg_stat_database`/`pg_database_size`/largest tables — every query independently\nguarded so a managed instance that refuses one view nulls that fact instead of blanking the panel.\n\n**Honesty rules baked in (keep them):** no Postgres \"disk free\" anywhere (unknowable from SQL); no\nRedis memory % when `maxmemory` is unset (no ceiling exists — prose instead of a gauge); the\ncontainer memory gauge reads the **cgroup limit** (v2 then v1), NEVER `os.totalmem()` (inside Docker\nthat is the HOST's RAM); heap gauge is against the V8 heap limit; a failed probe is a **null** point\nthat breaks the sparkline rather than drawing a reassuring zero; hit rates with no traffic are \"—\",\nnot 100%. Every status carries its word (Healthy/Degraded/Down, Pass/Slow/Fail) — never colour alone.\nThe **readiness checks table is `/ready` rendered** — measured vs threshold vs verdict — so ops and\nthe dashboard share one truth. `GET /admin/health/overview` (admin-guarded) is the page's single read.\n\n### ~~P7.13a — Accounts, roles, and first-run identity~~ ✅ **DONE**\n*Split from P7.13 by agreement (Abbas, 2026-07-17): the auth rewrite lands and is verified on its\nown, the way P7.9/P7.9b did, rather than in one blast radius with the UI gating.*\n\n**The finding: there was no user.** One shared `ADMIN_PASSWORD` in an environment variable\nauthenticated everyone, and everything else followed from it — the audit trail could only ever say\n\"someone with the password\", nobody could be added or removed without a redeploy, the second factor\nbelonged to the *gateway* rather than to a person, and \"viewer\" was unreachable for a human without\npasting a machine token into the password box. The dashboard also **threw away the role** the\ngateway returned at sign-in, which is why P7.7's viewer-gating was blocked on two things, not one.\n\n**The accounts primitive** (migration `0015`, additive): `AdminUser` (name/email/scrypt\nhash/role/status/per-user TOTP/recovery key/source) and `AdminInvite`. Recovery codes gained an\nowner; `AdminApiToken` gained `createdById`; `AuditLog` gained `actorId` + `actorName` —\n**deliberately no foreign key, and the name copied not joined**, because an audit record has to\noutlive the account it describes.\n\n**`ADMIN_PASSWORD` changed job** (Abbas's call: *\"close the door\"*). It claims the first owner\naccount and authorises the reset-wipe; it is refused as a sign-in the moment an owner exists. It\nlives in the deployer's `.env` and nowhere else, so it is proof you *installed* this gateway rather\nthan merely found the port — which is what defeats the first-run land-grab. **Nothing changes at\nupgrade:** until the gateway is claimed, sign-in behaves exactly as Phase 6 left it, and every Phase\n6 test still passes against that state on purpose. Claiming **carries an existing authenticator and\nunused recovery codes across**, then NULLS the original — a live secret must exist in one place.\n\n**Three roles** (owner / admin / viewer). Admin runs the gateway day to day; owner additionally\ndecides who it belongs to. All 55 owner-guarded routes re-gated: the default became owner-or-admin,\nwith an explicit owner-only list — people, invites, SSO, compliance, the master API key, **the SSRF\npolicy** (a control the party it constrains must not be able to edit) and **`POST /admin/settings`**\n(an arbitrary key/value setter — every owner-only setting at once wearing a generic coat; an admin\nholding it could set `NEXUS_API_KEY` to a value they already knew).\n\n**Sessions name their subject, not their authority.** `{uid}` in Redis; role and status are read\nfrom the account on every request. Baking the role in would leave a demoted admin with owner rights\nfor twelve hours and a suspended account signed in until its TTL ran out. Verified live: suspending\nsomeone killed their open session on the very next request.\n\n**The master API key is show-once** (Abbas's call). It was the only credential stored in plain text\nwhile team keys were hashed and provider keys encrypted. Converted at boot, not in SQL (hashing in a\nmigration would demand the pgcrypto extension of everyone's database): the existing key **keeps\nworking** and is printed one last time in that boot's log. Connect shows a mask and a Rotate button.\n\n**Also:** SSO now provisions a real account from the `email` claim (the claim's role applies only to\na *new* account — otherwise an IdP's groups could silently re-promote someone an owner had demoted);\ninvites are hand-over links, not emails (delivery is off by default — an email-only invite would be\nthe P7.11 trap again); passwords use **scrypt from Node's own crypto** (memory-hard, no new\ndependency, no native build, async so it cannot spike the event-loop lag the Health tab measures).\nThe CodeQL justification was rewritten: it claimed in writing that no password is ever stored, which\nthis phase ended — the exclusion stays (it fires on our high-entropy token digests, where it is\nwrong) but the guarantee is now held by our own test asserting the stored hash IS scrypt.\n\n**Bug found by live testing, fixed in-phase:** Connect printed\n`…/v1/v1/chat/completions` — the gateway sends an origin-plus-`/v1` base and the endpoint paths\ncarry their own `/v1`. Wrong since the P7.9 cutover, on the one page whose entire job is \"copy this\nand it works\", and the unit test never caught it because its fixture used a base URL with no `/v1`,\nwhich no real deployment sends. Fixed, with the fixture made realistic.\n\nGreen on both packages: backend **638 tests** (+81), web **122** (+11), lint/typecheck/build 0,\n0 vulnerabilities. Verified end-to-end against a real Postgres + Redis: migration applied, the\nplaintext key converted and the old key still authenticating (400 = past auth) while a wrong key is\n401, claim refused without the env secret, password refused after claiming, invite ignoring an\ninjected email/role, an admin refused on all five owner-only routes, the last-owner invariant\nholding on the one path that reaches it (an owner **API token**, which has no self to check), and\nthe audit trail naming Abbas and Liaqat — with `(nobody)` exactly where there is genuinely nobody.\n\n### ~~P7.13a-fix — four defects found by clicking~~ ✅ **DONE** *(2026-07-17)*\nAbbas found these in the running dashboard, not in the suite. Cause of the first, and of why it\nsurvived: **`web/src/api.ts` had no test file at all** — the seam every page reaches the gateway\nthrough, and every page test mocks it away.\n\n- **Every bodyless request was rejected before its route ran.** The client sent\n `Content-Type: application/json` unconditionally; Fastify answers `FST_ERR_CTP_EMPTY_JSON_BODY`\n when that header arrives with no body. Broke **11 call sites**: setting up two-factor, enabling\n and disabling a provider key, marking notifications read (twice), and all 7 `DELETE`s — including\n **Remove person** and **Revoke invite**, i.e. the offboarding P7.13a had just shipped. The\n notification calls end in `.catch(() => {})`, so they had been failing silently. Fix: send the\n header only when a body accompanies it. Guarded by `web/src/api.test.ts` (new, 13 tests).\n- **Framework errors printed raw JSON at the user.** `{\"statusCode\":400,\"code\":\"FST_ERR_CTP…\"}`\n rendered verbatim on the Security page. `errorText()` now extracts the sentence — reading\n `message` for Fastify's shape and `error` for ours, told apart by `statusCode`, which Fastify\n always sends and our routes never do. (The first cut of this read `error` first and would have\n shown \"Bad Request\" for every framework failure; the test caught it.)\n- **Overview → \"Active models\" linked to `/models`**, removed when models were folded into Nexus in\n P7.4b. Right number, click landed on \"Not found\". Now `/nexus`. Asserting the href would not have\n caught it — the href was what someone typed. The new test checks **every** Overview link against\n `SECTIONS`, and was confirmed to fail on the real bug before being kept.\n- **Enterprise's placeholder named a phase, and the name was wrong.** \"Built in P7.8\" — P7.8 shipped\n as Teams; Enterprise was never in it. Now reads \"Coming soon\". Our phase numbers are a promise in\n a private language, which is worse than no promise.\n\n**Live-verified against a source build + real Postgres/Redis:** the old client behaviour reproduces\nthe exact error from Abbas's screenshot; the new one returns a secret; two-factor setup completes in\na real browser; Enterprise reads \"Coming soon\"; every Overview link resolves. Green both packages —\nbackend 638, web **137** (+14).\n\n> **Trap that cost an hour, worth writing down:** `docker compose up -d --build` **does not build\n> this repo.** `docker-compose.yml` pulls the *published* image (that is the README's promise to\n> users); building from source needs `-f docker-compose.dev.yml`. The stack on `localhost:3000` was\n> the last release — **five** migrations against our fifteen — so it authenticated `ADMIN_PASSWORD`\n> against pre-accounts code and mislabelled the authenticator `admin`. Both \"findings\" evaporated on\n> a real build. **Check what is actually running before believing what it tells you.**\n\n### P7.13b — Sessions & devices, viewer gating, danger zone\nSessions & devices (browser, IP, last-seen; revoke one, sign out everywhere) — the honest version of\nthe plan's \"device fingerprint\", which cannot be an auth factor because any client can forge it.\nReal role-gating across every section (**P7.7's deferral, now unblocked** — the dashboard stores the\nrole as of 7.13a). The double-confirmed reset-wipe (standing decision #2), which cannot leave an\naudit record of itself: it destroys the table it would write to, so it logs to stdout and says so.\n\n*In the gap, an admin who clicks an owner-only action gets a clear 403 naming what they have and\nwhat is needed — which is exactly today's behaviour, so the split regresses nothing.*\n\n### P7.14 — Public URL truth *(small; the `https` problem)*\n**The question:** deployed on `xyz.com`, does Connect still say `localhost`? **No — that part\nalready works.** The base URL is built from how the client actually reached us\n(`X-Forwarded-Proto`/`X-Forwarded-Host`, else `Host`). Verified live: with a proxy's headers it\nreturns `https://xyz.com/v1`.\n\n**The real hole is the scheme.** A reverse proxy terminates TLS and speaks plain HTTP to the\ngateway, so the gateway *cannot observe* that the visitor used HTTPS — it only knows if the proxy\nsays so. A proxy that forwards `Host` but forgets `X-Forwarded-Proto` makes Connect print\n`http://xyz.com/v1`: verified live, and the worst kind of wrong, because the page looks fine and\nthe operator copies it. Then either their client is redirected (works, silently), or it is refused\non a strict deployment, or an HTTPS dashboard fetches an HTTP URL and the browser blocks it as mixed\ncontent. We cannot fix someone's proxy — but we must not *print a confident lie* because of one.\n\nThree layers, cheapest first:\n- **`PUBLIC_URL` env var wins over everything.** The deployer states the address; no inference can\n contradict it. This is the only source that is true by construction, and it is what a locked-down\n deployment wants anyway.\n- **The dashboard cross-checks.** It is served by the gateway from the same origin, so the browser's\n own address bar is ground truth — it cannot be lied to about the scheme it used. If\n `window.location` disagrees with what the gateway believes, say so on the page rather than hand\n over a URL we have reason to doubt.\n- **Document the two proxy headers** in the README, next to the deployment section.\n\n### ~~P7.15 — End-to-end proof~~ ✅ **DONE** *(2026-07-17 — built out of order, before 13b, on purpose: the remaining phases get built with the net already under them)*\n\n**What exists now: `e2e/` — 42 tests against the compiled `dist/server.js`, a real PostgreSQL, a\nreal Redis, and a real Chromium.** Nothing under test is imported; everything is reached over a\nsocket, exactly as a deployment is. New CI job `e2e` (postgres+redis services, chromium) alongside\nthe unit jobs — NOT behind them: the unit tests mock the database away, so services there would\nprove nothing. Full local run ≈ 35s including both builds: `cd e2e && npx playwright test`.\n\n**Architecture, and the reasoning that shaped it:**\n- **Two gateway stacks** (`nexus_e2e_api` on :3100 / redis DB 1, `nexus_e2e_ui` on :3101 / DB 2) —\n a gateway can be CLAIMED exactly once per database, and both the API story and the browser story\n must prove that flow from its unclaimed beginning. Isolated from a developer's live gateway by\n construction (own DBs; the redis-flush script refuses DB 0 outright).\n- **Every run re-proves the migrations**: setup is `prisma migrate reset` onto an empty database —\n all fifteen apply from scratch, every time.\n- **A mock upstream provider with a LEDGER** (`/__requests`). The ledger is how a negative is\n proven: a rate-limited request must appear as a refusal at our door AND an absence upstream. A\n limit that rejects the caller but still bills the provider passes any test that only reads the\n response. Admitted via `SSRF_ALLOWLIST` — a real deployment knob, not a test backdoor.\n- **A hand-rolled 20-line RFC-6238 TOTP** plays the authenticator app — nothing under test may\n verify itself, so `src/lib/totp.ts` is not imported.\n- **No retries, one worker, serial stories.** A retry policy is a flakiness amnesty. The suites\n are stories (claim → sign in → invite → suspend → remove), each step's precondition the previous\n step's outcome. The lockout spec is pinned LAST (`99-`) because the lockout it triggers bars the\n source address for everything after it.\n- **Latency asserted nowhere**, per the plan: runner timings are noise, and an ignored gate is\n worse than none.\n\n**What it caught before it was even finished — four findings in one day:**\n1. **The local build was serving GHOST ROUTES.** `require('./routes/admin')` resolved to\n `dist/routes/admin.js` — a compiled FOSSIL of the pre-split monolithic router that tsc never\n cleans up, and Node prefers a file over a directory. Every local `node dist/server.js` since the\n admin split ran a Phase-6-era admin API; Docker never saw it (images build from a clean copy),\n which is why the container kept working while local boots 404'd `/admin/login`. Fix: `npm run\n build` now removes `dist/` first. Diagnosed via `onRoute` instrumentation after printRoutes\n string-matching lied.\n2. **The provider auth header was assembled wrong in six places.** `` `${authPrefix} ${key}` ``\n turned an operator-typed `Bearer ` (trailing space — the natural way to type a prefix) into\n `Bearer <key>`, and a deliberately empty prefix into ` <key>`. Providers refuse both, and the\n header is never displayed anywhere, so the pool just failed auth with nothing to debug. Caught\n by the ledger — the only place the assembled header is visible. Fix: `providerAuthHeader()` in\n `lib/providerHeaders.ts`, ONE place, used by proxyDispatch, completionsProxy, and all four\n nexus.service probes (+6 unit tests).\n3. **The Logs page never showed the names 7.13a promised.** `actorName` was in the database and on\n the wire, but the dashboard's `AuditEntry` type never learned the field, so the Actor column\n showed only roles. \"The log records names\" was true in the database and invisible on screen.\n Caught by the browser suite looking for a removed person's name (+1 web test).\n4. **Confirmed fail-fast is real**: a gateway with an unreachable database refuses to start with\n the actionable startup message — the earlier \"half-booted server\" scare was finding #1 wearing\n a costume.\n\n**Coverage:** first-run claim (unclaimed → Phase-6 sign-in intact → wrong/weak refusals → claim →\nrecovery key → master password dead → double-claim 409) · invites (flat 404s, spent tokens,\ninjected email/role ignored) · roles at the wire (admin writes, owner-only 403s with the sentence) ·\nper-user TOTP with computed codes · suspension killing a live session on the next request ·\nlast-owner/self-change refusals · removal revoking access while the trail keeps the name · source\nlockout (right password refused too) · the full proxy path (credential isolation proven upstream,\nteam keys, revocation, budget 429 before upstream, RPM exhausted at exactly its budget → 503 +\nRetry-After, TPM likewise, rotation killing the old key) · and the browser story through real\nChromium in two contexts (claim screen, recovery-key-once, the exact 2FA flow from the P7.13a-fix\nscreenshot, invite link handover, suspension mid-session, removal + the trail surviving).\n\n*Deliberate scope cuts:* audit-pipeline latency is tolerated with a bounded reload loop, not raced\n(batched inserts are the design); streaming completions not yet exercised (the mock answers\nnon-streamed); SSO needs a fake IdP — later, if it earns it.\n\n### Backlog (unscheduled — pull in when they earn it)\n- **Team stats: Comparison mode** *(Abbas, 2026-07-16)* — a \"Compare\" button in Team stats to select\n 2 teams or 2 members and show them side-by-side / overlaid. Deferred by agreement as a future\n enhancement once the base Team stats tab is in use; the stats endpoint already returns everything a\n comparison needs, so this is a UI-only addition (two fetches, one layout).\n- **Budget cascade — the levels the schema cannot back yet** *(from P7.10)*: an **Org/pool budget\n parent** (blocked on the missing `Org` model — pools are shared infrastructure, so a pool budget\n would need a new owner concept), a **per-tier % split** of a team's budget (needs two caps per team\n and a per-tier spend counter), **per-team email recipients** (notification config is global today),\n and **quarterly/annual** budget windows (the counter TTLs and `periodKey` assume ≤ monthly).\n- **Benchmarks + branded PDF** — run tests on demand, clean report, downloadable. Its home now\n exists: the Health → Benchmarks tab (P7.12), which states this plan and stays honestly empty\n until the feature is real.\n- **First-class provider presets** — xAI, Azure, Bedrock, Vertex, HuggingFace, Together. They work\n **today** only via the generic `custom` OpenAI-compatible path (baseUrl + auth + modelIdPath); there\n are no presets. Purely a convenience layer.\n- **Enterprise/Org section** — no `Org` model exists (`Team` was shaped to take an `orgId` later).\n- **Per-team analytics** — the Analytics aggregate is global; a team filter is a small extension.\n\n---\n\n## Further changes notes (post-completion polish — DEFER until every phase is done)\n\n> Captured 2026-07-16 from Abbas's first real click-through of the live product (Docker). These are\n> **cosmetic/UX refinements, not functional gaps** — the decision is to finish all functional phases\n> first, then return to these so we ship a complete product before polishing. Do NOT action these mid-phase.\n\n1. **Content does not fill the width — too much empty space on every tab.** The main content column is\n effectively capped/narrow, so pages (Overview especially) leave a large blank area on the right and\n look under-filled on a wide screen. Audit the page/shell max-width and padding; let content breathe\n into the full width (or introduce a sensible max with better use of the space — e.g. more columns).\n Applies to **all tabs**, so it is a shell/layout fix, not a per-page one.\n2. **Notifications tab is cluttered — split it up.** (a) Give **Email** its own sub-tab (Resend API\n key + From address + recipients), separate from the event list. (b) The events section should sit\n under a single master **\"Enable email notifications\"** toggle; when off, the whole event list is\n disabled/hidden. Right now the master \"Send alerts\" switch and the per-event switches read as one\n undifferentiated cluster. Reorganise into: master toggle → (when on) event checklist + Email config\n in its own tab.\n3. **\"Add provider pool\" dialog — clarity, not new fields.** Users don't know what SLUG / Auth header /\n Auth prefix / Model ID path / Extra headers mean. Add short inline hints on each (one line each,\n like the \"url-safe id\" hint already on SLUG). **Also make it discoverable that model *capabilities*\n (chat/embedding/image/…) are chosen on the *models inside a pool*, not on the pool** — e.g. a note in\n the dialog (\"you'll pick each model's capabilities after creating the pool\") and/or a prompt to add a\n model right after a pool is created. This is a known-correct design (pool = credentials; model =\n capabilities/tier/pricing) that simply isn't obvious in the UI.\n\n> Captured 2026-07-17 from Abbas's click-through of the LIVE Railway deployment (screenshots in\n> C:\\Users\\KineticLLC\\Desktop\\issues). Same rule: polish batch, do NOT action mid-phase. The two\n> functional items from that session (key-test showing \"Failed\" on success; /v1/v1 tolerance) were\n> fixed immediately and are NOT in this list.\n\n4. **First-run claim screen — make it world-class.** (a) An **account type selector** (Personal /\n Organization): Personal → First name + Last name fields; Organization → Company name field.\n (Server: still one `name` string — compose \"First Last\" or company name client-side; no schema\n change.) (b) **Eye toggles** on every password input (admin password + new password) so what was\n typed can be verified. (c) A **Confirm password** field that must match before submit enables.\n (d) A larger, more premium card — this screen is the product's first impression.\n5. **Recovery key display (claim success screen).** The hyphenated key wraps to two lines and looks\n squeezed — render on ONE line (smaller font / wider box / `white-space: nowrap`). **Copy button\n must animate** (flash/checkmark on the key itself). Add a **Download** button: a `.txt` containing\n the heading `Alayra Nexus API key:` and the key below it.\n6. **2FA enrolment must show a real QR code.** Today only the base32 setup key + otpauth URI are\n shown with copy buttons — the text says \"scan\" but there is nothing to scan; typing a 32-char\n secret into a phone is hostile. Render the otpauth URI as a QR (needs a tiny QR lib in web/ or a\n hand-rolled encoder — backend stays zero-dep, this is browser-side only).\n7. **TOTP recovery codes — download button.** Alongside \"Copy all\": download a file titled\n `Alayra Nexus TOTP Recovery code` (heading) + the ten codes, filename `nexus-recovery.txt`.\n8. **Model picker in Add/Edit key — opt-IN, not opt-out — AND harvest pricing at fetch time.**\n Fetching OpenRouter returns 339 models rendered as delete-me chips; the user must close 338 to\n keep one. Replace with a searchable list (search icon + filter box) where each row has a select\n control that animates to a clear tick; **only ticked models are added**. Bulk actions (select\n all / none). **Pricing:** \"Auto-fill\" only matches the bundled static catalog (majors), so a\n niche id like `kwaipilot/kat-coder-pro-v2.5` gets \"No catalog match\" — but OpenRouter's own\n `/models` response CARRIES per-model pricing (and context length). Harvest it during the fetch\n and pre-fill each selected model's prices/context automatically; the catalog stays the fallback\n for providers whose /models is bare (OpenAI/Anthropic). This is what makes cost tracking real\n without hand-typing 339 prices. **Selected-models display (Abbas, 2026-07-18): selections\n append into a collapsed summary strip — when more than ~2 are picked it shows up to 4 in the\n box and turns scrollable above that, expanding on click — so a big selection never sprawls\n the dialog.**\n9. **Teams → Access keys table.** (a) The NAME/KEY/TEAM/CREATED header row visually merges with the\n card header above — needs spacing/divider. (b) Copy button gives no feedback — every copy button\n in the product should animate + show \"Copied\". (c) **A \"connection card\" per key**: one click\n opens a card with the (revealable) full key + copy, the gateway base URL + copy, and the models\n URL — so an admin doesn't visit three tabs (Connect for URL, Teams for key) to onboard one user.\n10. **Copy feedback everywhere.** Audit every Copy control (recovery key, setup key, otpauth, invite\n link, team keys, Connect URLs): all must give animated \"Copied\" confirmation. One shared\n component (`CopyButton`) — several ad-hoc implementations exist.\n\n---\n\n## 4. Standing decisions (do not re-litigate)\n\n1. **Rendering: Vite + Preact → static build.** Fastify keeps serving static files; single-container\n deployment unchanged. *(2026-07-11)*\n2. **First-run reset = full database wipe.** Loud, double-confirmed, documented. An attacker who knows\n the trick gains nothing. *(2026-07-11)*\n3. **Teams before Budgeting.** A budget cascade needs teams that exist in the UI. *(2026-07-14)*\n4. **Parity before cutover.** Never remove working operator capability to ship a prettier shell.\n *(2026-07-14)*\n5. **Models stays folded into Nexus.** A pool owns its models, keys, limits and pricing. *(P7.4b)*\n6. **`kinetic-admin` is reference only.** Never modified.\n\n## 5. Known lies in the product to fix as we pass them\n\n- ~~**`Team.assignedTier` is stored and silently ignored by routing.**~~ ✅ Fixed in P7.8 — it now\n biases the model-first candidate ordering (preferred tier first, normal failover after).\n- ~~**There is no user.**~~ ✅ Fixed in P7.13a — one shared `ADMIN_PASSWORD` authenticated everyone,\n so the audit trail could only say \"someone with the password\" and nobody could be offboarded.\n Accounts, three roles, and per-person second factors now; the trail names people.\n- ~~**The dashboard throws away the role it is given at sign-in.**~~ ✅ Fixed in P7.13a — `login()`\n kept the token and dropped the role, which is why viewer-gating was never possible client-side.\n- **No `Org` model.** The Enterprise section cannot be honest until one exists or the scope shrinks.\n\n---\n\n*Every phase ends with the standing green gate — lint / typecheck / test / build / audit all clean on\n**both** the gateway and the dashboard — plus a `nexus-changes.md` entry, a commit, and a push.*\n"}}}} |