1 line
8.4 KiB
JSON
1 line
8.4 KiB
JSON
{"repo": "almondsun/nutshellm", "files": {"README.md": {"c874874911103240a8bda4ec34a254d1188b264f": {"t": 1785596423.4420974, "n": 8126, "h": "24df79907c237ba5", "c": "<div align=\"center\">\n\n<img src=\"frontend/public/brand/nuto-mark.svg\" width=\"88\" alt=\"Nuto, the nutsheLLM acorn mascot\">\n\n# nutsheLLM\n\n**Your context, in a nutsheLLM.**\n\n[](https://github.com/Paritok-official/paritok-4b-v1)\n[](https://nutshellm-production.up.railway.app)\n[](https://youtu.be/5b_JzvC0T-o)\n[](LICENSE)\n\n</div>\n\nnutsheLLM is a quality-aware context optimization workbench for AI applications and\nagents. Nuto, its acorn guide, helps make the safety story memorable: trim the\nshell, guard the kernel, and prove the answer. The application compresses eligible context with\n[Paritok](https://github.com/Paritok-official/paritok-4b-v1), checks whether exact\nfacts and task performance survived, and automatically retries with safer context\nwhen they did not.\n\nThe core question is not only “how many tokens did we remove?” It is “how many\ntokens can we remove while preserving useful task performance?”\n\n## What it proves\n\n- **Fair baseline comparison.** The task model, parameters, and instructions are\n identical; only context changes.\n- **Explicit context decisions.** Every segment is classified as immutable,\n compressible, or disposable.\n- **Exact fact preservation.** Paths, identifiers, signatures, errors, timestamps,\n measurements, stack frames, URLs, and quoted literals must survive byte-for-byte.\n- **Real fallback.** nutsheLLM tries PariTok `L2`, then `L1`, then `L0`, and finally\n restores the original context.\n- **Honest accounting.** PariTok savings, deterministic duplicate removal, task\n input, evaluation overhead, estimated spend, and latency are reported separately.\n\n## Final live result\n\nThe final three-scenario suite achieved **52.2% median input saving with a 100%\ndeterministic pass rate (3/3)**. Individual savings were 52.2%, 0%, and 71.58%.\nThe zero-savings checkout result is retained because the safety contract takes\npriority over the headline number. See the\n[final benchmark](docs/final-benchmark.md) and\n[structured evidence](examples/benchmark-summary.json).\n\n## Architecture\n\n```text\nReact workbench\n │ POST /api/v1/runs\n ▼\nFastAPI + temporary SQLite job queue\n │\n ├─ classify segments and lock critical facts\n ├─ call Paritok hosted /compress\n ├─ restore any segment that loses an immutable span\n ├─ call configured OpenAI-compatible task model\n ├─ validate quality and retry at safer levels\n └─ persist content-free aggregate metrics\n```\n\nThe production artifact is one Docker image. FastAPI serves the compiled React\nworkbench, runs one recoverable background worker, and stores short-lived results in\nSQLite on a mounted volume. See [Architecture](docs/architecture.md) and\n[Evaluation methodology](docs/evaluation.md).\n\n## Curated demonstrations\n\n1. **Code debugging:** find a retry-cache key bug while preserving exact files,\n symbols, and test evidence.\n2. **Incident response:** retain a checkout failure’s service, start time, peak error\n rate, exact exception, and triggering deployment.\n3. **Tool-output analysis:** remove repeated worker health output without losing the\n single disk-pressure anomaly.\n\nCurated answers use deterministic fact validators. Custom tasks use exact invariant\nchecks and can optionally enable a blind semantic judge. If no reliable validator is\navailable, the UI says `unverified`; it does not claim success.\n\n## Local development\n\nRequirements: Python 3.11+, Node.js 22+, a PariTok hosted API key, and an\nOpenAI-Chat-Completions-compatible model endpoint.\n\n```bash\npython -m venv .venv\nsource .venv/bin/activate\npython -m pip install -e \".[dev]\"\ncd frontend && npm ci && cd ..\n```\n\nSet runtime values in your shell or secret manager. `config.example` lists every\nsupported name; it contains placeholders only.\n\n```bash\nexport PARITOK_API_KEY=\"...\"\nexport TASK_MODEL_API_KEY=\"your-Gemini-API-key\"\nexport TASK_MODEL_BASE_URL=\"https://generativelanguage.googleapis.com/v1beta/openai\"\nexport TASK_MODEL=\"gemini-3.5-flash-lite\"\nexport SESSION_SIGNING_SECRET=\"$(openssl rand -hex 32)\"\n```\n\nRun the API and frontend in separate terminals:\n\n```bash\nmake api\nmake web\n```\n\nOpen `http://127.0.0.1:5173`. API documentation is available at\n`http://127.0.0.1:8000/api/docs` outside production.\n\n## Docker\n\n```bash\ndocker build -t nutshellm .\ndocker run --rm -p 8000:8000 \\\n -v nutshellm-data:/app/data \\\n -e PARITOK_API_KEY \\\n -e TASK_MODEL_API_KEY \\\n -e SESSION_SIGNING_SECRET \\\n nutshellm\n```\n\nFor a production public demo, use the same image:\n\n```bash\ndocker build -t nutshellm .\n```\n\nThen configure `ENVIRONMENT=production`, both `TURNSTILE_SITE_KEY` and\n`TURNSTILE_SECRET_KEY`, strict quotas, and explicit model prices. The public site\nkey is delivered to the frontend at runtime; it is not baked into the image.\n`/readyz` fails closed if required production settings are missing. See\n[Configuration](docs/configuration.md).\n\n## Task optimization API\n\nStart a curated comparison:\n\n```bash\ncurl -X POST http://127.0.0.1:8000/api/v1/runs \\\n -H \"Content-Type: application/json\" \\\n -c cookies.txt \\\n -d '{\"scenario_id\":\"checkout-incident\",\"mode\":\"compare\"}'\n```\n\nPoll the returned private status URL with the same anonymous-session cookie:\n\n```bash\ncurl -b cookies.txt http://127.0.0.1:8000/api/v1/runs/RUN_ID\n```\n\nCustom input accepts a task and up to 12 typed context segments:\n\n```json\n{\n \"task\": \"What caused the failure?\",\n \"mode\": \"compare\",\n \"segments\": [\n {\n \"id\": \"logs\",\n \"kind\": \"log_output\",\n \"source\": \"api.log\",\n \"content\": \"...\"\n }\n ],\n \"turnstile_token\": \"required for every production run\"\n}\n```\n\nSupported kinds are `file_read`, `log_output`, `tool_result`, `history`,\n`documentation`, and `other`. Provider URLs and model names are deployment\nconfiguration, never user input.\n\n## Security and privacy\n\n- Provider and PariTok keys remain backend-only and are never returned or logged.\n- Custom content and outputs exist only in the temporary job record and are deleted\n after `RESULT_TTL_SECONDS` (one hour by default).\n- Aggregate metrics contain no task, context, or output text.\n- Run results are bound to a signed anonymous session, not exposed as public links.\n- Every production run requires Turnstile, independent per-IP and per-session\n limits, a global run limit,\n and a global estimated-spend cap.\n- Model-generated code is rendered as text and is never executed.\n- Request size, segment count, upstream model, retries, timeouts, and provider hosts\n are bounded server-side.\n\n**Operational action:** the PariTok key previously written in local project notes\nmust be rotated before deployment. This repository contains no copy of that key.\n\n## Validation\n\n```bash\nmake test\nmake lint\nmake typecheck\nmake build\nmake docker\n```\n\nCI runs Python tests and lint, frontend lint/type-check/build, and a Docker build.\nLive provider smoke tests are intentionally secret-gated and must not run for\nuntrusted pull requests.\n\n## Limits\n\n- v1 supports one environment-configured OpenAI-compatible task-model endpoint.\n- It is a task optimization API, not a transparent Chat Completions proxy.\n- Arbitrary repositories and generated patches are not executed.\n- SQLite and the internal worker require one application replica.\n- Token counts use provider-reported usage when available and otherwise are marked\n as estimates.\n- Custom task quality is `unverified` unless the optional semantic judge is enabled.\n\n## Submission\n\nThe public deployment, measured benchmark export, screenshots, demo recording, and\nDevpost fields are operational release tasks tracked in\n[the submission checklist](docs/submission-checklist.md). Do not publish a savings\nclaim until it comes from a completed live run.\n\n## License\n\nApache License 2.0. Built with [Paritok](https://github.com/Paritok-official/paritok-4b-v1).\n"}}}} |