1 line
4.2 KiB
JSON
1 line
4.2 KiB
JSON
{"repo": "bagadir/eidon-swebench", "files": {"README.md": {"8c3133ff1c7d5cacb2f2be7e4eed9dacd4363751": {"t": 1785746596.4309294, "n": 1935, "h": "a0cee7423c7ef41a", "c": "# Eidon × DeepSeek-R1 on SWE-bench Verified\n\n**Eidon** is a local-first repository intelligence engine. This repo runs it against the [SWE-bench Verified](https://www.swebench.com/) benchmark (500 real GitHub issues) using DeepSeek's thinking-mode model.\n\n## How it works\n\n**4-stage pipeline per task:**\n\n```\nStage 1 — Encode eidon analyze --fresh → .eidon/encoding (L0–L3 graph)\nStage 2 — Localize deepseek-chat → top-5 files most likely to fix the bug\nStage 3 — Patch deepseek-reasoner → think step-by-step → unified diff\nStage 4 — Repair deepseek-reasoner → fix hunk-offset errors, re-apply\n```\n\nEidon's `encoding` gives the model a **compressed L0–L3 architecture map** of the repo (purpose → interface → usage → full source) before it writes a single line of diff. This is the key differentiator vs. agents that search files at random.\n\n## Key choices\n\n| Decision | Value | Why |\n|----------|-------|-----|\n| Localize | `deepseek-chat` | Fast, structured JSON, no reasoning overhead |\n| Patch | `deepseek-reasoner` | Thinking mode — explicit chain-of-thought on root cause |\n| Repair | `deepseek-reasoner` | Reasons about why the hunk failed |\n| Token budget | 32K (Eidon encoding) | Full architecture context |\n| Patch tokens | 8K | Reasoner needs room for thinking tokens |\n\n## Running locally\n\n```bash\nnpm install -g eidoncore\neidon activate \"$EIDON_LICENSE_KEY\"\npip install -r swebench/requirements.txt\nexport DEEPSEEK_API_KEY=sk-...\npython swebench/eidon_agent.py --tasks 10\n```\n\n## Running on GitHub Actions (500 tasks, 5 shards)\n\n1. Fork this repo\n2. Add secrets: `DEEPSEEK_API_KEY`, `EIDON_LICENSE_KEY`\n3. Push — the workflow triggers automatically\n\n**Estimated cost:** ~$7–10 for all 500 tasks at $0.28/1M input, $0.42/1M output.\n\n## Submitting\n\n```bash\npip install sb-cli\nsb-cli submit --predictions predictions.json --split verified\n```\n\n## Leaderboard tag\n\n`eidon-deepseek-reasoner`\n"}, "2e82d1b36268842ee9220194e164c3ac7aceae4c": {"t": 1785766175.5827298, "n": 1935, "h": "a0cee7423c7ef41a", "c": "# Eidon × DeepSeek-R1 on SWE-bench Verified\n\n**Eidon** is a local-first repository intelligence engine. This repo runs it against the [SWE-bench Verified](https://www.swebench.com/) benchmark (500 real GitHub issues) using DeepSeek's thinking-mode model.\n\n## How it works\n\n**4-stage pipeline per task:**\n\n```\nStage 1 — Encode eidon analyze --fresh → .eidon/encoding (L0–L3 graph)\nStage 2 — Localize deepseek-chat → top-5 files most likely to fix the bug\nStage 3 — Patch deepseek-reasoner → think step-by-step → unified diff\nStage 4 — Repair deepseek-reasoner → fix hunk-offset errors, re-apply\n```\n\nEidon's `encoding` gives the model a **compressed L0–L3 architecture map** of the repo (purpose → interface → usage → full source) before it writes a single line of diff. This is the key differentiator vs. agents that search files at random.\n\n## Key choices\n\n| Decision | Value | Why |\n|----------|-------|-----|\n| Localize | `deepseek-chat` | Fast, structured JSON, no reasoning overhead |\n| Patch | `deepseek-reasoner` | Thinking mode — explicit chain-of-thought on root cause |\n| Repair | `deepseek-reasoner` | Reasons about why the hunk failed |\n| Token budget | 32K (Eidon encoding) | Full architecture context |\n| Patch tokens | 8K | Reasoner needs room for thinking tokens |\n\n## Running locally\n\n```bash\nnpm install -g eidoncore\neidon activate \"$EIDON_LICENSE_KEY\"\npip install -r swebench/requirements.txt\nexport DEEPSEEK_API_KEY=sk-...\npython swebench/eidon_agent.py --tasks 10\n```\n\n## Running on GitHub Actions (500 tasks, 5 shards)\n\n1. Fork this repo\n2. Add secrets: `DEEPSEEK_API_KEY`, `EIDON_LICENSE_KEY`\n3. Push — the workflow triggers automatically\n\n**Estimated cost:** ~$7–10 for all 500 tasks at $0.28/1M input, $0.42/1M output.\n\n## Submitting\n\n```bash\npip install sb-cli\nsb-cli submit --predictions predictions.json --split verified\n```\n\n## Leaderboard tag\n\n`eidon-deepseek-reasoner`\n"}}}} |