OpenCode
68a9071cf1
Local branch had diverged from origin/master (sibling commits on the same base). Rewrote local history linearly on top of origin/master, folding in all local content: exa/hot-platform discovery hunters, leaks ledger, nightly sweep orchestrator, updated .gitignore and skill docs, plus the latest hunt output and vault state. Remote-only files (vault keys, channel scripts) were restored rather than dropped, so the resulting tree is a full union of both sides.
1 line
9.5 KiB
JSON
1 line
9.5 KiB
JSON
{"repo": "adrianzevenster/data-analyst-agent", "files": {"docs/llm_reasoning.md": {"2859ad8bf727638c6fabf06f82726c10ffe93929": {"t": 1785747233.0873876, "n": 3015, "h": "30fa8cbc7b416d8d", "c": "# LLM Reasoning Layer\n\nThe API can optionally use a local/open-weight LLM through an OpenAI-compatible\n`/v1/chat/completions` endpoint. When enabled, the LLM performs two bounded jobs:\n\n1. Plan analytics tool calls from the user request, dataset schema, sample rows,\n and retrieved corpus context.\n2. Synthesize a concise analyst response from executed tool results and a\n deterministic dataset analysis context.\n\nThe existing deterministic planner remains the fallback when the LLM is disabled,\nunavailable, or returns unusable tool JSON.\n\nThe LLM does not compute over the full raw file directly. The API first builds a\nbounded, deterministic context containing schema, missingness, numeric summaries,\ncategorical top values, sample rows, and strongest numeric correlations. This\nkeeps the model grounded while still allowing dataset-specific interpretation.\n\nRetrieved corpus chunks are passed to the LLM as bounded `rag_context` entries\nwith source IDs, retrieval scores, and excerpts. The LLM uses this context for\ndomain guidance, while dataset facts and tool outputs remain the source of truth\nfor claims about the uploaded data.\n\n## Recommended Open-Weight Models\n\nUse an OpenAI-compatible serving layer such as vLLM, SGLang, Ollama, or LM Studio.\n\n| Use case | Recommended model | Why |\n| --- | --- | --- |\n| Best practical default | `Qwen/Qwen3-32B` | Strong reasoning, coding, and instruction following at a size that is realistic for a high-memory single GPU or quantized local deployment. |\n| Highest reasoning quality | `Qwen/Qwen3-235B-A22B` or a DeepSeek-R1 class model | Better deep reasoning for complex analytical planning, but requires multi-GPU serving or aggressive quantization. |\n| Tool-call reliability first | Mistral Small 3.2 | Official function-calling support and good mid-size agent behavior. |\n| Lightweight local testing | Qwen3 8B/14B or a distilled DeepSeek-R1 variant | Lower cost and easier local iteration, with less reliable planning on ambiguous data questions. |\n\nFor this application, `Qwen/Qwen3-32B` is the default because the agent needs a\nbalance of reasoning quality, JSON discipline, and practical deployability.\n\n## Configuration\n\nSet these variables in `.env` or the API container environment:\n\n```bash\nLLM_ENABLED=true\nLLM_BASE_URL=http://localhost:8000/v1\nLLM_API_KEY=\nLLM_MODEL=Qwen/Qwen3-32B\nLLM_TEMPERATURE=0.2\nLLM_TIMEOUT_SECONDS=120\nLLM_MAX_TOOL_CALLS=4\nLLM_ANALYSIS_SAMPLE_ROWS=5000\nLLM_ANALYSIS_PREVIEW_ROWS=5\nLLM_ANALYSIS_MAX_COLUMNS=80\nLLM_RAG_MAX_CHUNKS=6\nLLM_RAG_MAX_CHARS_PER_CHUNK=1200\n```\n\nWhen running through Docker Compose, `LLM_BASE_URL` defaults to\n`http://host.docker.internal:8000/v1` so the API container can call a model server\nrunning on the host.\n\n## Example vLLM Server\n\n```bash\nvllm serve Qwen/Qwen3-32B \\\n --host 0.0.0.0 \\\n --port 8000 \\\n --max-model-len 32768\n```\n\nFor Qwen3 thinking models, keep planning prompts strict and low temperature.\nThe agent validates returned tool names against the local registry before\nexecution.\n"}, "9a26c7afe1b237e6801623d5b7575f2fe7107f6a": {"t": 1785771564.057926, "n": 3015, "h": "30fa8cbc7b416d8d", "c": "# LLM Reasoning Layer\n\nThe API can optionally use a local/open-weight LLM through an OpenAI-compatible\n`/v1/chat/completions` endpoint. When enabled, the LLM performs two bounded jobs:\n\n1. Plan analytics tool calls from the user request, dataset schema, sample rows,\n and retrieved corpus context.\n2. Synthesize a concise analyst response from executed tool results and a\n deterministic dataset analysis context.\n\nThe existing deterministic planner remains the fallback when the LLM is disabled,\nunavailable, or returns unusable tool JSON.\n\nThe LLM does not compute over the full raw file directly. The API first builds a\nbounded, deterministic context containing schema, missingness, numeric summaries,\ncategorical top values, sample rows, and strongest numeric correlations. This\nkeeps the model grounded while still allowing dataset-specific interpretation.\n\nRetrieved corpus chunks are passed to the LLM as bounded `rag_context` entries\nwith source IDs, retrieval scores, and excerpts. The LLM uses this context for\ndomain guidance, while dataset facts and tool outputs remain the source of truth\nfor claims about the uploaded data.\n\n## Recommended Open-Weight Models\n\nUse an OpenAI-compatible serving layer such as vLLM, SGLang, Ollama, or LM Studio.\n\n| Use case | Recommended model | Why |\n| --- | --- | --- |\n| Best practical default | `Qwen/Qwen3-32B` | Strong reasoning, coding, and instruction following at a size that is realistic for a high-memory single GPU or quantized local deployment. |\n| Highest reasoning quality | `Qwen/Qwen3-235B-A22B` or a DeepSeek-R1 class model | Better deep reasoning for complex analytical planning, but requires multi-GPU serving or aggressive quantization. |\n| Tool-call reliability first | Mistral Small 3.2 | Official function-calling support and good mid-size agent behavior. |\n| Lightweight local testing | Qwen3 8B/14B or a distilled DeepSeek-R1 variant | Lower cost and easier local iteration, with less reliable planning on ambiguous data questions. |\n\nFor this application, `Qwen/Qwen3-32B` is the default because the agent needs a\nbalance of reasoning quality, JSON discipline, and practical deployability.\n\n## Configuration\n\nSet these variables in `.env` or the API container environment:\n\n```bash\nLLM_ENABLED=true\nLLM_BASE_URL=http://localhost:8000/v1\nLLM_API_KEY=\nLLM_MODEL=Qwen/Qwen3-32B\nLLM_TEMPERATURE=0.2\nLLM_TIMEOUT_SECONDS=120\nLLM_MAX_TOOL_CALLS=4\nLLM_ANALYSIS_SAMPLE_ROWS=5000\nLLM_ANALYSIS_PREVIEW_ROWS=5\nLLM_ANALYSIS_MAX_COLUMNS=80\nLLM_RAG_MAX_CHUNKS=6\nLLM_RAG_MAX_CHARS_PER_CHUNK=1200\n```\n\nWhen running through Docker Compose, `LLM_BASE_URL` defaults to\n`http://host.docker.internal:8000/v1` so the API container can call a model server\nrunning on the host.\n\n## Example vLLM Server\n\n```bash\nvllm serve Qwen/Qwen3-32B \\\n --host 0.0.0.0 \\\n --port 8000 \\\n --max-model-len 32768\n```\n\nFor Qwen3 thinking models, keep planning prompts strict and low temperature.\nThe agent validates returned tool names against the local registry before\nexecution.\n"}, "d00005add5c4e0aea5b6166b0186efe9ffc20c23": {"t": 1786068574.6621084, "n": 3015, "h": "30fa8cbc7b416d8d", "c": "# LLM Reasoning Layer\n\nThe API can optionally use a local/open-weight LLM through an OpenAI-compatible\n`/v1/chat/completions` endpoint. When enabled, the LLM performs two bounded jobs:\n\n1. Plan analytics tool calls from the user request, dataset schema, sample rows,\n and retrieved corpus context.\n2. Synthesize a concise analyst response from executed tool results and a\n deterministic dataset analysis context.\n\nThe existing deterministic planner remains the fallback when the LLM is disabled,\nunavailable, or returns unusable tool JSON.\n\nThe LLM does not compute over the full raw file directly. The API first builds a\nbounded, deterministic context containing schema, missingness, numeric summaries,\ncategorical top values, sample rows, and strongest numeric correlations. This\nkeeps the model grounded while still allowing dataset-specific interpretation.\n\nRetrieved corpus chunks are passed to the LLM as bounded `rag_context` entries\nwith source IDs, retrieval scores, and excerpts. The LLM uses this context for\ndomain guidance, while dataset facts and tool outputs remain the source of truth\nfor claims about the uploaded data.\n\n## Recommended Open-Weight Models\n\nUse an OpenAI-compatible serving layer such as vLLM, SGLang, Ollama, or LM Studio.\n\n| Use case | Recommended model | Why |\n| --- | --- | --- |\n| Best practical default | `Qwen/Qwen3-32B` | Strong reasoning, coding, and instruction following at a size that is realistic for a high-memory single GPU or quantized local deployment. |\n| Highest reasoning quality | `Qwen/Qwen3-235B-A22B` or a DeepSeek-R1 class model | Better deep reasoning for complex analytical planning, but requires multi-GPU serving or aggressive quantization. |\n| Tool-call reliability first | Mistral Small 3.2 | Official function-calling support and good mid-size agent behavior. |\n| Lightweight local testing | Qwen3 8B/14B or a distilled DeepSeek-R1 variant | Lower cost and easier local iteration, with less reliable planning on ambiguous data questions. |\n\nFor this application, `Qwen/Qwen3-32B` is the default because the agent needs a\nbalance of reasoning quality, JSON discipline, and practical deployability.\n\n## Configuration\n\nSet these variables in `.env` or the API container environment:\n\n```bash\nLLM_ENABLED=true\nLLM_BASE_URL=http://localhost:8000/v1\nLLM_API_KEY=\nLLM_MODEL=Qwen/Qwen3-32B\nLLM_TEMPERATURE=0.2\nLLM_TIMEOUT_SECONDS=120\nLLM_MAX_TOOL_CALLS=4\nLLM_ANALYSIS_SAMPLE_ROWS=5000\nLLM_ANALYSIS_PREVIEW_ROWS=5\nLLM_ANALYSIS_MAX_COLUMNS=80\nLLM_RAG_MAX_CHUNKS=6\nLLM_RAG_MAX_CHARS_PER_CHUNK=1200\n```\n\nWhen running through Docker Compose, `LLM_BASE_URL` defaults to\n`http://host.docker.internal:8000/v1` so the API container can call a model server\nrunning on the host.\n\n## Example vLLM Server\n\n```bash\nvllm serve Qwen/Qwen3-32B \\\n --host 0.0.0.0 \\\n --port 8000 \\\n --max-model-len 32768\n```\n\nFor Qwen3 thinking models, keep planning prompts strict and low temperature.\nThe agent validates returned tool names against the local registry before\nexecution.\n"}}}} |