{"repo": "austin-holdheide/searxng-ai-summary-plugin", "files": {"README.md": {"c7926fe07952eb49ec0a0f84174fc053a7f4bee5": {"t": 1785674377.3232417, "n": 10010, "h": "cffd3e304819ca49", "c": "# SearXNG AI Summary Plugin\n\n![SearXNG AI Summary Plugin](images/Ai%20Summary.png)\n\nAdds a streaming AI summary above search results using any OpenAI-compatible endpoint — just like Google's AI Overview and DuckDuckGo's Search Assist. Built for self-hosted SearXNG with Docker.\n\n---\n\n## Features\n\n- **Instant results** — search results load at full speed, summary streams in after\n- **Typewriter effect** — text appears character by character as the LLM generates it\n- **\"More\" button** — expands into a structured deep-dive powered by a smarter model\n- **Progressive rendering** — sections appear as they complete, not all at once\n- **Code blocks** — commands render with syntax highlighting and a Copy button\n- **Generating indicator** — spinning loader shows while the More panel is still generating\n- **Auto-injected** — no template files need editing, ever\n- **Dark/light theme** — follows SearXNG's theme automatically\n- **Any OpenAI-compatible backend** — LM Studio, Ollama, OpenAI, Groq, and more\n\n---\n\n\n## How It Works\n\n### The Async Flow\n\n```\nUser types query\n      │\n      ▼\nSearXNG queries engines ───────────────── Results appear instantly (no wait)\n      │\n      ▼\nBrowser loads ai_summary.js             (auto-injected by the plugin)\n      │\n      ▼\nJS creates summary box with blinking cursor above results\n      │\n      ▼\nJS reads result snippets from the page DOM\n      │\n      ▼\nJS opens SSE stream → POST /ai_summary\n      │\n      ▼\nFlask streams tokens from fast model ──── Characters appear one by one\n      │\n      ▼\nStream ends → \"More ▾\" button appears\n      │\n      ▼ (user clicks More)\nJS opens SSE stream → POST /ai_summary_more\n      │\n      ├── overview arrives   → renders paragraph\n      ├── section 1 closes   → renders heading + bullets/code blocks\n      ├── section 2 closes   → renders heading + bullets/code blocks\n      ├── follow_up closes   → renders \"Explore More\" questions\n      └── stream ends        → generating indicator disappears\n```\n\n### Why Two Settings Blocks\n\nSearXNG's `PluginCfg` dataclass only accepts `active: true/false`. Putting any\nother key under a plugin entry crashes SearXNG on startup:\n\n```\nTypeError: PluginCfg.__init__() got an unexpected keyword argument 'base_url'\n```\n\nThe fix is two separate blocks in `settings.yml`:\n\n```yaml\n# Block 1 — inside plugins: (ONLY active: allowed here)\nplugins:\n  searx.plugins.ai_summary.SXNGPlugin:\n    active: true\n\n# Block 2 — new top-level key (all LLM config goes here)\nai_summary:\n  base_url: \"http://127.0.0.1:1234/v1\"\n  model: \"google/gemma-3-1b\"\n  model_more: \"openai/gpt-oss-20b\"\n  ...\n```\n\nThe plugin reads Block 2 via `get_setting(\"ai_summary.base_url\")`.\n\n### How the Script Is Injected\n\nThe plugin's `init(app)` method runs once at Flask startup and registers:\n\n1. **`POST /ai_summary`** — streaming SSE endpoint for the compact summary\n2. **`POST /ai_summary_more`** — streaming SSE endpoint for the More panel\n3. **`after_request` hook** — appends `<script src=\"ai_summary.js\">` to every\n   HTML page that contains `id=\"results\"`. No template editing ever needed.\n\n### How Streaming Works\n\nBoth endpoints use `stream: true` in the LLM request and return\n`text/event-stream` (SSE). The browser reads each chunk using the Fetch\n`ReadableStream` API.\n\n**Compact summary** — characters are pushed into a queue and drained at a\nspeed that matches the LLM: fast when tokens arrive in bursts, slow when\nthe model pauses. The blinking cursor shows while generation is ongoing.\n\n**More panel** — raw JSON streams in. A progressive parser scans the\nincomplete buffer on every chunk using balanced-brace counting and regex\nmatching to extract completed pieces and render them immediately:\n- `\"overview\": \"...\"` → paragraph rendered\n- Each completed `{...}` in `sections` → section rendered with code blocks\n- `\"follow_up\": [...]` → questions rendered\n\n---\n\n## File Layout\n\n```\nsearxng-ai-summary/\n├── docker-compose.yml     ← start here\n├── ai_summary.py          ← plugin: Flask routes + script injection\n├── ai_summary.js          ← frontend: streaming loader + progressive renderer\n└── searxng/\n    └── settings.yml       ← all configuration lives here\n```\n\n---\n\n## Setup\n\n### Prerequisites\n\n- Docker Desktop installed and running\n- LM Studio (or any OpenAI-compatible API) running on your network\n- At least one model loaded and ready\n\n### Step 1 — Get the files\n\nYour folder must look exactly like this:\n\n```\nyour-folder/\n├── docker-compose.yml\n├── ai_summary.py\n├── ai_summary.js\n└── searxng/\n    └── settings.yml\n```\n\n### Step 2 — Edit `searxng/settings.yml`\n\n**Enable the plugin** — add this inside the `plugins:` section:\n\n```yaml\n  searx.plugins.ai_summary.SXNGPlugin:\n    active: true\n```\n\n**Configure the LLM** — add this as a new top-level key (not inside `plugins:`):\nMake sure it matchs the api endponit of you choosing that are relevent. This one it confurged for [LM Studio](https://lmstudio.ai/) local server.\n\n```yaml\nai_summary:\n  base_url: \"http://127.0.0.1:1234/v1\"   # your LM Studio IP:port\n  api_key:  \"lm-studio\"                       # any string for local providers\n  model:       \"google/gemma-3-1b\"            # fast model — compact summary\n  model_more:  \"openai/gpt-oss-20b\"           # smart model — More panel\n  max_results: 5\n  max_tokens:  300\n  max_tokens_more: 800\n  timeout: 60\n```\n\n**Find your IP address** (Windows):\n```powershell\nipconfig\n# Look for IPv4 Address under your active network adapter\n```\n\n**Find your model name** — open LM Studio → Developer tab → copy the\nmodel identifier shown under \"API Model Identifier\".\n\n**Set a secret key**:\n```yaml\nserver:\n  secret_key: \"replace-this-with-something-long-and-random\"\n```\n\n### Step 3 — Start\n\n```powershell\ncd C:\\path\\to\\your-folder\ndocker compose up -d\n```\n\n### Step 4 — Open SearXNG\n\n```\nhttp://localhost:8080\n```\n\nSearch for anything. Results load immediately. The AI summary types itself\nin above the results. Click **More** for the structured deep-dive.\n\n---\n\n## Configuration Reference\n\nAll settings live in `searxng/settings.yml` under the `ai_summary:` key.\n\n| Key | Description |\n|-----|-------------|\n| `base_url` | OpenAI-compatible API root URL (no trailing slash) |\n| `api_key` | API key — any non-empty string for local providers |\n| `model` | Fast model used for the compact summary |\n| `model_more` | Smart model used for the More panel |\n| `max_results` | Result snippets sent to the LLM (default: 5) |\n| `max_tokens` | Max tokens for compact summary (default: 300) |\n| `max_tokens_more` | Max tokens for More panel (default: 800) |\n| `timeout` | Request timeout in seconds (default: 60) |\n| `system_prompt` | System prompt for compact summary |\n| `system_prompt_more` | System prompt for More panel — must instruct the model to return JSON |\n\n### Supported Backends\n\n| Provider | `base_url` | `api_key` |\n|----------|-----------|-----------|\n| LM Studio | `http://YOUR_IP:1234/v1` | `lm-studio` |\n| Ollama | `http://YOUR_IP:11434/v1` | `ollama` |\n| OpenAI | `https://api.openai.com/v1` | `sk-...` |\n| Groq | `https://api.groq.com/openai/v1` | `gsk_...` |\n| Together AI | `https://api.together.xyz/v1` | your key |\n| vLLM | `http://YOUR_IP:8000/v1` | any string |\n\n### Recommended Models\n\n| Model | Speed | Quality | Best for |\n|-------|-------|---------|----------|\n| `google/gemma-3-1b-it` | ⚡⚡⚡⚡ | ★★★ | `model` — fastest compact summary |\n| `google/gemma-3-4b-it` | ⚡⚡⚡ | ★★★★ | `model` — best speed/quality balance |\n| `microsoft/phi-4-mini-instruct` | ⚡⚡⚡ | ★★★★ | `model` — great instruction following |\n| `qwen/qwen2.5-7b-instruct` | ⚡⚡ | ★★★★★ | `model_more` — best structured output |\n| `mistralai/mistral-7b-instruct` | ⚡⚡ | ★★★★ | `model_more` — reliable JSON output |\n| `openai/gpt-oss-20b` | ⚡ | ★★★★★ | `model_more` — highest quality |\n\nAlways use `-it` or `-instruct` variants. The More panel requires a model\nthat reliably returns valid JSON — larger models do this better.\n\n---\n\n## After Any Change\n\n```powershell\ndocker compose restart searxng\n```\n\nChanges to `settings.yml`, `ai_summary.py`, or `ai_summary.js` all take\neffect after a restart. No rebuild needed — all files are volume-mounted.\n\n---\n\n## Troubleshooting\n\n```powershell\n# Watch live logs\ndocker compose logs -f searxng\n\n# Filter for plugin messages only\ndocker compose logs searxng | findstr ai_summary\n```\n\n| Symptom | Cause | Fix |\n|---------|-------|-----|\n| `PluginCfg unexpected keyword` | LLM settings inside `plugins:` block | Move them to the separate `ai_summary:` top-level block |\n| SearXNG won't start | YAML indentation error | Check that all plugin keys have exactly 2 spaces indent |\n| `Connection refused` | LM Studio server not started | Open LM Studio → Developer → flip Status toggle to ON |\n| No summary, no error | Plugin not loaded | Check `active: true` is set and `ai_summary.py` is volume-mounted |\n| Summary never appears | LLM timeout | Increase `timeout` or use a smaller model |\n| More panel shows raw JSON | Old JS file cached | Hard refresh (`Ctrl+Shift+R`) or restart container |\n| More panel errors | Smart model too slow | Increase `timeout`, reduce `max_tokens_more`, or use a faster model |\n| HTTP 401/403 | Wrong API key | Fix `api_key` in settings.yml |\n| HTTP 404 | Wrong URL or model name | Fix `base_url` or `model` / `model_more` |\n| Summary has markdown | Model ignoring system prompt | Switch to an `-instruct` or `-it` model variant |\n\n### Test your LLM directly\n\n```powershell\n@'\n{\n  \"model\": \"YOUR_MODEL_NAME\",\n  \"max_tokens\": 150,\n  \"stream\": false,\n  \"messages\": [\n    {\"role\": \"system\", \"content\": \"Summarize in 2 sentences. No markdown.\"},\n    {\"role\": \"user\",   \"content\": \"What is Apache web server?\"}\n  ]\n}\n'@ | Out-File -FilePath test.json -Encoding utf8\n\ncurl.exe -X POST http://YOUR_IP:1234/v1/chat/completions `\n  -H \"Content-Type: application/json\" `\n  -d \"@test.json\"\n```\n\nA working response has `choices[0].message.content` with plain text. If this\nworks but the plugin doesn't, check the Docker logs for the exact error.\n\n---\n\n## License\n\nMIT\n"}}}}