1 line
11 KiB
JSON
1 line
11 KiB
JSON
{"repo": "austin-holdheide/searxng-ai-summary-plugin", "files": {"README.md": {"c7926fe07952eb49ec0a0f84174fc053a7f4bee5": {"t": 1785674377.3232417, "n": 10010, "h": "cffd3e304819ca49", "c": "# SearXNG AI Summary Plugin\n\n\n\nAdds a streaming AI summary above search results using any OpenAI-compatible endpoint — just like Google's AI Overview and DuckDuckGo's Search Assist. Built for self-hosted SearXNG with Docker.\n\n---\n\n## Features\n\n- **Instant results** — search results load at full speed, summary streams in after\n- **Typewriter effect** — text appears character by character as the LLM generates it\n- **\"More\" button** — expands into a structured deep-dive powered by a smarter model\n- **Progressive rendering** — sections appear as they complete, not all at once\n- **Code blocks** — commands render with syntax highlighting and a Copy button\n- **Generating indicator** — spinning loader shows while the More panel is still generating\n- **Auto-injected** — no template files need editing, ever\n- **Dark/light theme** — follows SearXNG's theme automatically\n- **Any OpenAI-compatible backend** — LM Studio, Ollama, OpenAI, Groq, and more\n\n---\n\n\n## How It Works\n\n### The Async Flow\n\n```\nUser types query\n │\n ▼\nSearXNG queries engines ───────────────── Results appear instantly (no wait)\n │\n ▼\nBrowser loads ai_summary.js (auto-injected by the plugin)\n │\n ▼\nJS creates summary box with blinking cursor above results\n │\n ▼\nJS reads result snippets from the page DOM\n │\n ▼\nJS opens SSE stream → POST /ai_summary\n │\n ▼\nFlask streams tokens from fast model ──── Characters appear one by one\n │\n ▼\nStream ends → \"More ▾\" button appears\n │\n ▼ (user clicks More)\nJS opens SSE stream → POST /ai_summary_more\n │\n ├── overview arrives → renders paragraph\n ├── section 1 closes → renders heading + bullets/code blocks\n ├── section 2 closes → renders heading + bullets/code blocks\n ├── follow_up closes → renders \"Explore More\" questions\n └── stream ends → generating indicator disappears\n```\n\n### Why Two Settings Blocks\n\nSearXNG's `PluginCfg` dataclass only accepts `active: true/false`. Putting any\nother key under a plugin entry crashes SearXNG on startup:\n\n```\nTypeError: PluginCfg.__init__() got an unexpected keyword argument 'base_url'\n```\n\nThe fix is two separate blocks in `settings.yml`:\n\n```yaml\n# Block 1 — inside plugins: (ONLY active: allowed here)\nplugins:\n searx.plugins.ai_summary.SXNGPlugin:\n active: true\n\n# Block 2 — new top-level key (all LLM config goes here)\nai_summary:\n base_url: \"http://127.0.0.1:1234/v1\"\n model: \"google/gemma-3-1b\"\n model_more: \"openai/gpt-oss-20b\"\n ...\n```\n\nThe plugin reads Block 2 via `get_setting(\"ai_summary.base_url\")`.\n\n### How the Script Is Injected\n\nThe plugin's `init(app)` method runs once at Flask startup and registers:\n\n1. **`POST /ai_summary`** — streaming SSE endpoint for the compact summary\n2. **`POST /ai_summary_more`** — streaming SSE endpoint for the More panel\n3. **`after_request` hook** — appends `<script src=\"ai_summary.js\">` to every\n HTML page that contains `id=\"results\"`. No template editing ever needed.\n\n### How Streaming Works\n\nBoth endpoints use `stream: true` in the LLM request and return\n`text/event-stream` (SSE). The browser reads each chunk using the Fetch\n`ReadableStream` API.\n\n**Compact summary** — characters are pushed into a queue and drained at a\nspeed that matches the LLM: fast when tokens arrive in bursts, slow when\nthe model pauses. The blinking cursor shows while generation is ongoing.\n\n**More panel** — raw JSON streams in. A progressive parser scans the\nincomplete buffer on every chunk using balanced-brace counting and regex\nmatching to extract completed pieces and render them immediately:\n- `\"overview\": \"...\"` → paragraph rendered\n- Each completed `{...}` in `sections` → section rendered with code blocks\n- `\"follow_up\": [...]` → questions rendered\n\n---\n\n## File Layout\n\n```\nsearxng-ai-summary/\n├── docker-compose.yml ← start here\n├── ai_summary.py ← plugin: Flask routes + script injection\n├── ai_summary.js ← frontend: streaming loader + progressive renderer\n└── searxng/\n └── settings.yml ← all configuration lives here\n```\n\n---\n\n## Setup\n\n### Prerequisites\n\n- Docker Desktop installed and running\n- LM Studio (or any OpenAI-compatible API) running on your network\n- At least one model loaded and ready\n\n### Step 1 — Get the files\n\nYour folder must look exactly like this:\n\n```\nyour-folder/\n├── docker-compose.yml\n├── ai_summary.py\n├── ai_summary.js\n└── searxng/\n └── settings.yml\n```\n\n### Step 2 — Edit `searxng/settings.yml`\n\n**Enable the plugin** — add this inside the `plugins:` section:\n\n```yaml\n searx.plugins.ai_summary.SXNGPlugin:\n active: true\n```\n\n**Configure the LLM** — add this as a new top-level key (not inside `plugins:`):\nMake sure it matchs the api endponit of you choosing that are relevent. This one it confurged for [LM Studio](https://lmstudio.ai/) local server.\n\n```yaml\nai_summary:\n base_url: \"http://127.0.0.1:1234/v1\" # your LM Studio IP:port\n api_key: \"lm-studio\" # any string for local providers\n model: \"google/gemma-3-1b\" # fast model — compact summary\n model_more: \"openai/gpt-oss-20b\" # smart model — More panel\n max_results: 5\n max_tokens: 300\n max_tokens_more: 800\n timeout: 60\n```\n\n**Find your IP address** (Windows):\n```powershell\nipconfig\n# Look for IPv4 Address under your active network adapter\n```\n\n**Find your model name** — open LM Studio → Developer tab → copy the\nmodel identifier shown under \"API Model Identifier\".\n\n**Set a secret key**:\n```yaml\nserver:\n secret_key: \"replace-this-with-something-long-and-random\"\n```\n\n### Step 3 — Start\n\n```powershell\ncd C:\\path\\to\\your-folder\ndocker compose up -d\n```\n\n### Step 4 — Open SearXNG\n\n```\nhttp://localhost:8080\n```\n\nSearch for anything. Results load immediately. The AI summary types itself\nin above the results. Click **More** for the structured deep-dive.\n\n---\n\n## Configuration Reference\n\nAll settings live in `searxng/settings.yml` under the `ai_summary:` key.\n\n| Key | Description |\n|-----|-------------|\n| `base_url` | OpenAI-compatible API root URL (no trailing slash) |\n| `api_key` | API key — any non-empty string for local providers |\n| `model` | Fast model used for the compact summary |\n| `model_more` | Smart model used for the More panel |\n| `max_results` | Result snippets sent to the LLM (default: 5) |\n| `max_tokens` | Max tokens for compact summary (default: 300) |\n| `max_tokens_more` | Max tokens for More panel (default: 800) |\n| `timeout` | Request timeout in seconds (default: 60) |\n| `system_prompt` | System prompt for compact summary |\n| `system_prompt_more` | System prompt for More panel — must instruct the model to return JSON |\n\n### Supported Backends\n\n| Provider | `base_url` | `api_key` |\n|----------|-----------|-----------|\n| LM Studio | `http://YOUR_IP:1234/v1` | `lm-studio` |\n| Ollama | `http://YOUR_IP:11434/v1` | `ollama` |\n| OpenAI | `https://api.openai.com/v1` | `sk-...` |\n| Groq | `https://api.groq.com/openai/v1` | `gsk_...` |\n| Together AI | `https://api.together.xyz/v1` | your key |\n| vLLM | `http://YOUR_IP:8000/v1` | any string |\n\n### Recommended Models\n\n| Model | Speed | Quality | Best for |\n|-------|-------|---------|----------|\n| `google/gemma-3-1b-it` | ⚡⚡⚡⚡ | ★★★ | `model` — fastest compact summary |\n| `google/gemma-3-4b-it` | ⚡⚡⚡ | ★★★★ | `model` — best speed/quality balance |\n| `microsoft/phi-4-mini-instruct` | ⚡⚡⚡ | ★★★★ | `model` — great instruction following |\n| `qwen/qwen2.5-7b-instruct` | ⚡⚡ | ★★★★★ | `model_more` — best structured output |\n| `mistralai/mistral-7b-instruct` | ⚡⚡ | ★★★★ | `model_more` — reliable JSON output |\n| `openai/gpt-oss-20b` | ⚡ | ★★★★★ | `model_more` — highest quality |\n\nAlways use `-it` or `-instruct` variants. The More panel requires a model\nthat reliably returns valid JSON — larger models do this better.\n\n---\n\n## After Any Change\n\n```powershell\ndocker compose restart searxng\n```\n\nChanges to `settings.yml`, `ai_summary.py`, or `ai_summary.js` all take\neffect after a restart. No rebuild needed — all files are volume-mounted.\n\n---\n\n## Troubleshooting\n\n```powershell\n# Watch live logs\ndocker compose logs -f searxng\n\n# Filter for plugin messages only\ndocker compose logs searxng | findstr ai_summary\n```\n\n| Symptom | Cause | Fix |\n|---------|-------|-----|\n| `PluginCfg unexpected keyword` | LLM settings inside `plugins:` block | Move them to the separate `ai_summary:` top-level block |\n| SearXNG won't start | YAML indentation error | Check that all plugin keys have exactly 2 spaces indent |\n| `Connection refused` | LM Studio server not started | Open LM Studio → Developer → flip Status toggle to ON |\n| No summary, no error | Plugin not loaded | Check `active: true` is set and `ai_summary.py` is volume-mounted |\n| Summary never appears | LLM timeout | Increase `timeout` or use a smaller model |\n| More panel shows raw JSON | Old JS file cached | Hard refresh (`Ctrl+Shift+R`) or restart container |\n| More panel errors | Smart model too slow | Increase `timeout`, reduce `max_tokens_more`, or use a faster model |\n| HTTP 401/403 | Wrong API key | Fix `api_key` in settings.yml |\n| HTTP 404 | Wrong URL or model name | Fix `base_url` or `model` / `model_more` |\n| Summary has markdown | Model ignoring system prompt | Switch to an `-instruct` or `-it` model variant |\n\n### Test your LLM directly\n\n```powershell\n@'\n{\n \"model\": \"YOUR_MODEL_NAME\",\n \"max_tokens\": 150,\n \"stream\": false,\n \"messages\": [\n {\"role\": \"system\", \"content\": \"Summarize in 2 sentences. No markdown.\"},\n {\"role\": \"user\", \"content\": \"What is Apache web server?\"}\n ]\n}\n'@ | Out-File -FilePath test.json -Encoding utf8\n\ncurl.exe -X POST http://YOUR_IP:1234/v1/chat/completions `\n -H \"Content-Type: application/json\" `\n -d \"@test.json\"\n```\n\nA working response has `choices[0].message.content` with plain text. If this\nworks but the plugin doesn't, check the Docker logs for the exact error.\n\n---\n\n## License\n\nMIT\n"}}}} |