Files
hack/tools/scripts/llm-key-hunter/results/_cache/content/above-os_terminus-apps.json
T
OpenCode 68a9071cf1 feat: reconcile local master with origin (exa/hot-platform hunters, leaks ledger, latest results)
Local branch had diverged from origin/master (sibling commits on the same base). Rewrote local history linearly on top of origin/master, folding in all local content: exa/hot-platform discovery hunters, leaks ledger, nightly sweep orchestrator, updated .gitignore and skill docs, plus the latest hunt output and vault state. Remote-only files (vault keys, channel scripts) were restored rather than dropped, so the resulting tree is a full union of both sides.
2026-08-07 12:53:47 +08:00

1 line
190 KiB
JSON
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
{"repo": "above-os/terminus-apps", "files": {"deerflowv2/templates/configmap.yaml": {"0a43d0a1a5353625feba099509ff5fe18f5e6bc0": {"t": 1785745229.4663084, "n": 47208, "h": "488ab46599e8e4de", "c": "---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: deerflowv2-env-config\n namespace: {{ .Release.Namespace }}\ndata:\n # ──────────────────────────────────────────────────────────────────────────\n # System locale -- forces UTF-8 inside the slim Python container.\n # Upstream commit 82e7936d bakes these into the Dockerfile, but our v2.0.1\n # backend image was built before that commit. Without them, any path or\n # tool output containing non-ASCII characters (Chinese filenames, emoji\n # in markdown, etc.) raises UnicodeEncodeError when Python opens the file\n # or writes to stdout. Safe to leave on permanently.\n # ──────────────────────────────────────────────────────────────────────────\n LANG: \"C.UTF-8\"\n LC_ALL: \"C.UTF-8\"\n PYTHONIOENCODING: \"utf-8\"\n\n # ──────────────────────────────────────────────────────────────────────────\n # Web search & crawl provider keys.\n # Tavily / Jina / InfoQuest are the original three. Serper / Firecrawl /\n # Exa were added by upstream in 2026-Q2 (Serper PR #2630 in commit\n # 44ab21fc). Leave empty unless the corresponding tool is referenced in\n # config.yaml's `tools:` list -- empty values are fine, deer-flow only\n # raises if a $VAR reference resolves to nothing.\n # ──────────────────────────────────────────────────────────────────────────\n TAVILY_API_KEY: \"\"\n JINA_API_KEY: \"\"\n INFOQUEST_API_KEY: \"\"\n SERPER_API_KEY: \"\"\n FIRECRAWL_API_KEY: \"\"\n EXA_API_KEY: \"\"\n\n # ──────────────────────────────────────────────────────────────────────────\n # Model provider keys. Reference these via $VAR in config.yaml's `models:`\n # section. Empty values are safe; only models that actually use the key\n # will fail to register at startup.\n # ──────────────────────────────────────────────────────────────────────────\n OPENAI_API_KEY: \"\"\n DEEPSEEK_API_KEY: \"\"\n ANTHROPIC_API_KEY: \"\"\n GOOGLE_API_KEY: \"\"\n GEMINI_API_KEY: \"\"\n VOLCENGINE_API_KEY: \"\"\n NOVITA_API_KEY: \"\"\n MINIMAX_API_KEY: \"\"\n MOONSHOT_API_KEY: \"\"\n VLLM_API_KEY: \"\"\n\n # LangSmith tracing (set to \"true\" and fill LANGSMITH_API_KEY to enable)\n LANGCHAIN_TRACING_V2: \"false\"\n LANGSMITH_API_KEY: \"\"\n\n # ──────────────────────────────────────────────────────────────────────────\n # Upload limits. Default upstream caps are tuned for the docker-compose\n # demo (50 MiB per file, 100 MiB per request, 10 files); on Olares users\n # routinely drop in 4K screenshots, recordings, and multi-page PDFs that\n # exceed the per-file cap. Bumping the defaults to 100 MiB / 500 MiB / 20\n # is conservative for an at-home server -- raise further if you upload\n # large media. Sizes are bytes.\n # ──────────────────────────────────────────────────────────────────────────\n UPLOADS_MAX_FILES: \"20\"\n UPLOADS_MAX_FILE_SIZE: \"104857600\" # 100 MiB\n UPLOADS_MAX_TOTAL_SIZE: \"524288000\" # 500 MiB\n # ============================================================================\n # SUBAGENT LIMITS — tune to fix \"Recursion limit reached\" errors\n # ============================================================================\n # Each subagent (general-purpose, bash) runs as its own LangGraph instance.\n # `max_turns` caps how many LLM <-> tool roundtrips it can take before the\n # graph aborts with `GraphRecursionError`. `timeout_seconds` caps wall time.\n #\n # The values below are referenced by config.yaml as `$VAR`, resolved via\n # DeerFlow's `resolve_env_variables()` and coerced to int by Pydantic\n # (lax mode), so quoted strings here become real ints downstream.\n #\n # ─── WHEN TO ADJUST ─────────────────────────────────────────────────────────\n # Symptom → Action\n # \"Recursion limit of N reached\" in logs → raise the matching\n # *_MAX_TURNS\n # Subagent aborts with \"timed out\" → raise the matching\n # *_TIMEOUT_SECONDS\n # Local model burns CPU/$$ in obvious loops → lower *_MAX_TURNS to\n # surface failures fast\n #\n # ─── RECOMMENDED VALUES BY MODEL CLASS ──────────────────────────────────────\n # Model class gp_turns bash_turns\n # ────────────────────────────────────────────── ──────── ──────────\n # Strong frontier (Claude 4 / GPT-5 / Doubao 2) 150 80\n # Efficient MoE (Gemma 4 26B A4B) 200 100\n # Mid local (Qwen3 32B+ / DeepSeek V3.2 / Kimi 2.5) 300 150\n # Heavy local 30B-class (Qwen3.x 35B) ← default 500 300\n # Weak / heavily quantized (Qwen3 ≤ 14B, INT4) 1000 400\n #\n # gp_turns = SUBAGENT_GENERAL_PURPOSE_MAX_TURNS\n # bash_turns = SUBAGENT_BASH_MAX_TURNS\n # Image-baked builtin defaults: gp=500, bash=300 (already raised from\n # upstream's 100 / 60 in this fork to handle long delegated workflows).\n #\n # ─── HOW TO SIZE TIMEOUTS ───────────────────────────────────────────────────\n # Timeouts should be a safety net for hung/runaway processes — they should\n # NOT trigger before max_turns under normal operation, otherwise raising\n # max_turns has no effect. Rule of thumb:\n # timeout_seconds ≥ max_turns × realistic_seconds_per_turn\n #\n # Per-turn cost depends on model speed:\n # Strong cloud (Claude / GPT-5) 5-15 s/turn\n # Efficient MoE on Blackwell GPU 3-8 s/turn\n # 30B local on RTX 4090 Q4 15-30 s/turn\n # 30B local on CPU / iGPU 60-180 s/turn\n #\n # Defaults below assume 500 turns × 14 s/turn ≈ 2 h for general-purpose,\n # which covers cloud and mid-tier local GPUs. For pure-CPU or very slow\n # quantized local setups, push GENERAL_PURPOSE_TIMEOUT_SECONDS to 14400\n # (4 h) or more.\n #\n # ─── HOW TO APPLY CHANGES ───────────────────────────────────────────────────\n # 1. Edit the value below (or `kubectl edit configmap deerflowv2-env-config`).\n # 2. Restart the backend so the new env vars are picked up:\n # kubectl -n <namespace> rollout restart deploy/deerflowv2-backend\n # 3. Verify inside the pod:\n # kubectl -n <ns> exec deploy/deerflowv2-backend -c gateway -- \\\n # env | grep SUBAGENT_\n #\n # ─── CAVEATS ────────────────────────────────────────────────────────────────\n # - Values MUST stay quoted strings (e.g. \"500\", not 500). They are env vars.\n # - Do NOT delete these keys. `resolve_env_variables()` raises ValueError if\n # a `$VAR` reference can't be resolved, which crashes the pod on startup.\n # - For installs whose appData/config/config.yaml predates this chart (no\n # `subagents:` block, or hard-coded numbers): either edit it manually to\n # match the template below, OR delete it and let the init-container\n # regenerate from the template (this loses any custom `models:` you added,\n # so back it up first).\n # - Changes to this ConfigMap do NOT auto-restart the pod. The rollout\n # restart in step 2 is required.\n SUBAGENT_DEFAULT_TIMEOUT_SECONDS: \"1800\"\n # Global default max_turns. Per-agent values below override this, so it\n # only kicks in for any custom_agents the user defines without an explicit\n # max_turns override (e.g. a \"data-analysis\" subagent created via\n # subagents.custom_agents in config.yaml). Upstream builtin default is 50.\n SUBAGENT_DEFAULT_MAX_TURNS: \"200\"\n SUBAGENT_GENERAL_PURPOSE_MAX_TURNS: \"500\"\n SUBAGENT_GENERAL_PURPOSE_TIMEOUT_SECONDS: \"7200\"\n SUBAGENT_BASH_MAX_TURNS: \"300\"\n SUBAGENT_BASH_TIMEOUT_SECONDS: \"1800\"\n---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: deerflowv2-init-config\n namespace: {{ .Release.Namespace }}\ndata:\n config.yaml: |\n # Bumped to match upstream config.example.yaml (deer-flow main, May 2026).\n # The version number itself is purely a \"your config is outdated, run\n # config-upgrade\" warning trigger in deer-flow's startup log -- it does\n # not unlock or restrict any features.\n config_version: 31\n\n # Logger threshold for the `deerflow` and `app` loggers. Third-party\n # libraries (uvicorn, sqlalchemy, langgraph, ...) keep their own levels.\n # Use \"debug\" temporarily to surface tool selection / middleware decisions\n # when an agent misbehaves, then revert.\n log_level: info\n\n # Surface input/output/total token counts in the workspace UI when the\n # provider returns them. Helps spot a runaway Ultra session before it\n # eats a Tavily quota or burns local GPU time on a duplicate-dispatch.\n token_usage:\n enabled: true\n\n # =============================================\n # HOW TO ADD MODELS\n # =============================================\n # Edit this file at: appData/config/config.yaml\n #\n # Ollama example (use models you've pulled in Ollama):\n # - name: qwen3\n # display_name: Qwen3 (Ollama)\n # use: langchain_openai:ChatOpenAI\n # model: qwen3\n # api_key: ollama\n # base_url: $OLLAMA_OPENAI_BASE_URL\n # supports_thinking: true # enables Pro/Ultra modes (Qwen3+ supports thinking)\n # supports_vision: true # enables image input (only for vision models like Qwen2.5-VL)\n #\n # OpenAI example:\n # - name: gpt-4o\n # display_name: GPT-4o\n # use: langchain_openai:ChatOpenAI\n # model: gpt-4o\n # api_key: $OPENAI_API_KEY\n # supports_thinking: true\n # supports_vision: true\n #\n # Claude example:\n # - name: claude-sonnet\n # display_name: Claude 4 Sonnet\n # use: langchain_anthropic:ChatAnthropic\n # model: claude-sonnet-4-20250514\n # api_key: $ANTHROPIC_API_KEY\n # supports_thinking: true\n # supports_reasoning_effort: true # enables reasoning effort slider\n # supports_vision: true\n #\n # Key model capabilities:\n # supports_thinking: true/false - Pro/Ultra mode (thinking/planning before answering)\n # supports_reasoning_effort: true/false - reasoning effort slider (low/medium/high)\n # supports_vision: true/false - image/vision input support\n #\n # After editing, restart the app to apply changes.\n models: []\n\n tool_groups:\n - name: web\n - name: file:read\n - name: file:write\n - name: bash\n\n tools:\n - name: web_search\n group: web\n use: deerflow.community.tavily.tools:web_search_tool\n max_results: 5\n - name: web_fetch\n group: web\n use: deerflow.community.jina_ai.tools:web_fetch_tool\n timeout: 10\n - name: image_search\n group: web\n use: deerflow.community.image_search.tools:image_search_tool\n max_results: 5\n - name: ls\n group: file:read\n use: deerflow.sandbox.tools:ls_tool\n - name: read_file\n group: file:read\n use: deerflow.sandbox.tools:read_file_tool\n - name: write_file\n group: file:write\n use: deerflow.sandbox.tools:write_file_tool\n - name: str_replace\n group: file:write\n use: deerflow.sandbox.tools:str_replace_tool\n - name: bash\n group: bash\n use: deerflow.sandbox.tools:bash_tool\n\n tool_search:\n enabled: false\n\n # =============================================\n # UPLOADS\n # =============================================\n # All values are sourced from env vars in deerflowv2-env-config so\n # operators can resize without editing config.yaml. See that ConfigMap\n # for tuning notes. Sizes are bytes; the gateway also surfaces these\n # to the frontend so the file picker can reject too-large files before\n # they hit the wire.\n uploads:\n max_files: $UPLOADS_MAX_FILES\n max_file_size: $UPLOADS_MAX_FILE_SIZE\n max_total_size: $UPLOADS_MAX_TOTAL_SIZE\n # Office/PDF auto-conversion runs on the gateway host BEFORE the\n # sandbox isolates the file. Keep off unless uploads come from a\n # fully trusted source -- a malicious .docx can hit the parser before\n # any sandboxing applies. The image-ocr Skill handles user-driven\n # OCR explicitly via tesseract anyway.\n auto_convert_documents: false\n pdf_converter: auto\n\n sandbox:\n use: deerflow.sandbox.local:LocalSandboxProvider\n # Re-enable host bash for the LLM. Upstream PR #1547 (Security: Address\n # critical host-shell escape in LocalSandboxProvider) gates the `bash`\n # tool behind this flag, because LocalSandboxProvider executes commands\n # directly on the gateway pod's filesystem rather than inside an\n # isolated sandbox container. With the flag off, the LLM cannot\n # `pip install` or `apt-get install` runtime helpers (notably\n # pytesseract / tesseract-ocr for OCR), which silently breaks the\n # legacy \"ask the model to OCR an image\" workflow that worked before\n # we picked up the security patch.\n #\n # In a single-tenant Olares deployment the gateway pod is the trust\n # boundary anyway, so re-enabling this restores prior behavior without\n # widening exposure beyond what existed before. If you ever expose this\n # instance to untrusted users, switch to AioSandboxProvider (per-thread\n # container) instead and flip this back to false.\n allow_host_bash: true\n\n skills:\n container_path: /mnt/skills\n\n title:\n enabled: true\n max_words: 6\n max_chars: 60\n model_name: null\n\n summarization:\n enabled: true\n model_name: null\n trigger:\n - type: tokens\n value: 15564\n keep:\n type: messages\n value: 10\n trim_tokens_to_summarize: 15564\n summary_prompt: null\n # Recently-loaded skill files (e.g. the image-ocr SKILL.md the agent\n # just opened) are exempt from summarization, otherwise the agent\n # would lose its instructions mid-task. These are upstream defaults\n # pinned here so a future deer-flow update changing the defaults\n # cannot silently degrade the image-ocr workflow.\n preserve_recent_skill_count: 5\n preserve_recent_skill_tokens: 25000\n preserve_recent_skill_tokens_per_skill: 5000\n skill_file_read_tool_names:\n - read_file\n - read\n - view\n - cat\n\n memory:\n enabled: true\n storage_path: memory.json\n debounce_seconds: 30\n model_name: null\n max_facts: 100\n fact_confidence_threshold: 0.7\n injection_enabled: true\n max_injection_tokens: 2000\n\n # =============================================\n # PERSISTENCE (users, threads metadata, checkpoints, run events)\n # =============================================\n # The unified `database:` block drives EVERYTHING that needs durable\n # storage: gateway user store (users + sessions + tokens), thread\n # metadata, the LangGraph checkpointer, and the run-event journal\n # (token usage, tool-call traces). All routed to a single postgres\n # database provisioned automatically by the Olares postgres\n # middleware (declared in OlaresManifest.yaml as `middleware.postgres`).\n #\n # Connection string is built from .Values.postgres.* in deployment.yaml:\n # DATABASE_URL → SQLAlchemy ORM (asyncpg) + LangGraph checkpointer (psycopg)\n # Do not put libpq-only keepalive params in DATABASE_URL (asyncpg rejects them).\n # deer-flow resolves via AppConfig.resolve_env_variables() at startup.\n #\n # To switch back to sqlite (single-tenant, no postgres dependency):\n # 1. Remove the `middleware.postgres` block from OlaresManifest.yaml\n # 2. Remove the DATABASE_URL env entries from deployment.yaml\n # 3. Replace the block below with:\n # database:\n # backend: sqlite\n # sqlite_dir: /data/deer-flow/db\n # 4. Recreate the appData/config/config.yaml or run the migration\n # script in reverse.\n database:\n backend: postgres\n postgres_url: $DATABASE_URL\n\n # =============================================\n # SUBAGENT LIMITS\n # =============================================\n # All values resolved from env vars defined in the deerflowv2-env-config\n # ConfigMap — see that ConfigMap's comments for tuning guidance,\n # recommended values per model class, and how to apply changes.\n # Tune the env vars there, NOT the literals below.\n subagents:\n timeout_seconds: $SUBAGENT_DEFAULT_TIMEOUT_SECONDS\n max_turns: $SUBAGENT_DEFAULT_MAX_TURNS\n agents:\n general-purpose:\n timeout_seconds: $SUBAGENT_GENERAL_PURPOSE_TIMEOUT_SECONDS\n max_turns: $SUBAGENT_GENERAL_PURPOSE_MAX_TURNS\n bash:\n timeout_seconds: $SUBAGENT_BASH_TIMEOUT_SECONDS\n max_turns: $SUBAGENT_BASH_MAX_TURNS\n\n # =============================================\n # RUN EVENTS (message + tool-call telemetry)\n # =============================================\n # Upstream defaults this to \"memory\", which means every conversation's\n # full execution trace (subagent dispatches, tool invocations, token\n # counts, errors) is kept only in process RAM and lost on pod restart.\n # That makes it impossible to audit \"why did the lead burn 15 minutes\n # last night\" after the fact, and it makes the per-run token-usage\n # accounting in the UI reset on every restart.\n #\n # We already run sqlite for the user store + checkpointer, so flipping\n # run_events to the same backend is essentially free -- traces land in\n # the same deerflow.db file. Still bounded by max_trace_content (10 KiB\n # per individual trace blob) so a runaway tool output cannot bloat the\n # DB indefinitely.\n run_events:\n backend: db\n max_trace_content: 10240\n track_token_usage: true\n\n # =============================================\n # CIRCUIT BREAKER (LLM provider failure isolation)\n # =============================================\n # When the upstream model provider (Ollama, vLLM, OpenAI, ...) returns\n # 5 consecutive failures, fast-fail subsequent calls for 60 seconds\n # instead of letting LangGraph retry-loops hammer a dying provider.\n # Defaults below match upstream and work well for both local Ollama\n # (where 5 timeouts means the model is genuinely stuck) and cloud APIs\n # (where 5 failures usually means rate limit / outage). Pinned here so\n # the values are visible to operators rather than buried in code.\n circuit_breaker:\n failure_threshold: 5\n recovery_timeout_sec: 60\n\n extensions_config.json: |\n {\n \"mcpServers\": {},\n \"skills\": {}\n }\n\n # ============================================================================\n # MIGRATION SCRIPT: sqlite -> postgres database backend\n # ============================================================================\n # Mounted at /init-config/migrate_database_to_postgres.py and invoked by the\n # init-container in deployment.yaml. Idempotently rewrites the `database:`\n # block in an existing /config/config.yaml (the operator's persisted copy\n # on the appData hostPath) so chart upgrades flip the backend in place\n # without losing user-edited fields like `models:`.\n #\n # Why python instead of sed/awk: top-level YAML block detection has to\n # respect indentation, comment lines, and key order. A regex one-liner\n # gets it wrong on configs that operators have hand-edited, and the\n # failure mode (silently appending a duplicate `database:` block, then\n # PyYAML picks one of the two non-deterministically) is hard to debug.\n # The deer-flow image ships python3.12, so this is free.\n migrate_database_to_postgres.py: |\n \"\"\"Rewrite database:/checkpointer: blocks in config.yaml to use postgres.\n\n Usage: python3 migrate_database_to_postgres.py /path/to/config.yaml\n \"\"\"\n from __future__ import annotations\n\n import re\n import sys\n from pathlib import Path\n\n POSTGRES_BLOCK = (\n \"\\n\"\n \"# Auto-injected by init-container (chart >= 1.0.9). The Olares\\n\"\n \"# `middleware.postgres` provisioner exposes connection details via\\n\"\n \"# DATABASE_URL (SQLAlchemy + LangGraph checkpointer). To switch\\n\"\n \"# back to sqlite, remove the postgres middleware block from\\n\"\n \"# OlaresManifest.yaml and edit this section by hand.\\n\"\n \"database:\\n\"\n \" backend: postgres\\n\"\n \" postgres_url: $DATABASE_URL\\n\"\n )\n\n\n def strip_checkpointer_postgres_url(cfg: Path) -> None:\n \"\"\"Remove dual-URL leftover from chart < 1.0.7.\"\"\"\n text = cfg.read_text(encoding=\"utf-8\")\n if \"checkpointer_postgres_url:\" not in text:\n return\n lines = [\n ln for ln in text.splitlines(keepends=True)\n if \"checkpointer_postgres_url:\" not in ln\n ]\n cfg.write_text(\"\".join(lines), encoding=\"utf-8\")\n print(f\"migrate: removed checkpointer_postgres_url from {cfg}\")\n\n # Match a top-level YAML block whose key is `database` or the legacy\n # upstream `checkpointer`. The body is any consecutive run of indented\n # or blank lines following the key. Stops at the next top-level key\n # (a non-whitespace character at column 0) or end-of-file.\n BLOCK_RE = re.compile(\n r\"^(?:database|checkpointer):[ \\t]*\\n(?:[ \\t]+[^\\n]*\\n|[ \\t]*\\n)*\",\n re.MULTILINE,\n )\n\n\n def main(path: str) -> int:\n cfg = Path(path)\n if not cfg.exists():\n print(f\"migrate: {cfg} does not exist; nothing to do\")\n return 0\n\n original = cfg.read_text(encoding=\"utf-8\")\n # Drop ALL existing database:/checkpointer: blocks. There may be\n # more than one if a previous chart bug appended duplicates.\n stripped = BLOCK_RE.sub(\"\", original)\n # Trim trailing whitespace before re-appending so the final file\n # is not padded with blank lines after repeated runs.\n new = stripped.rstrip() + \"\\n\" + POSTGRES_BLOCK\n\n if new == original:\n print(f\"migrate: {cfg} already on postgres backend; no change\")\n strip_checkpointer_postgres_url(cfg)\n return 0\n\n cfg.write_text(new, encoding=\"utf-8\")\n print(f\"migrate: rewrote {cfg} database block -> postgres\")\n strip_checkpointer_postgres_url(cfg)\n return 0\n\n\n if __name__ == \"__main__\":\n sys.exit(main(sys.argv[1] if len(sys.argv) > 1 else \"/config/config.yaml\"))\n\n # ============================================================================\n # MIGRATION SCRIPT: legacy ephemeral SQLite -> postgres conversation history\n # ============================================================================\n # Mounted into the *gateway* container at /migrate (see deployment.yaml).\n # Run on demand by the operator AFTER:\n # 1. Upgrading to chart >= 1.1.4 (postgres backend + this script)\n # 2. Visiting /setup in the browser to create the first admin account\n #\n # kubectl -n <ns> exec -it deploy/deerflowv2-backend -c gateway -- \\\n # sh /migrate/migrate_legacy_sqlite.sh <admin_email>\n #\n # The script reads /data/deer-flow/checkpoints.db read-only, looks up the\n # admin user_id from the new PG `users` table, and writes every checkpoint\n # + pending write into PostgreSQL via AsyncPostgresSaver, stamping\n # metadata.user_id = <admin> so the migrated threads show up in the admin's\n # sidebar (langgraph_auth.add_owner_filter scopes the thread list to\n # metadata.user_id == ctx.user.identity).\n #\n # Idempotent: re-running upserts the same rows. Source SQLite is opened\n # read-only and never modified -- safe to run as many times as needed.\n # Mounted via items: so only these two files appear under /migrate, the\n # rest of init-config (config.yaml etc.) does not leak into the runtime\n # filesystem.\n migrate_legacy_sqlite.sh: |\n #!/bin/sh\n # Convenience wrapper around migrate_legacy_sqlite.py. Sets up PYTHONPATH\n # and uv so the script can import deerflow.* modules from the running\n # gateway image without the operator having to remember the venv layout.\n set -e\n\n if [ \"$#\" -lt 1 ]; then\n cat <<'EOF'\n Usage: sh /migrate/migrate_legacy_sqlite.sh <admin_email> [--src PATH] [--dry-run]\n\n Examples:\n sh /migrate/migrate_legacy_sqlite.sh admin@example.com\n sh /migrate/migrate_legacy_sqlite.sh admin@example.com --dry-run\n sh /migrate/migrate_legacy_sqlite.sh admin@example.com --src /data/deer-flow/db/checkpoints.db\n EOF\n exit 1\n fi\n\n ADMIN_EMAIL=\"$1\"\n shift\n\n cd /app/backend\n exec env PYTHONPATH=. uv run python /migrate/migrate_legacy_sqlite.py \\\n --admin-email \"$ADMIN_EMAIL\" \"$@\"\n\n migrate_legacy_sqlite.py: |\n \"\"\"Migrate legacy SQLite checkpointer data to the new PostgreSQL backend.\n\n Run inside the gateway container after upgrading to chart >= 1.1.4 and\n completing /setup (so an admin user exists in PG):\n\n kubectl -n <ns> exec -it deploy/deerflowv2-backend -c gateway -- \\\n sh /migrate/migrate_legacy_sqlite.sh <admin_email>\n\n Two-phase migration. Both phases run in one invocation:\n\n Phase 1 -- LangGraph checkpoints\n Opens /data/deer-flow/checkpoints.db read-only via AsyncSqliteSaver\n and copies every checkpoint + pending write into PostgreSQL via\n AsyncPostgresSaver, stamping metadata.user_id = <admin user_id> so\n langgraph_auth.add_owner_filter authorises the migrated threads.\n\n Phase 2 -- threads_meta rows (the sidebar list)\n DeerFlow's gateway lists conversations from a separate `threads_meta`\n SQL table -- distinct from LangGraph's checkpoints schema. Without\n a row here a migrated thread is invisible in the UI even though the\n underlying checkpoint exists. For each thread_id seen in phase 1,\n upsert a row with display_name = first user message (60 chars),\n user_id = admin, created/updated_at = first/last checkpoint ts.\n\n Idempotent on both phases: re-running upserts the same checkpoints and\n only inserts threads_meta rows that don't already exist (or backfills\n user_id / display_name on rows whose owner is NULL). The source SQLite\n file is opened read-only and never modified.\n \"\"\"\n from __future__ import annotations\n\n import argparse\n import asyncio\n import logging\n import os\n from collections import defaultdict\n from datetime import UTC, datetime\n from pathlib import Path\n from typing import Any\n\n logging.basicConfig(\n level=logging.INFO,\n format=\"%(asctime)s %(levelname)s %(name)s %(message)s\",\n )\n log = logging.getLogger(\"migrate\")\n\n DEFAULT_SQLITE = \"/data/deer-flow/checkpoints.db\"\n DISPLAY_NAME_MAX_LEN = 60\n\n\n # ------------------------------------------------------------------ helpers\n\n\n def _parse_iso(value: Any) -> datetime | None:\n \"\"\"Parse the LangGraph checkpoint `ts` field into an aware datetime.\n\n Checkpoints written by older LangGraph versions sometimes carry the\n `ts` as a millisecond integer; newer ones use ISO-8601 strings.\n Returns None when the value is missing or unparsable so the caller\n can fall back to `datetime.now(UTC)` rather than crashing the whole\n thread-meta backfill on one bad row.\n \"\"\"\n if not value:\n return None\n if isinstance(value, datetime):\n return value if value.tzinfo else value.replace(tzinfo=UTC)\n if isinstance(value, (int, float)):\n try:\n return datetime.fromtimestamp(float(value), UTC)\n except (OSError, OverflowError, ValueError):\n return None\n if isinstance(value, str):\n s = value.replace(\"Z\", \"+00:00\")\n try:\n dt = datetime.fromisoformat(s)\n except ValueError:\n return None\n return dt if dt.tzinfo else dt.replace(tzinfo=UTC)\n return None\n\n\n def _extract_first_user_text(checkpoint: dict) -> str | None:\n \"\"\"Find the first human/user message text in a checkpoint state.\n\n LangGraph deserialises message channels into LangChain Message\n objects (HumanMessage, AIMessage, ...) with `.type` and `.content`,\n but very old checkpoints may contain raw dicts instead. Handle\n both. Multimodal `content` (list of part dicts) is flattened to\n its concatenated text parts so an image-only first turn does not\n produce an empty title.\n \"\"\"\n channel_values = checkpoint.get(\"channel_values\") or {}\n messages = channel_values.get(\"messages\") or []\n for msg in messages:\n mtype = getattr(msg, \"type\", None) or (\n msg.get(\"type\") if isinstance(msg, dict) else None\n )\n if mtype not in {\"human\", \"user\"}:\n continue\n content = getattr(msg, \"content\", None)\n if content is None and isinstance(msg, dict):\n content = msg.get(\"content\")\n if isinstance(content, str):\n text = content.strip()\n elif isinstance(content, list):\n parts = []\n for part in content:\n if isinstance(part, dict) and part.get(\"type\") == \"text\":\n parts.append(str(part.get(\"text\", \"\")))\n elif isinstance(part, str):\n parts.append(part)\n text = \" \".join(p for p in parts if p).strip()\n else:\n text = \"\"\n if text:\n return text[:DISPLAY_NAME_MAX_LEN]\n return None\n\n\n # ----------------------------------------------------------------- phase 1\n\n\n async def _lookup_user_id(email: str) -> str:\n \"\"\"Resolve an admin email to its user_id row in the new PG users table.\"\"\"\n from sqlalchemy import select\n\n from deerflow.persistence.engine import get_session_factory\n from deerflow.persistence.user.model import UserRow\n\n sf = get_session_factory()\n if sf is None:\n raise SystemExit(\n \"session factory is None -- engine failed to initialise. \"\n \"Is DATABASE_URL set inside this pod?\"\n )\n async with sf() as session:\n row = (\n await session.execute(select(UserRow).where(UserRow.email == email))\n ).scalar_one_or_none()\n if row is None:\n raise SystemExit(\n f\"No user with email {email!r} in PG. Did you complete /setup?\"\n )\n return str(row.id)\n\n\n async def _migrate_checkpoints(\n src_path: str, target_user_id: str, dry_run: bool\n ) -> dict[str, dict]:\n \"\"\"Phase 1: copy every SQLite checkpoint into PG.\n\n Returns a per-thread summary {thread_id: {\"created\": dt, \"updated\": dt,\n \"display_name\": str|None}} so phase 2 can upsert threads_meta rows\n without re-reading the source file.\n \"\"\"\n from langgraph.checkpoint.postgres.aio import AsyncPostgresSaver\n from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver\n\n # file: URI with mode=ro guarantees we never write to the source DB,\n # even if AsyncSqliteSaver.setup() were to attempt a CREATE TABLE.\n src_uri = f\"file:{src_path}?mode=ro\"\n pg_url = (\n os.environ.get(\"CHECKPOINTER_DATABASE_URL\")\n or os.environ.get(\"DATABASE_URL\")\n )\n if not pg_url:\n raise SystemExit(\"CHECKPOINTER_DATABASE_URL / DATABASE_URL not set\")\n\n log.info(\"phase 1: LangGraph checkpoints\")\n log.info(\" source : %s (read-only)\", src_path)\n log.info(\" target : postgres (CHECKPOINTER_DATABASE_URL)\")\n log.info(\" user_id: %s\", target_user_id)\n if dry_run:\n log.info(\" DRY RUN -- nothing will be written to PG\")\n\n threads_info: dict[str, dict] = {}\n cp_count = 0\n pw_count = 0\n skipped = 0\n\n async with AsyncSqliteSaver.from_conn_string(src_uri) as src:\n async with AsyncPostgresSaver.from_conn_string(pg_url) as dst:\n if not dry_run:\n await dst.setup()\n\n # Passing config=None enumerates every checkpoint across every\n # thread. Iteration order is by checkpoint_id descending so the\n # *first* checkpoint we see for a given thread is the latest\n # one -- important for picking up the most recent user message\n # as the title fallback.\n async for tup in src.alist(None):\n try:\n config = dict(tup.config)\n config[\"configurable\"] = dict(config.get(\"configurable\", {}))\n thread_id = config[\"configurable\"].get(\"thread_id\", \"<no-thread>\")\n\n metadata = dict(tup.metadata or {})\n metadata[\"user_id\"] = target_user_id\n\n checkpoint = tup.checkpoint\n new_versions = dict(checkpoint.get(\"channel_versions\") or {})\n\n if dry_run:\n new_config = config\n else:\n new_config = await dst.aput(\n config, checkpoint, metadata, new_versions\n )\n\n cp_count += 1\n\n # Phase-2 summary -- updated on every checkpoint, but\n # `display_name` only gets *set* on the earliest user\n # message we encounter (which, given the descending\n # iteration, is updated until we hit step 0).\n ts = _parse_iso(checkpoint.get(\"ts\"))\n info = threads_info.setdefault(\n thread_id,\n {\"created\": ts, \"updated\": ts, \"display_name\": None},\n )\n if ts is not None:\n if info[\"created\"] is None or ts < info[\"created\"]:\n info[\"created\"] = ts\n if info[\"updated\"] is None or ts > info[\"updated\"]:\n info[\"updated\"] = ts\n text = _extract_first_user_text(checkpoint)\n if text:\n # Overwrite so we end up with the title from the\n # *earliest* checkpoint (chronologically the user's\n # opening question), not whichever step happened to\n # be enumerated first.\n info[\"display_name\"] = text\n\n if tup.pending_writes:\n grouped: dict[str, list[tuple[str, object]]] = defaultdict(list)\n for task_id, channel, value in tup.pending_writes:\n grouped[task_id].append((channel, value))\n for task_id, writes in grouped.items():\n if not dry_run:\n await dst.aput_writes(new_config, writes, task_id)\n pw_count += len(writes)\n\n if cp_count % 100 == 0:\n log.info(\n \" ...%d checkpoints, %d pending writes, %d threads so far\",\n cp_count, pw_count, len(threads_info),\n )\n except Exception as exc:\n # One bad row should not abort a 15 MB migration.\n skipped += 1\n log.warning(\" skipped checkpoint due to error: %s\", exc)\n\n log.info(\n \" done: %d checkpoints, %d pending writes, %d threads, %d skipped\",\n cp_count, pw_count, len(threads_info), skipped,\n )\n return threads_info\n\n\n # ----------------------------------------------------------------- phase 2\n\n\n async def _backfill_thread_meta(\n target_user_id: str,\n threads_info: dict[str, dict],\n dry_run: bool,\n ) -> None:\n \"\"\"Phase 2: insert/update one threads_meta row per migrated thread.\n\n Without this, /api/threads/search returns nothing for the migrated\n threads and the sidebar stays empty even though the checkpoints are\n in PG. Idempotent: existing rows whose user_id is None get adopted\n by the admin; existing rows already owned by someone are left alone.\n \"\"\"\n from deerflow.persistence.engine import get_session_factory\n from deerflow.persistence.thread_meta.model import ThreadMetaRow\n\n sf = get_session_factory()\n if sf is None:\n raise SystemExit(\"session factory is None\")\n\n log.info(\"phase 2: threads_meta backfill (sidebar entries)\")\n if dry_run:\n log.info(\" DRY RUN -- nothing will be written to PG\")\n for tid, info in list(threads_info.items())[:5]:\n log.info(\n \" would upsert thread=%s title=%r updated=%s\",\n tid,\n info.get(\"display_name\") or \"Untitled (migrated)\",\n info.get(\"updated\"),\n )\n log.info(\" ...and %d more thread(s)\", max(0, len(threads_info) - 5))\n return\n\n inserted = 0\n adopted = 0\n skipped = 0\n now = datetime.now(UTC)\n\n async with sf() as session:\n for thread_id, info in threads_info.items():\n try:\n existing = await session.get(ThreadMetaRow, thread_id)\n if existing is None:\n row = ThreadMetaRow(\n thread_id=thread_id,\n user_id=target_user_id,\n display_name=info.get(\"display_name\")\n or \"Untitled (migrated)\",\n status=\"idle\",\n metadata_json={\"migrated_from\": \"legacy_sqlite\"},\n created_at=info.get(\"created\") or now,\n updated_at=info.get(\"updated\") or now,\n )\n session.add(row)\n inserted += 1\n elif existing.user_id is None:\n # Pre-existing row from before auth was wired up.\n # Adopt it for the admin and fill in any missing\n # cosmetic fields.\n existing.user_id = target_user_id\n if not existing.display_name:\n existing.display_name = (\n info.get(\"display_name\") or \"Untitled (migrated)\"\n )\n existing.updated_at = info.get(\"updated\") or now\n adopted += 1\n else:\n # Already owned -- never silently steal a thread\n # that's been claimed by some other user (or even\n # the same admin, in which case the row is fine).\n skipped += 1\n except Exception as exc:\n log.warning(\n \" skipped thread_meta for %s due to error: %s\",\n thread_id, exc,\n )\n await session.commit()\n\n log.info(\n \" done: %d inserted, %d adopted (user_id was NULL), %d already owned\",\n inserted, adopted, skipped,\n )\n\n\n # --------------------------------------------------------------- driver\n\n\n async def _run(args: argparse.Namespace) -> None:\n from deerflow.config.app_config import get_app_config\n from deerflow.persistence.engine import close_engine, init_engine_from_config\n\n app_config = get_app_config()\n await init_engine_from_config(app_config.database)\n try:\n target_user_id = await _lookup_user_id(args.admin_email)\n log.info(\"resolved admin user_id: %s\", target_user_id)\n threads_info = await _migrate_checkpoints(\n args.src, target_user_id, args.dry_run\n )\n await _backfill_thread_meta(target_user_id, threads_info, args.dry_run)\n finally:\n await close_engine()\n\n log.info(\"=\" * 60)\n log.info(\"migration complete -- refresh the DeerFlow UI to see the\")\n log.info(\"migrated conversations in the admin sidebar.\")\n log.info(\"Safe to re-run if you upload a fresh sqlite dump.\")\n log.info(\"=\" * 60)\n\n\n def main() -> int:\n p = argparse.ArgumentParser(description=__doc__)\n p.add_argument(\n \"--admin-email\",\n required=True,\n help=\"Email of the admin user created via /setup\",\n )\n p.add_argument(\n \"--src\",\n default=DEFAULT_SQLITE,\n help=f\"Legacy SQLite path (default: {DEFAULT_SQLITE})\",\n )\n p.add_argument(\n \"--dry-run\",\n action=\"store_true\",\n help=\"Iterate the source DB but do not write anything to PG\",\n )\n args = p.parse_args()\n\n if not Path(args.src).exists():\n log.warning(\n \"source SQLite not found at %s -- nothing to migrate. \"\n \"Pass --src <path> if your file lives elsewhere.\",\n args.src,\n )\n return 0\n\n asyncio.run(_run(args))\n return 0\n\n\n if __name__ == \"__main__\":\n raise SystemExit(main())\n\n # ============================================================================\n # SKILL: image-ocr -- canonical OCR workflow for uploaded images\n # ============================================================================\n # deer-flow's <uploaded_files> middleware nudges the model toward read_file /\n # grep / glob (text-file affordances) regardless of whether the upload is\n # actually an image. Without further guidance the lead agent tends to either\n # (a) hallucinate the image contents, or (b) reach for a heavyweight OCR\n # stack like paddleocr / easyocr that downloads ~600 MB and blows past the\n # 600s subprocess.run timeout in LocalSandbox.execute_command.\n #\n # Skills are deer-flow's first-class extension point for this kind of fix:\n # any SKILL.md placed under <skills_root>/custom/<name>/ is auto-enabled\n # (extensions_config.is_skill_enabled defaults to True for the \"custom\"\n # category), and its `description` frontmatter is injected into the lead\n # agent system prompt as <available_skills>. When the user query matches\n # the description, the agent calls read_file on the SKILL.md to load the\n # full instructions, then follows them verbatim.\n #\n # The init-container copies this skill to /skills/custom/image-ocr/SKILL.md\n # on every pod start, so chart bumps push updated skill content. Operators\n # who want a different policy can edit the file directly, then bump the\n # `name` in the frontmatter so init does not overwrite it; or disable the\n # skill from the Olares extensions UI.\n image-ocr-skill.md: |\n ---\n name: image-ocr\n description: >-\n Extract text from an uploaded image. Trigger when the user asks to OCR\n an image, read text from a picture, recognize characters, transcribe a\n screenshot, identify what is written on a poster / receipt / business\n card / handwritten note, or extract any textual content from an image\n file. Common phrasings include \"OCR this image\", \"OCR it\", \"extract\n text from screenshot\", \"transcribe this image\", \"what does this image\n say\", \"read the text in this picture\", \"recognize the characters in\n this photo\", and equivalent requests in any other language (the model\n should match by intent, not literal wording). Trigger whenever a .png,\n .jpg, .jpeg, .webp, or .heic file is uploaded and the user wants the\n textual content rather than a visual description.\n ---\n\n # Image OCR\n\n Canonical, fast path for extracting text from an uploaded image. Follow\n this strictly to avoid the two failure modes operators have reported:\n\n 1. Hallucinating image contents instead of actually reading the file.\n 2. Picking heavyweight OCR stacks (paddleocr, easyocr, mmocr, rapidocr)\n that download hundreds of megabytes and time out the bash tool.\n\n ## Decision tree\n\n ### Step 1 -- Try `view_image` first (best when available)\n\n Look at your own tool list. If `view_image` is present, call it directly:\n\n ```\n view_image(image_path=\"/mnt/user-data/uploads/<filename>\")\n ```\n\n This injects the actual image bytes into your context as a vision input.\n Read the text directly from what you see and return it. No installation,\n no sandbox call, sub-second result, exact characters preserved.\n\n If `view_image` is not in your tool list (the selected model has no\n vision capability) or the call returns an error, fall through to Step 2.\n\n ### Step 2 -- pytesseract (only if Step 1 unavailable)\n\n Install `pytesseract` plus the `tesseract` binary and the Chinese +\n English language packs in one shot. Use these exact commands; do NOT\n substitute paddleocr, easyocr, mmocr, rapidocr, or any other OCR stack:\n they are 10x to 100x larger than tesseract and routinely exceed the\n bash tool's 600s wall-clock timeout.\n\n ```bash\n which tesseract >/dev/null 2>&1 || (apt-get update -qq && apt-get install -y -qq tesseract-ocr tesseract-ocr-chi-sim tesseract-ocr-chi-tra tesseract-ocr-eng > /dev/null)\n python -c \"import pytesseract\" 2>/dev/null || pip install -q pytesseract Pillow -i https://pypi.tuna.tsinghua.edu.cn/simple\n ```\n\n Then run OCR. The `lang` argument matters; without it Chinese text comes\n back as garbage:\n\n ```bash\n python - <<'PY'\n import pytesseract\n from PIL import Image\n text = pytesseract.image_to_string(\n Image.open('/mnt/user-data/uploads/<filename>'),\n lang='chi_sim+chi_tra+eng',\n )\n print(text)\n PY\n ```\n\n Total wall time on a fresh sandbox: roughly 30 to 60 seconds for the\n first install (apt fetches plus tesseract data files), under 2 seconds\n for subsequent calls in the same conversation since both apt and pip\n are cached inside the gateway pod for its lifetime.\n\n ## Constraints\n\n - Do not invent text. If the image is unreadable or OCR returns\n nothing, say so explicitly. Do not guess at the contents.\n - Do not save the OCR output as a file under `/mnt/user-data/outputs`\n unless the user explicitly asks for a file deliverable. Just\n include the extracted text inline in your response.\n - Do not chain multiple OCR libraries in a single turn. Pick one path\n (Step 1 OR Step 2) and stick with it. If Step 2 fails, report the\n failure to the user instead of falling back to a third option.\n - For multi-page PDFs that have already been converted to images, run\n Step 2 once per page and concatenate the results in page order.\n"}, "def169d049a102bbc457ce34bfc2ae04cbc616a8": {"t": 1785748955.35624, "n": 47208, "h": "488ab46599e8e4de", "c": "---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: deerflowv2-env-config\n namespace: {{ .Release.Namespace }}\ndata:\n # ──────────────────────────────────────────────────────────────────────────\n # System locale -- forces UTF-8 inside the slim Python container.\n # Upstream commit 82e7936d bakes these into the Dockerfile, but our v2.0.1\n # backend image was built before that commit. Without them, any path or\n # tool output containing non-ASCII characters (Chinese filenames, emoji\n # in markdown, etc.) raises UnicodeEncodeError when Python opens the file\n # or writes to stdout. Safe to leave on permanently.\n # ──────────────────────────────────────────────────────────────────────────\n LANG: \"C.UTF-8\"\n LC_ALL: \"C.UTF-8\"\n PYTHONIOENCODING: \"utf-8\"\n\n # ──────────────────────────────────────────────────────────────────────────\n # Web search & crawl provider keys.\n # Tavily / Jina / InfoQuest are the original three. Serper / Firecrawl /\n # Exa were added by upstream in 2026-Q2 (Serper PR #2630 in commit\n # 44ab21fc). Leave empty unless the corresponding tool is referenced in\n # config.yaml's `tools:` list -- empty values are fine, deer-flow only\n # raises if a $VAR reference resolves to nothing.\n # ──────────────────────────────────────────────────────────────────────────\n TAVILY_API_KEY: \"\"\n JINA_API_KEY: \"\"\n INFOQUEST_API_KEY: \"\"\n SERPER_API_KEY: \"\"\n FIRECRAWL_API_KEY: \"\"\n EXA_API_KEY: \"\"\n\n # ──────────────────────────────────────────────────────────────────────────\n # Model provider keys. Reference these via $VAR in config.yaml's `models:`\n # section. Empty values are safe; only models that actually use the key\n # will fail to register at startup.\n # ──────────────────────────────────────────────────────────────────────────\n OPENAI_API_KEY: \"\"\n DEEPSEEK_API_KEY: \"\"\n ANTHROPIC_API_KEY: \"\"\n GOOGLE_API_KEY: \"\"\n GEMINI_API_KEY: \"\"\n VOLCENGINE_API_KEY: \"\"\n NOVITA_API_KEY: \"\"\n MINIMAX_API_KEY: \"\"\n MOONSHOT_API_KEY: \"\"\n VLLM_API_KEY: \"\"\n\n # LangSmith tracing (set to \"true\" and fill LANGSMITH_API_KEY to enable)\n LANGCHAIN_TRACING_V2: \"false\"\n LANGSMITH_API_KEY: \"\"\n\n # ──────────────────────────────────────────────────────────────────────────\n # Upload limits. Default upstream caps are tuned for the docker-compose\n # demo (50 MiB per file, 100 MiB per request, 10 files); on Olares users\n # routinely drop in 4K screenshots, recordings, and multi-page PDFs that\n # exceed the per-file cap. Bumping the defaults to 100 MiB / 500 MiB / 20\n # is conservative for an at-home server -- raise further if you upload\n # large media. Sizes are bytes.\n # ──────────────────────────────────────────────────────────────────────────\n UPLOADS_MAX_FILES: \"20\"\n UPLOADS_MAX_FILE_SIZE: \"104857600\" # 100 MiB\n UPLOADS_MAX_TOTAL_SIZE: \"524288000\" # 500 MiB\n # ============================================================================\n # SUBAGENT LIMITS — tune to fix \"Recursion limit reached\" errors\n # ============================================================================\n # Each subagent (general-purpose, bash) runs as its own LangGraph instance.\n # `max_turns` caps how many LLM <-> tool roundtrips it can take before the\n # graph aborts with `GraphRecursionError`. `timeout_seconds` caps wall time.\n #\n # The values below are referenced by config.yaml as `$VAR`, resolved via\n # DeerFlow's `resolve_env_variables()` and coerced to int by Pydantic\n # (lax mode), so quoted strings here become real ints downstream.\n #\n # ─── WHEN TO ADJUST ─────────────────────────────────────────────────────────\n # Symptom → Action\n # \"Recursion limit of N reached\" in logs → raise the matching\n # *_MAX_TURNS\n # Subagent aborts with \"timed out\" → raise the matching\n # *_TIMEOUT_SECONDS\n # Local model burns CPU/$$ in obvious loops → lower *_MAX_TURNS to\n # surface failures fast\n #\n # ─── RECOMMENDED VALUES BY MODEL CLASS ──────────────────────────────────────\n # Model class gp_turns bash_turns\n # ────────────────────────────────────────────── ──────── ──────────\n # Strong frontier (Claude 4 / GPT-5 / Doubao 2) 150 80\n # Efficient MoE (Gemma 4 26B A4B) 200 100\n # Mid local (Qwen3 32B+ / DeepSeek V3.2 / Kimi 2.5) 300 150\n # Heavy local 30B-class (Qwen3.x 35B) ← default 500 300\n # Weak / heavily quantized (Qwen3 ≤ 14B, INT4) 1000 400\n #\n # gp_turns = SUBAGENT_GENERAL_PURPOSE_MAX_TURNS\n # bash_turns = SUBAGENT_BASH_MAX_TURNS\n # Image-baked builtin defaults: gp=500, bash=300 (already raised from\n # upstream's 100 / 60 in this fork to handle long delegated workflows).\n #\n # ─── HOW TO SIZE TIMEOUTS ───────────────────────────────────────────────────\n # Timeouts should be a safety net for hung/runaway processes — they should\n # NOT trigger before max_turns under normal operation, otherwise raising\n # max_turns has no effect. Rule of thumb:\n # timeout_seconds ≥ max_turns × realistic_seconds_per_turn\n #\n # Per-turn cost depends on model speed:\n # Strong cloud (Claude / GPT-5) 5-15 s/turn\n # Efficient MoE on Blackwell GPU 3-8 s/turn\n # 30B local on RTX 4090 Q4 15-30 s/turn\n # 30B local on CPU / iGPU 60-180 s/turn\n #\n # Defaults below assume 500 turns × 14 s/turn ≈ 2 h for general-purpose,\n # which covers cloud and mid-tier local GPUs. For pure-CPU or very slow\n # quantized local setups, push GENERAL_PURPOSE_TIMEOUT_SECONDS to 14400\n # (4 h) or more.\n #\n # ─── HOW TO APPLY CHANGES ───────────────────────────────────────────────────\n # 1. Edit the value below (or `kubectl edit configmap deerflowv2-env-config`).\n # 2. Restart the backend so the new env vars are picked up:\n # kubectl -n <namespace> rollout restart deploy/deerflowv2-backend\n # 3. Verify inside the pod:\n # kubectl -n <ns> exec deploy/deerflowv2-backend -c gateway -- \\\n # env | grep SUBAGENT_\n #\n # ─── CAVEATS ────────────────────────────────────────────────────────────────\n # - Values MUST stay quoted strings (e.g. \"500\", not 500). They are env vars.\n # - Do NOT delete these keys. `resolve_env_variables()` raises ValueError if\n # a `$VAR` reference can't be resolved, which crashes the pod on startup.\n # - For installs whose appData/config/config.yaml predates this chart (no\n # `subagents:` block, or hard-coded numbers): either edit it manually to\n # match the template below, OR delete it and let the init-container\n # regenerate from the template (this loses any custom `models:` you added,\n # so back it up first).\n # - Changes to this ConfigMap do NOT auto-restart the pod. The rollout\n # restart in step 2 is required.\n SUBAGENT_DEFAULT_TIMEOUT_SECONDS: \"1800\"\n # Global default max_turns. Per-agent values below override this, so it\n # only kicks in for any custom_agents the user defines without an explicit\n # max_turns override (e.g. a \"data-analysis\" subagent created via\n # subagents.custom_agents in config.yaml). Upstream builtin default is 50.\n SUBAGENT_DEFAULT_MAX_TURNS: \"200\"\n SUBAGENT_GENERAL_PURPOSE_MAX_TURNS: \"500\"\n SUBAGENT_GENERAL_PURPOSE_TIMEOUT_SECONDS: \"7200\"\n SUBAGENT_BASH_MAX_TURNS: \"300\"\n SUBAGENT_BASH_TIMEOUT_SECONDS: \"1800\"\n---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: deerflowv2-init-config\n namespace: {{ .Release.Namespace }}\ndata:\n config.yaml: |\n # Bumped to match upstream config.example.yaml (deer-flow main, May 2026).\n # The version number itself is purely a \"your config is outdated, run\n # config-upgrade\" warning trigger in deer-flow's startup log -- it does\n # not unlock or restrict any features.\n config_version: 31\n\n # Logger threshold for the `deerflow` and `app` loggers. Third-party\n # libraries (uvicorn, sqlalchemy, langgraph, ...) keep their own levels.\n # Use \"debug\" temporarily to surface tool selection / middleware decisions\n # when an agent misbehaves, then revert.\n log_level: info\n\n # Surface input/output/total token counts in the workspace UI when the\n # provider returns them. Helps spot a runaway Ultra session before it\n # eats a Tavily quota or burns local GPU time on a duplicate-dispatch.\n token_usage:\n enabled: true\n\n # =============================================\n # HOW TO ADD MODELS\n # =============================================\n # Edit this file at: appData/config/config.yaml\n #\n # Ollama example (use models you've pulled in Ollama):\n # - name: qwen3\n # display_name: Qwen3 (Ollama)\n # use: langchain_openai:ChatOpenAI\n # model: qwen3\n # api_key: ollama\n # base_url: $OLLAMA_OPENAI_BASE_URL\n # supports_thinking: true # enables Pro/Ultra modes (Qwen3+ supports thinking)\n # supports_vision: true # enables image input (only for vision models like Qwen2.5-VL)\n #\n # OpenAI example:\n # - name: gpt-4o\n # display_name: GPT-4o\n # use: langchain_openai:ChatOpenAI\n # model: gpt-4o\n # api_key: $OPENAI_API_KEY\n # supports_thinking: true\n # supports_vision: true\n #\n # Claude example:\n # - name: claude-sonnet\n # display_name: Claude 4 Sonnet\n # use: langchain_anthropic:ChatAnthropic\n # model: claude-sonnet-4-20250514\n # api_key: $ANTHROPIC_API_KEY\n # supports_thinking: true\n # supports_reasoning_effort: true # enables reasoning effort slider\n # supports_vision: true\n #\n # Key model capabilities:\n # supports_thinking: true/false - Pro/Ultra mode (thinking/planning before answering)\n # supports_reasoning_effort: true/false - reasoning effort slider (low/medium/high)\n # supports_vision: true/false - image/vision input support\n #\n # After editing, restart the app to apply changes.\n models: []\n\n tool_groups:\n - name: web\n - name: file:read\n - name: file:write\n - name: bash\n\n tools:\n - name: web_search\n group: web\n use: deerflow.community.tavily.tools:web_search_tool\n max_results: 5\n - name: web_fetch\n group: web\n use: deerflow.community.jina_ai.tools:web_fetch_tool\n timeout: 10\n - name: image_search\n group: web\n use: deerflow.community.image_search.tools:image_search_tool\n max_results: 5\n - name: ls\n group: file:read\n use: deerflow.sandbox.tools:ls_tool\n - name: read_file\n group: file:read\n use: deerflow.sandbox.tools:read_file_tool\n - name: write_file\n group: file:write\n use: deerflow.sandbox.tools:write_file_tool\n - name: str_replace\n group: file:write\n use: deerflow.sandbox.tools:str_replace_tool\n - name: bash\n group: bash\n use: deerflow.sandbox.tools:bash_tool\n\n tool_search:\n enabled: false\n\n # =============================================\n # UPLOADS\n # =============================================\n # All values are sourced from env vars in deerflowv2-env-config so\n # operators can resize without editing config.yaml. See that ConfigMap\n # for tuning notes. Sizes are bytes; the gateway also surfaces these\n # to the frontend so the file picker can reject too-large files before\n # they hit the wire.\n uploads:\n max_files: $UPLOADS_MAX_FILES\n max_file_size: $UPLOADS_MAX_FILE_SIZE\n max_total_size: $UPLOADS_MAX_TOTAL_SIZE\n # Office/PDF auto-conversion runs on the gateway host BEFORE the\n # sandbox isolates the file. Keep off unless uploads come from a\n # fully trusted source -- a malicious .docx can hit the parser before\n # any sandboxing applies. The image-ocr Skill handles user-driven\n # OCR explicitly via tesseract anyway.\n auto_convert_documents: false\n pdf_converter: auto\n\n sandbox:\n use: deerflow.sandbox.local:LocalSandboxProvider\n # Re-enable host bash for the LLM. Upstream PR #1547 (Security: Address\n # critical host-shell escape in LocalSandboxProvider) gates the `bash`\n # tool behind this flag, because LocalSandboxProvider executes commands\n # directly on the gateway pod's filesystem rather than inside an\n # isolated sandbox container. With the flag off, the LLM cannot\n # `pip install` or `apt-get install` runtime helpers (notably\n # pytesseract / tesseract-ocr for OCR), which silently breaks the\n # legacy \"ask the model to OCR an image\" workflow that worked before\n # we picked up the security patch.\n #\n # In a single-tenant Olares deployment the gateway pod is the trust\n # boundary anyway, so re-enabling this restores prior behavior without\n # widening exposure beyond what existed before. If you ever expose this\n # instance to untrusted users, switch to AioSandboxProvider (per-thread\n # container) instead and flip this back to false.\n allow_host_bash: true\n\n skills:\n container_path: /mnt/skills\n\n title:\n enabled: true\n max_words: 6\n max_chars: 60\n model_name: null\n\n summarization:\n enabled: true\n model_name: null\n trigger:\n - type: tokens\n value: 15564\n keep:\n type: messages\n value: 10\n trim_tokens_to_summarize: 15564\n summary_prompt: null\n # Recently-loaded skill files (e.g. the image-ocr SKILL.md the agent\n # just opened) are exempt from summarization, otherwise the agent\n # would lose its instructions mid-task. These are upstream defaults\n # pinned here so a future deer-flow update changing the defaults\n # cannot silently degrade the image-ocr workflow.\n preserve_recent_skill_count: 5\n preserve_recent_skill_tokens: 25000\n preserve_recent_skill_tokens_per_skill: 5000\n skill_file_read_tool_names:\n - read_file\n - read\n - view\n - cat\n\n memory:\n enabled: true\n storage_path: memory.json\n debounce_seconds: 30\n model_name: null\n max_facts: 100\n fact_confidence_threshold: 0.7\n injection_enabled: true\n max_injection_tokens: 2000\n\n # =============================================\n # PERSISTENCE (users, threads metadata, checkpoints, run events)\n # =============================================\n # The unified `database:` block drives EVERYTHING that needs durable\n # storage: gateway user store (users + sessions + tokens), thread\n # metadata, the LangGraph checkpointer, and the run-event journal\n # (token usage, tool-call traces). All routed to a single postgres\n # database provisioned automatically by the Olares postgres\n # middleware (declared in OlaresManifest.yaml as `middleware.postgres`).\n #\n # Connection string is built from .Values.postgres.* in deployment.yaml:\n # DATABASE_URL → SQLAlchemy ORM (asyncpg) + LangGraph checkpointer (psycopg)\n # Do not put libpq-only keepalive params in DATABASE_URL (asyncpg rejects them).\n # deer-flow resolves via AppConfig.resolve_env_variables() at startup.\n #\n # To switch back to sqlite (single-tenant, no postgres dependency):\n # 1. Remove the `middleware.postgres` block from OlaresManifest.yaml\n # 2. Remove the DATABASE_URL env entries from deployment.yaml\n # 3. Replace the block below with:\n # database:\n # backend: sqlite\n # sqlite_dir: /data/deer-flow/db\n # 4. Recreate the appData/config/config.yaml or run the migration\n # script in reverse.\n database:\n backend: postgres\n postgres_url: $DATABASE_URL\n\n # =============================================\n # SUBAGENT LIMITS\n # =============================================\n # All values resolved from env vars defined in the deerflowv2-env-config\n # ConfigMap — see that ConfigMap's comments for tuning guidance,\n # recommended values per model class, and how to apply changes.\n # Tune the env vars there, NOT the literals below.\n subagents:\n timeout_seconds: $SUBAGENT_DEFAULT_TIMEOUT_SECONDS\n max_turns: $SUBAGENT_DEFAULT_MAX_TURNS\n agents:\n general-purpose:\n timeout_seconds: $SUBAGENT_GENERAL_PURPOSE_TIMEOUT_SECONDS\n max_turns: $SUBAGENT_GENERAL_PURPOSE_MAX_TURNS\n bash:\n timeout_seconds: $SUBAGENT_BASH_TIMEOUT_SECONDS\n max_turns: $SUBAGENT_BASH_MAX_TURNS\n\n # =============================================\n # RUN EVENTS (message + tool-call telemetry)\n # =============================================\n # Upstream defaults this to \"memory\", which means every conversation's\n # full execution trace (subagent dispatches, tool invocations, token\n # counts, errors) is kept only in process RAM and lost on pod restart.\n # That makes it impossible to audit \"why did the lead burn 15 minutes\n # last night\" after the fact, and it makes the per-run token-usage\n # accounting in the UI reset on every restart.\n #\n # We already run sqlite for the user store + checkpointer, so flipping\n # run_events to the same backend is essentially free -- traces land in\n # the same deerflow.db file. Still bounded by max_trace_content (10 KiB\n # per individual trace blob) so a runaway tool output cannot bloat the\n # DB indefinitely.\n run_events:\n backend: db\n max_trace_content: 10240\n track_token_usage: true\n\n # =============================================\n # CIRCUIT BREAKER (LLM provider failure isolation)\n # =============================================\n # When the upstream model provider (Ollama, vLLM, OpenAI, ...) returns\n # 5 consecutive failures, fast-fail subsequent calls for 60 seconds\n # instead of letting LangGraph retry-loops hammer a dying provider.\n # Defaults below match upstream and work well for both local Ollama\n # (where 5 timeouts means the model is genuinely stuck) and cloud APIs\n # (where 5 failures usually means rate limit / outage). Pinned here so\n # the values are visible to operators rather than buried in code.\n circuit_breaker:\n failure_threshold: 5\n recovery_timeout_sec: 60\n\n extensions_config.json: |\n {\n \"mcpServers\": {},\n \"skills\": {}\n }\n\n # ============================================================================\n # MIGRATION SCRIPT: sqlite -> postgres database backend\n # ============================================================================\n # Mounted at /init-config/migrate_database_to_postgres.py and invoked by the\n # init-container in deployment.yaml. Idempotently rewrites the `database:`\n # block in an existing /config/config.yaml (the operator's persisted copy\n # on the appData hostPath) so chart upgrades flip the backend in place\n # without losing user-edited fields like `models:`.\n #\n # Why python instead of sed/awk: top-level YAML block detection has to\n # respect indentation, comment lines, and key order. A regex one-liner\n # gets it wrong on configs that operators have hand-edited, and the\n # failure mode (silently appending a duplicate `database:` block, then\n # PyYAML picks one of the two non-deterministically) is hard to debug.\n # The deer-flow image ships python3.12, so this is free.\n migrate_database_to_postgres.py: |\n \"\"\"Rewrite database:/checkpointer: blocks in config.yaml to use postgres.\n\n Usage: python3 migrate_database_to_postgres.py /path/to/config.yaml\n \"\"\"\n from __future__ import annotations\n\n import re\n import sys\n from pathlib import Path\n\n POSTGRES_BLOCK = (\n \"\\n\"\n \"# Auto-injected by init-container (chart >= 1.0.9). The Olares\\n\"\n \"# `middleware.postgres` provisioner exposes connection details via\\n\"\n \"# DATABASE_URL (SQLAlchemy + LangGraph checkpointer). To switch\\n\"\n \"# back to sqlite, remove the postgres middleware block from\\n\"\n \"# OlaresManifest.yaml and edit this section by hand.\\n\"\n \"database:\\n\"\n \" backend: postgres\\n\"\n \" postgres_url: $DATABASE_URL\\n\"\n )\n\n\n def strip_checkpointer_postgres_url(cfg: Path) -> None:\n \"\"\"Remove dual-URL leftover from chart < 1.0.7.\"\"\"\n text = cfg.read_text(encoding=\"utf-8\")\n if \"checkpointer_postgres_url:\" not in text:\n return\n lines = [\n ln for ln in text.splitlines(keepends=True)\n if \"checkpointer_postgres_url:\" not in ln\n ]\n cfg.write_text(\"\".join(lines), encoding=\"utf-8\")\n print(f\"migrate: removed checkpointer_postgres_url from {cfg}\")\n\n # Match a top-level YAML block whose key is `database` or the legacy\n # upstream `checkpointer`. The body is any consecutive run of indented\n # or blank lines following the key. Stops at the next top-level key\n # (a non-whitespace character at column 0) or end-of-file.\n BLOCK_RE = re.compile(\n r\"^(?:database|checkpointer):[ \\t]*\\n(?:[ \\t]+[^\\n]*\\n|[ \\t]*\\n)*\",\n re.MULTILINE,\n )\n\n\n def main(path: str) -> int:\n cfg = Path(path)\n if not cfg.exists():\n print(f\"migrate: {cfg} does not exist; nothing to do\")\n return 0\n\n original = cfg.read_text(encoding=\"utf-8\")\n # Drop ALL existing database:/checkpointer: blocks. There may be\n # more than one if a previous chart bug appended duplicates.\n stripped = BLOCK_RE.sub(\"\", original)\n # Trim trailing whitespace before re-appending so the final file\n # is not padded with blank lines after repeated runs.\n new = stripped.rstrip() + \"\\n\" + POSTGRES_BLOCK\n\n if new == original:\n print(f\"migrate: {cfg} already on postgres backend; no change\")\n strip_checkpointer_postgres_url(cfg)\n return 0\n\n cfg.write_text(new, encoding=\"utf-8\")\n print(f\"migrate: rewrote {cfg} database block -> postgres\")\n strip_checkpointer_postgres_url(cfg)\n return 0\n\n\n if __name__ == \"__main__\":\n sys.exit(main(sys.argv[1] if len(sys.argv) > 1 else \"/config/config.yaml\"))\n\n # ============================================================================\n # MIGRATION SCRIPT: legacy ephemeral SQLite -> postgres conversation history\n # ============================================================================\n # Mounted into the *gateway* container at /migrate (see deployment.yaml).\n # Run on demand by the operator AFTER:\n # 1. Upgrading to chart >= 1.1.4 (postgres backend + this script)\n # 2. Visiting /setup in the browser to create the first admin account\n #\n # kubectl -n <ns> exec -it deploy/deerflowv2-backend -c gateway -- \\\n # sh /migrate/migrate_legacy_sqlite.sh <admin_email>\n #\n # The script reads /data/deer-flow/checkpoints.db read-only, looks up the\n # admin user_id from the new PG `users` table, and writes every checkpoint\n # + pending write into PostgreSQL via AsyncPostgresSaver, stamping\n # metadata.user_id = <admin> so the migrated threads show up in the admin's\n # sidebar (langgraph_auth.add_owner_filter scopes the thread list to\n # metadata.user_id == ctx.user.identity).\n #\n # Idempotent: re-running upserts the same rows. Source SQLite is opened\n # read-only and never modified -- safe to run as many times as needed.\n # Mounted via items: so only these two files appear under /migrate, the\n # rest of init-config (config.yaml etc.) does not leak into the runtime\n # filesystem.\n migrate_legacy_sqlite.sh: |\n #!/bin/sh\n # Convenience wrapper around migrate_legacy_sqlite.py. Sets up PYTHONPATH\n # and uv so the script can import deerflow.* modules from the running\n # gateway image without the operator having to remember the venv layout.\n set -e\n\n if [ \"$#\" -lt 1 ]; then\n cat <<'EOF'\n Usage: sh /migrate/migrate_legacy_sqlite.sh <admin_email> [--src PATH] [--dry-run]\n\n Examples:\n sh /migrate/migrate_legacy_sqlite.sh admin@example.com\n sh /migrate/migrate_legacy_sqlite.sh admin@example.com --dry-run\n sh /migrate/migrate_legacy_sqlite.sh admin@example.com --src /data/deer-flow/db/checkpoints.db\n EOF\n exit 1\n fi\n\n ADMIN_EMAIL=\"$1\"\n shift\n\n cd /app/backend\n exec env PYTHONPATH=. uv run python /migrate/migrate_legacy_sqlite.py \\\n --admin-email \"$ADMIN_EMAIL\" \"$@\"\n\n migrate_legacy_sqlite.py: |\n \"\"\"Migrate legacy SQLite checkpointer data to the new PostgreSQL backend.\n\n Run inside the gateway container after upgrading to chart >= 1.1.4 and\n completing /setup (so an admin user exists in PG):\n\n kubectl -n <ns> exec -it deploy/deerflowv2-backend -c gateway -- \\\n sh /migrate/migrate_legacy_sqlite.sh <admin_email>\n\n Two-phase migration. Both phases run in one invocation:\n\n Phase 1 -- LangGraph checkpoints\n Opens /data/deer-flow/checkpoints.db read-only via AsyncSqliteSaver\n and copies every checkpoint + pending write into PostgreSQL via\n AsyncPostgresSaver, stamping metadata.user_id = <admin user_id> so\n langgraph_auth.add_owner_filter authorises the migrated threads.\n\n Phase 2 -- threads_meta rows (the sidebar list)\n DeerFlow's gateway lists conversations from a separate `threads_meta`\n SQL table -- distinct from LangGraph's checkpoints schema. Without\n a row here a migrated thread is invisible in the UI even though the\n underlying checkpoint exists. For each thread_id seen in phase 1,\n upsert a row with display_name = first user message (60 chars),\n user_id = admin, created/updated_at = first/last checkpoint ts.\n\n Idempotent on both phases: re-running upserts the same checkpoints and\n only inserts threads_meta rows that don't already exist (or backfills\n user_id / display_name on rows whose owner is NULL). The source SQLite\n file is opened read-only and never modified.\n \"\"\"\n from __future__ import annotations\n\n import argparse\n import asyncio\n import logging\n import os\n from collections import defaultdict\n from datetime import UTC, datetime\n from pathlib import Path\n from typing import Any\n\n logging.basicConfig(\n level=logging.INFO,\n format=\"%(asctime)s %(levelname)s %(name)s %(message)s\",\n )\n log = logging.getLogger(\"migrate\")\n\n DEFAULT_SQLITE = \"/data/deer-flow/checkpoints.db\"\n DISPLAY_NAME_MAX_LEN = 60\n\n\n # ------------------------------------------------------------------ helpers\n\n\n def _parse_iso(value: Any) -> datetime | None:\n \"\"\"Parse the LangGraph checkpoint `ts` field into an aware datetime.\n\n Checkpoints written by older LangGraph versions sometimes carry the\n `ts` as a millisecond integer; newer ones use ISO-8601 strings.\n Returns None when the value is missing or unparsable so the caller\n can fall back to `datetime.now(UTC)` rather than crashing the whole\n thread-meta backfill on one bad row.\n \"\"\"\n if not value:\n return None\n if isinstance(value, datetime):\n return value if value.tzinfo else value.replace(tzinfo=UTC)\n if isinstance(value, (int, float)):\n try:\n return datetime.fromtimestamp(float(value), UTC)\n except (OSError, OverflowError, ValueError):\n return None\n if isinstance(value, str):\n s = value.replace(\"Z\", \"+00:00\")\n try:\n dt = datetime.fromisoformat(s)\n except ValueError:\n return None\n return dt if dt.tzinfo else dt.replace(tzinfo=UTC)\n return None\n\n\n def _extract_first_user_text(checkpoint: dict) -> str | None:\n \"\"\"Find the first human/user message text in a checkpoint state.\n\n LangGraph deserialises message channels into LangChain Message\n objects (HumanMessage, AIMessage, ...) with `.type` and `.content`,\n but very old checkpoints may contain raw dicts instead. Handle\n both. Multimodal `content` (list of part dicts) is flattened to\n its concatenated text parts so an image-only first turn does not\n produce an empty title.\n \"\"\"\n channel_values = checkpoint.get(\"channel_values\") or {}\n messages = channel_values.get(\"messages\") or []\n for msg in messages:\n mtype = getattr(msg, \"type\", None) or (\n msg.get(\"type\") if isinstance(msg, dict) else None\n )\n if mtype not in {\"human\", \"user\"}:\n continue\n content = getattr(msg, \"content\", None)\n if content is None and isinstance(msg, dict):\n content = msg.get(\"content\")\n if isinstance(content, str):\n text = content.strip()\n elif isinstance(content, list):\n parts = []\n for part in content:\n if isinstance(part, dict) and part.get(\"type\") == \"text\":\n parts.append(str(part.get(\"text\", \"\")))\n elif isinstance(part, str):\n parts.append(part)\n text = \" \".join(p for p in parts if p).strip()\n else:\n text = \"\"\n if text:\n return text[:DISPLAY_NAME_MAX_LEN]\n return None\n\n\n # ----------------------------------------------------------------- phase 1\n\n\n async def _lookup_user_id(email: str) -> str:\n \"\"\"Resolve an admin email to its user_id row in the new PG users table.\"\"\"\n from sqlalchemy import select\n\n from deerflow.persistence.engine import get_session_factory\n from deerflow.persistence.user.model import UserRow\n\n sf = get_session_factory()\n if sf is None:\n raise SystemExit(\n \"session factory is None -- engine failed to initialise. \"\n \"Is DATABASE_URL set inside this pod?\"\n )\n async with sf() as session:\n row = (\n await session.execute(select(UserRow).where(UserRow.email == email))\n ).scalar_one_or_none()\n if row is None:\n raise SystemExit(\n f\"No user with email {email!r} in PG. Did you complete /setup?\"\n )\n return str(row.id)\n\n\n async def _migrate_checkpoints(\n src_path: str, target_user_id: str, dry_run: bool\n ) -> dict[str, dict]:\n \"\"\"Phase 1: copy every SQLite checkpoint into PG.\n\n Returns a per-thread summary {thread_id: {\"created\": dt, \"updated\": dt,\n \"display_name\": str|None}} so phase 2 can upsert threads_meta rows\n without re-reading the source file.\n \"\"\"\n from langgraph.checkpoint.postgres.aio import AsyncPostgresSaver\n from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver\n\n # file: URI with mode=ro guarantees we never write to the source DB,\n # even if AsyncSqliteSaver.setup() were to attempt a CREATE TABLE.\n src_uri = f\"file:{src_path}?mode=ro\"\n pg_url = (\n os.environ.get(\"CHECKPOINTER_DATABASE_URL\")\n or os.environ.get(\"DATABASE_URL\")\n )\n if not pg_url:\n raise SystemExit(\"CHECKPOINTER_DATABASE_URL / DATABASE_URL not set\")\n\n log.info(\"phase 1: LangGraph checkpoints\")\n log.info(\" source : %s (read-only)\", src_path)\n log.info(\" target : postgres (CHECKPOINTER_DATABASE_URL)\")\n log.info(\" user_id: %s\", target_user_id)\n if dry_run:\n log.info(\" DRY RUN -- nothing will be written to PG\")\n\n threads_info: dict[str, dict] = {}\n cp_count = 0\n pw_count = 0\n skipped = 0\n\n async with AsyncSqliteSaver.from_conn_string(src_uri) as src:\n async with AsyncPostgresSaver.from_conn_string(pg_url) as dst:\n if not dry_run:\n await dst.setup()\n\n # Passing config=None enumerates every checkpoint across every\n # thread. Iteration order is by checkpoint_id descending so the\n # *first* checkpoint we see for a given thread is the latest\n # one -- important for picking up the most recent user message\n # as the title fallback.\n async for tup in src.alist(None):\n try:\n config = dict(tup.config)\n config[\"configurable\"] = dict(config.get(\"configurable\", {}))\n thread_id = config[\"configurable\"].get(\"thread_id\", \"<no-thread>\")\n\n metadata = dict(tup.metadata or {})\n metadata[\"user_id\"] = target_user_id\n\n checkpoint = tup.checkpoint\n new_versions = dict(checkpoint.get(\"channel_versions\") or {})\n\n if dry_run:\n new_config = config\n else:\n new_config = await dst.aput(\n config, checkpoint, metadata, new_versions\n )\n\n cp_count += 1\n\n # Phase-2 summary -- updated on every checkpoint, but\n # `display_name` only gets *set* on the earliest user\n # message we encounter (which, given the descending\n # iteration, is updated until we hit step 0).\n ts = _parse_iso(checkpoint.get(\"ts\"))\n info = threads_info.setdefault(\n thread_id,\n {\"created\": ts, \"updated\": ts, \"display_name\": None},\n )\n if ts is not None:\n if info[\"created\"] is None or ts < info[\"created\"]:\n info[\"created\"] = ts\n if info[\"updated\"] is None or ts > info[\"updated\"]:\n info[\"updated\"] = ts\n text = _extract_first_user_text(checkpoint)\n if text:\n # Overwrite so we end up with the title from the\n # *earliest* checkpoint (chronologically the user's\n # opening question), not whichever step happened to\n # be enumerated first.\n info[\"display_name\"] = text\n\n if tup.pending_writes:\n grouped: dict[str, list[tuple[str, object]]] = defaultdict(list)\n for task_id, channel, value in tup.pending_writes:\n grouped[task_id].append((channel, value))\n for task_id, writes in grouped.items():\n if not dry_run:\n await dst.aput_writes(new_config, writes, task_id)\n pw_count += len(writes)\n\n if cp_count % 100 == 0:\n log.info(\n \" ...%d checkpoints, %d pending writes, %d threads so far\",\n cp_count, pw_count, len(threads_info),\n )\n except Exception as exc:\n # One bad row should not abort a 15 MB migration.\n skipped += 1\n log.warning(\" skipped checkpoint due to error: %s\", exc)\n\n log.info(\n \" done: %d checkpoints, %d pending writes, %d threads, %d skipped\",\n cp_count, pw_count, len(threads_info), skipped,\n )\n return threads_info\n\n\n # ----------------------------------------------------------------- phase 2\n\n\n async def _backfill_thread_meta(\n target_user_id: str,\n threads_info: dict[str, dict],\n dry_run: bool,\n ) -> None:\n \"\"\"Phase 2: insert/update one threads_meta row per migrated thread.\n\n Without this, /api/threads/search returns nothing for the migrated\n threads and the sidebar stays empty even though the checkpoints are\n in PG. Idempotent: existing rows whose user_id is None get adopted\n by the admin; existing rows already owned by someone are left alone.\n \"\"\"\n from deerflow.persistence.engine import get_session_factory\n from deerflow.persistence.thread_meta.model import ThreadMetaRow\n\n sf = get_session_factory()\n if sf is None:\n raise SystemExit(\"session factory is None\")\n\n log.info(\"phase 2: threads_meta backfill (sidebar entries)\")\n if dry_run:\n log.info(\" DRY RUN -- nothing will be written to PG\")\n for tid, info in list(threads_info.items())[:5]:\n log.info(\n \" would upsert thread=%s title=%r updated=%s\",\n tid,\n info.get(\"display_name\") or \"Untitled (migrated)\",\n info.get(\"updated\"),\n )\n log.info(\" ...and %d more thread(s)\", max(0, len(threads_info) - 5))\n return\n\n inserted = 0\n adopted = 0\n skipped = 0\n now = datetime.now(UTC)\n\n async with sf() as session:\n for thread_id, info in threads_info.items():\n try:\n existing = await session.get(ThreadMetaRow, thread_id)\n if existing is None:\n row = ThreadMetaRow(\n thread_id=thread_id,\n user_id=target_user_id,\n display_name=info.get(\"display_name\")\n or \"Untitled (migrated)\",\n status=\"idle\",\n metadata_json={\"migrated_from\": \"legacy_sqlite\"},\n created_at=info.get(\"created\") or now,\n updated_at=info.get(\"updated\") or now,\n )\n session.add(row)\n inserted += 1\n elif existing.user_id is None:\n # Pre-existing row from before auth was wired up.\n # Adopt it for the admin and fill in any missing\n # cosmetic fields.\n existing.user_id = target_user_id\n if not existing.display_name:\n existing.display_name = (\n info.get(\"display_name\") or \"Untitled (migrated)\"\n )\n existing.updated_at = info.get(\"updated\") or now\n adopted += 1\n else:\n # Already owned -- never silently steal a thread\n # that's been claimed by some other user (or even\n # the same admin, in which case the row is fine).\n skipped += 1\n except Exception as exc:\n log.warning(\n \" skipped thread_meta for %s due to error: %s\",\n thread_id, exc,\n )\n await session.commit()\n\n log.info(\n \" done: %d inserted, %d adopted (user_id was NULL), %d already owned\",\n inserted, adopted, skipped,\n )\n\n\n # --------------------------------------------------------------- driver\n\n\n async def _run(args: argparse.Namespace) -> None:\n from deerflow.config.app_config import get_app_config\n from deerflow.persistence.engine import close_engine, init_engine_from_config\n\n app_config = get_app_config()\n await init_engine_from_config(app_config.database)\n try:\n target_user_id = await _lookup_user_id(args.admin_email)\n log.info(\"resolved admin user_id: %s\", target_user_id)\n threads_info = await _migrate_checkpoints(\n args.src, target_user_id, args.dry_run\n )\n await _backfill_thread_meta(target_user_id, threads_info, args.dry_run)\n finally:\n await close_engine()\n\n log.info(\"=\" * 60)\n log.info(\"migration complete -- refresh the DeerFlow UI to see the\")\n log.info(\"migrated conversations in the admin sidebar.\")\n log.info(\"Safe to re-run if you upload a fresh sqlite dump.\")\n log.info(\"=\" * 60)\n\n\n def main() -> int:\n p = argparse.ArgumentParser(description=__doc__)\n p.add_argument(\n \"--admin-email\",\n required=True,\n help=\"Email of the admin user created via /setup\",\n )\n p.add_argument(\n \"--src\",\n default=DEFAULT_SQLITE,\n help=f\"Legacy SQLite path (default: {DEFAULT_SQLITE})\",\n )\n p.add_argument(\n \"--dry-run\",\n action=\"store_true\",\n help=\"Iterate the source DB but do not write anything to PG\",\n )\n args = p.parse_args()\n\n if not Path(args.src).exists():\n log.warning(\n \"source SQLite not found at %s -- nothing to migrate. \"\n \"Pass --src <path> if your file lives elsewhere.\",\n args.src,\n )\n return 0\n\n asyncio.run(_run(args))\n return 0\n\n\n if __name__ == \"__main__\":\n raise SystemExit(main())\n\n # ============================================================================\n # SKILL: image-ocr -- canonical OCR workflow for uploaded images\n # ============================================================================\n # deer-flow's <uploaded_files> middleware nudges the model toward read_file /\n # grep / glob (text-file affordances) regardless of whether the upload is\n # actually an image. Without further guidance the lead agent tends to either\n # (a) hallucinate the image contents, or (b) reach for a heavyweight OCR\n # stack like paddleocr / easyocr that downloads ~600 MB and blows past the\n # 600s subprocess.run timeout in LocalSandbox.execute_command.\n #\n # Skills are deer-flow's first-class extension point for this kind of fix:\n # any SKILL.md placed under <skills_root>/custom/<name>/ is auto-enabled\n # (extensions_config.is_skill_enabled defaults to True for the \"custom\"\n # category), and its `description` frontmatter is injected into the lead\n # agent system prompt as <available_skills>. When the user query matches\n # the description, the agent calls read_file on the SKILL.md to load the\n # full instructions, then follows them verbatim.\n #\n # The init-container copies this skill to /skills/custom/image-ocr/SKILL.md\n # on every pod start, so chart bumps push updated skill content. Operators\n # who want a different policy can edit the file directly, then bump the\n # `name` in the frontmatter so init does not overwrite it; or disable the\n # skill from the Olares extensions UI.\n image-ocr-skill.md: |\n ---\n name: image-ocr\n description: >-\n Extract text from an uploaded image. Trigger when the user asks to OCR\n an image, read text from a picture, recognize characters, transcribe a\n screenshot, identify what is written on a poster / receipt / business\n card / handwritten note, or extract any textual content from an image\n file. Common phrasings include \"OCR this image\", \"OCR it\", \"extract\n text from screenshot\", \"transcribe this image\", \"what does this image\n say\", \"read the text in this picture\", \"recognize the characters in\n this photo\", and equivalent requests in any other language (the model\n should match by intent, not literal wording). Trigger whenever a .png,\n .jpg, .jpeg, .webp, or .heic file is uploaded and the user wants the\n textual content rather than a visual description.\n ---\n\n # Image OCR\n\n Canonical, fast path for extracting text from an uploaded image. Follow\n this strictly to avoid the two failure modes operators have reported:\n\n 1. Hallucinating image contents instead of actually reading the file.\n 2. Picking heavyweight OCR stacks (paddleocr, easyocr, mmocr, rapidocr)\n that download hundreds of megabytes and time out the bash tool.\n\n ## Decision tree\n\n ### Step 1 -- Try `view_image` first (best when available)\n\n Look at your own tool list. If `view_image` is present, call it directly:\n\n ```\n view_image(image_path=\"/mnt/user-data/uploads/<filename>\")\n ```\n\n This injects the actual image bytes into your context as a vision input.\n Read the text directly from what you see and return it. No installation,\n no sandbox call, sub-second result, exact characters preserved.\n\n If `view_image` is not in your tool list (the selected model has no\n vision capability) or the call returns an error, fall through to Step 2.\n\n ### Step 2 -- pytesseract (only if Step 1 unavailable)\n\n Install `pytesseract` plus the `tesseract` binary and the Chinese +\n English language packs in one shot. Use these exact commands; do NOT\n substitute paddleocr, easyocr, mmocr, rapidocr, or any other OCR stack:\n they are 10x to 100x larger than tesseract and routinely exceed the\n bash tool's 600s wall-clock timeout.\n\n ```bash\n which tesseract >/dev/null 2>&1 || (apt-get update -qq && apt-get install -y -qq tesseract-ocr tesseract-ocr-chi-sim tesseract-ocr-chi-tra tesseract-ocr-eng > /dev/null)\n python -c \"import pytesseract\" 2>/dev/null || pip install -q pytesseract Pillow -i https://pypi.tuna.tsinghua.edu.cn/simple\n ```\n\n Then run OCR. The `lang` argument matters; without it Chinese text comes\n back as garbage:\n\n ```bash\n python - <<'PY'\n import pytesseract\n from PIL import Image\n text = pytesseract.image_to_string(\n Image.open('/mnt/user-data/uploads/<filename>'),\n lang='chi_sim+chi_tra+eng',\n )\n print(text)\n PY\n ```\n\n Total wall time on a fresh sandbox: roughly 30 to 60 seconds for the\n first install (apt fetches plus tesseract data files), under 2 seconds\n for subsequent calls in the same conversation since both apt and pip\n are cached inside the gateway pod for its lifetime.\n\n ## Constraints\n\n - Do not invent text. If the image is unreadable or OCR returns\n nothing, say so explicitly. Do not guess at the contents.\n - Do not save the OCR output as a file under `/mnt/user-data/outputs`\n unless the user explicitly asks for a file deliverable. Just\n include the extracted text inline in your response.\n - Do not chain multiple OCR libraries in a single turn. Pick one path\n (Step 1 OR Step 2) and stick with it. If Step 2 fails, report the\n failure to the user instead of falling back to a third option.\n - For multi-page PDFs that have already been converted to images, run\n Step 2 once per page and concatenate the results in page order.\n"}, "8b8f6708e3bb7c44200022066c0f05e091c52726": {"t": 1785748956.6958275, "n": 47782, "h": "c3be56297dae673c", "c": "---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: deerflowv2-env-config\n namespace: {{ .Release.Namespace }}\ndata:\n # ──────────────────────────────────────────────────────────────────────────\n # System locale -- forces UTF-8 inside the slim Python container.\n # Upstream commit 82e7936d bakes these into the Dockerfile, but our v2.0.1\n # backend image was built before that commit. Without them, any path or\n # tool output containing non-ASCII characters (Chinese filenames, emoji\n # in markdown, etc.) raises UnicodeEncodeError when Python opens the file\n # or writes to stdout. Safe to leave on permanently.\n # ──────────────────────────────────────────────────────────────────────────\n LANG: \"C.UTF-8\"\n LC_ALL: \"C.UTF-8\"\n PYTHONIOENCODING: \"utf-8\"\n\n # ──────────────────────────────────────────────────────────────────────────\n # Web search & crawl provider keys.\n # Tavily / Jina / InfoQuest are the original three. Serper / Firecrawl /\n # Exa were added by upstream in 2026-Q2 (Serper PR #2630 in commit\n # 44ab21fc). Leave empty unless the corresponding tool is referenced in\n # config.yaml's `tools:` list -- empty values are fine, deer-flow only\n # raises if a $VAR reference resolves to nothing.\n # ──────────────────────────────────────────────────────────────────────────\n TAVILY_API_KEY: \"\"\n JINA_API_KEY: \"\"\n INFOQUEST_API_KEY: \"\"\n SERPER_API_KEY: \"\"\n FIRECRAWL_API_KEY: \"\"\n EXA_API_KEY: \"\"\n\n # ──────────────────────────────────────────────────────────────────────────\n # Model provider keys. Reference these via $VAR in config.yaml's `models:`\n # section. Empty values are safe; only models that actually use the key\n # will fail to register at startup.\n # ──────────────────────────────────────────────────────────────────────────\n OPENAI_API_KEY: \"\"\n DEEPSEEK_API_KEY: \"\"\n ANTHROPIC_API_KEY: \"\"\n GOOGLE_API_KEY: \"\"\n GEMINI_API_KEY: \"\"\n VOLCENGINE_API_KEY: \"\"\n NOVITA_API_KEY: \"\"\n MINIMAX_API_KEY: \"\"\n MOONSHOT_API_KEY: \"\"\n VLLM_API_KEY: \"\"\n\n # LangSmith tracing (set to \"true\" and fill LANGSMITH_API_KEY to enable)\n LANGCHAIN_TRACING_V2: \"false\"\n LANGSMITH_API_KEY: \"\"\n\n # ──────────────────────────────────────────────────────────────────────────\n # Upload limits. Default upstream caps are tuned for the docker-compose\n # demo (50 MiB per file, 100 MiB per request, 10 files); on Olares users\n # routinely drop in 4K screenshots, recordings, and multi-page PDFs that\n # exceed the per-file cap. Bumping the defaults to 100 MiB / 500 MiB / 20\n # is conservative for an at-home server -- raise further if you upload\n # large media. Sizes are bytes.\n # ──────────────────────────────────────────────────────────────────────────\n UPLOADS_MAX_FILES: \"20\"\n UPLOADS_MAX_FILE_SIZE: \"104857600\" # 100 MiB\n UPLOADS_MAX_TOTAL_SIZE: \"524288000\" # 500 MiB\n # ============================================================================\n # SUBAGENT LIMITS — tune to fix \"Recursion limit reached\" errors\n # ============================================================================\n # Each subagent (general-purpose, bash) runs as its own LangGraph instance.\n # `max_turns` caps how many LLM <-> tool roundtrips it can take before the\n # graph aborts with `GraphRecursionError`. `timeout_seconds` caps wall time.\n #\n # The values below are referenced by config.yaml as `$VAR`, resolved via\n # DeerFlow's `resolve_env_variables()` and coerced to int by Pydantic\n # (lax mode), so quoted strings here become real ints downstream.\n #\n # ─── WHEN TO ADJUST ─────────────────────────────────────────────────────────\n # Symptom → Action\n # \"Recursion limit of N reached\" in logs → raise the matching\n # *_MAX_TURNS\n # Subagent aborts with \"timed out\" → raise the matching\n # *_TIMEOUT_SECONDS\n # Local model burns CPU/$$ in obvious loops → lower *_MAX_TURNS to\n # surface failures fast\n #\n # ─── RECOMMENDED VALUES BY MODEL CLASS ──────────────────────────────────────\n # Model class gp_turns bash_turns\n # ────────────────────────────────────────────── ──────── ──────────\n # Strong frontier (Claude 4 / GPT-5 / Doubao 2) 150 80\n # Efficient MoE (Gemma 4 26B A4B) 200 100\n # Mid local (Qwen3 32B+ / DeepSeek V3.2 / Kimi 2.5) 300 150\n # Heavy local 30B-class (Qwen3.x 35B) ← default 500 300\n # Weak / heavily quantized (Qwen3 ≤ 14B, INT4) 1000 400\n #\n # gp_turns = SUBAGENT_GENERAL_PURPOSE_MAX_TURNS\n # bash_turns = SUBAGENT_BASH_MAX_TURNS\n # Image-baked builtin defaults: gp=500, bash=300 (already raised from\n # upstream's 100 / 60 in this fork to handle long delegated workflows).\n #\n # ─── HOW TO SIZE TIMEOUTS ───────────────────────────────────────────────────\n # Timeouts should be a safety net for hung/runaway processes — they should\n # NOT trigger before max_turns under normal operation, otherwise raising\n # max_turns has no effect. Rule of thumb:\n # timeout_seconds ≥ max_turns × realistic_seconds_per_turn\n #\n # Per-turn cost depends on model speed:\n # Strong cloud (Claude / GPT-5) 5-15 s/turn\n # Efficient MoE on Blackwell GPU 3-8 s/turn\n # 30B local on RTX 4090 Q4 15-30 s/turn\n # 30B local on CPU / iGPU 60-180 s/turn\n #\n # Defaults below assume 500 turns × 14 s/turn ≈ 2 h for general-purpose,\n # which covers cloud and mid-tier local GPUs. For pure-CPU or very slow\n # quantized local setups, push GENERAL_PURPOSE_TIMEOUT_SECONDS to 14400\n # (4 h) or more.\n #\n # ─── HOW TO APPLY CHANGES ───────────────────────────────────────────────────\n # 1. Edit the value below (or `kubectl edit configmap deerflowv2-env-config`).\n # 2. Restart the backend so the new env vars are picked up:\n # kubectl -n <namespace> rollout restart deploy/deerflowv2-backend\n # 3. Verify inside the pod:\n # kubectl -n <ns> exec deploy/deerflowv2-backend -c gateway -- \\\n # env | grep SUBAGENT_\n #\n # ─── CAVEATS ────────────────────────────────────────────────────────────────\n # - Values MUST stay quoted strings (e.g. \"500\", not 500). They are env vars.\n # - Do NOT delete these keys. `resolve_env_variables()` raises ValueError if\n # a `$VAR` reference can't be resolved, which crashes the pod on startup.\n # - For installs whose appData/config/config.yaml predates this chart (no\n # `subagents:` block, or hard-coded numbers): either edit it manually to\n # match the template below, OR delete it and let the init-container\n # regenerate from the template (this loses any custom `models:` you added,\n # so back it up first).\n # - Changes to this ConfigMap do NOT auto-restart the pod. The rollout\n # restart in step 2 is required.\n SUBAGENT_DEFAULT_TIMEOUT_SECONDS: \"1800\"\n # Global default max_turns. Per-agent values below override this, so it\n # only kicks in for any custom_agents the user defines without an explicit\n # max_turns override (e.g. a \"data-analysis\" subagent created via\n # subagents.custom_agents in config.yaml). Upstream builtin default is 50.\n SUBAGENT_DEFAULT_MAX_TURNS: \"200\"\n SUBAGENT_GENERAL_PURPOSE_MAX_TURNS: \"500\"\n SUBAGENT_GENERAL_PURPOSE_TIMEOUT_SECONDS: \"7200\"\n SUBAGENT_BASH_MAX_TURNS: \"300\"\n SUBAGENT_BASH_TIMEOUT_SECONDS: \"1800\"\n---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: deerflowv2-init-config\n namespace: {{ .Release.Namespace }}\ndata:\n config.yaml: |\n # Bumped to match upstream config.example.yaml (deer-flow main, May 2026).\n # The version number itself is purely a \"your config is outdated, run\n # config-upgrade\" warning trigger in deer-flow's startup log -- it does\n # not unlock or restrict any features.\n config_version: 8\n\n # Logger threshold for the `deerflow` and `app` loggers. Third-party\n # libraries (uvicorn, sqlalchemy, langgraph, ...) keep their own levels.\n # Use \"debug\" temporarily to surface tool selection / middleware decisions\n # when an agent misbehaves, then revert.\n log_level: info\n\n # Surface input/output/total token counts in the workspace UI when the\n # provider returns them. Helps spot a runaway Ultra session before it\n # eats a Tavily quota or burns local GPU time on a duplicate-dispatch.\n token_usage:\n enabled: true\n\n # =============================================\n # HOW TO ADD MODELS\n # =============================================\n # Edit this file at: appData/config/config.yaml\n #\n # Ollama example (use models you've pulled in Ollama):\n # - name: qwen3\n # display_name: Qwen3 (Ollama)\n # use: langchain_openai:ChatOpenAI\n # model: qwen3\n # api_key: ollama\n # base_url: $OLLAMA_OPENAI_BASE_URL\n # supports_thinking: true # enables Pro/Ultra modes (Qwen3+ supports thinking)\n # supports_vision: true # enables image input (only for vision models like Qwen2.5-VL)\n #\n # OpenAI example:\n # - name: gpt-4o\n # display_name: GPT-4o\n # use: langchain_openai:ChatOpenAI\n # model: gpt-4o\n # api_key: $OPENAI_API_KEY\n # supports_thinking: true\n # supports_vision: true\n #\n # Claude example:\n # - name: claude-sonnet\n # display_name: Claude 4 Sonnet\n # use: langchain_anthropic:ChatAnthropic\n # model: claude-sonnet-4-20250514\n # api_key: $ANTHROPIC_API_KEY\n # supports_thinking: true\n # supports_reasoning_effort: true # enables reasoning effort slider\n # supports_vision: true\n #\n # Key model capabilities:\n # supports_thinking: true/false - Pro/Ultra mode (thinking/planning before answering)\n # supports_reasoning_effort: true/false - reasoning effort slider (low/medium/high)\n # supports_vision: true/false - image/vision input support\n #\n # After editing, restart the app to apply changes.\n models: []\n\n tool_groups:\n - name: web\n - name: file:read\n - name: file:write\n - name: bash\n\n tools:\n - name: web_search\n group: web\n use: deerflow.community.tavily.tools:web_search_tool\n max_results: 5\n - name: web_fetch\n group: web\n use: deerflow.community.jina_ai.tools:web_fetch_tool\n timeout: 10\n - name: image_search\n group: web\n use: deerflow.community.image_search.tools:image_search_tool\n max_results: 5\n - name: ls\n group: file:read\n use: deerflow.sandbox.tools:ls_tool\n - name: read_file\n group: file:read\n use: deerflow.sandbox.tools:read_file_tool\n - name: write_file\n group: file:write\n use: deerflow.sandbox.tools:write_file_tool\n - name: str_replace\n group: file:write\n use: deerflow.sandbox.tools:str_replace_tool\n - name: bash\n group: bash\n use: deerflow.sandbox.tools:bash_tool\n\n tool_search:\n enabled: false\n\n # =============================================\n # UPLOADS\n # =============================================\n # All values are sourced from env vars in deerflowv2-env-config so\n # operators can resize without editing config.yaml. See that ConfigMap\n # for tuning notes. Sizes are bytes; the gateway also surfaces these\n # to the frontend so the file picker can reject too-large files before\n # they hit the wire.\n uploads:\n max_files: $UPLOADS_MAX_FILES\n max_file_size: $UPLOADS_MAX_FILE_SIZE\n max_total_size: $UPLOADS_MAX_TOTAL_SIZE\n # Office/PDF auto-conversion runs on the gateway host BEFORE the\n # sandbox isolates the file. Keep off unless uploads come from a\n # fully trusted source -- a malicious .docx can hit the parser before\n # any sandboxing applies. The image-ocr Skill handles user-driven\n # OCR explicitly via tesseract anyway.\n auto_convert_documents: false\n pdf_converter: auto\n\n sandbox:\n use: deerflow.sandbox.local:LocalSandboxProvider\n # Re-enable host bash for the LLM. Upstream PR #1547 (Security: Address\n # critical host-shell escape in LocalSandboxProvider) gates the `bash`\n # tool behind this flag, because LocalSandboxProvider executes commands\n # directly on the gateway pod's filesystem rather than inside an\n # isolated sandbox container. With the flag off, the LLM cannot\n # `pip install` or `apt-get install` runtime helpers (notably\n # pytesseract / tesseract-ocr for OCR), which silently breaks the\n # legacy \"ask the model to OCR an image\" workflow that worked before\n # we picked up the security patch.\n #\n # In a single-tenant Olares deployment the gateway pod is the trust\n # boundary anyway, so re-enabling this restores prior behavior without\n # widening exposure beyond what existed before. If you ever expose this\n # instance to untrusted users, switch to AioSandboxProvider (per-thread\n # container) instead and flip this back to false.\n allow_host_bash: true\n\n skills:\n container_path: /mnt/skills\n\n title:\n enabled: true\n max_words: 6\n max_chars: 60\n model_name: null\n\n summarization:\n enabled: true\n model_name: null\n trigger:\n - type: tokens\n value: 15564\n keep:\n type: messages\n value: 10\n trim_tokens_to_summarize: 15564\n summary_prompt: null\n # Recently-loaded skill files (e.g. the image-ocr SKILL.md the agent\n # just opened) are exempt from summarization, otherwise the agent\n # would lose its instructions mid-task. These are upstream defaults\n # pinned here so a future deer-flow update changing the defaults\n # cannot silently degrade the image-ocr workflow.\n preserve_recent_skill_count: 5\n preserve_recent_skill_tokens: 25000\n preserve_recent_skill_tokens_per_skill: 5000\n skill_file_read_tool_names:\n - read_file\n - read\n - view\n - cat\n\n memory:\n enabled: true\n storage_path: memory.json\n debounce_seconds: 30\n model_name: null\n max_facts: 100\n fact_confidence_threshold: 0.7\n injection_enabled: true\n max_injection_tokens: 2000\n\n # =============================================\n # PERSISTENCE (users, threads metadata, checkpoints, run events)\n # =============================================\n # The unified `database:` block drives EVERYTHING that needs durable\n # storage: gateway user store (users + sessions + tokens), thread\n # metadata, the LangGraph checkpointer, and the run-event journal\n # (token usage, tool-call traces). All routed to a single postgres\n # database provisioned automatically by the Olares postgres\n # middleware (declared in OlaresManifest.yaml as `middleware.postgres`).\n #\n # Connection strings are built from .Values.postgres.* in deployment.yaml:\n # DATABASE_URL → SQLAlchemy ORM (asyncpg) — **no** libpq-only params\n # CHECKPOINTER_DATABASE_URL → LangGraph checkpointer (psycopg) — TCP keepalives OK\n # deer-flow resolves both via AppConfig.resolve_env_variables() at startup.\n #\n # To switch back to sqlite (single-tenant, no postgres dependency):\n # 1. Remove the `middleware.postgres` block from OlaresManifest.yaml\n # 2. Remove the DATABASE_URL env entries from deployment.yaml\n # 3. Replace the block below with:\n # database:\n # backend: sqlite\n # sqlite_dir: /data/deer-flow/db\n # 4. Recreate the appData/config/config.yaml or run the migration\n # script in reverse.\n database:\n backend: postgres\n postgres_url: $DATABASE_URL\n checkpointer_postgres_url: $CHECKPOINTER_DATABASE_URL\n\n # =============================================\n # SUBAGENT LIMITS\n # =============================================\n # All values resolved from env vars defined in the deerflowv2-env-config\n # ConfigMap — see that ConfigMap's comments for tuning guidance,\n # recommended values per model class, and how to apply changes.\n # Tune the env vars there, NOT the literals below.\n subagents:\n timeout_seconds: $SUBAGENT_DEFAULT_TIMEOUT_SECONDS\n max_turns: $SUBAGENT_DEFAULT_MAX_TURNS\n agents:\n general-purpose:\n timeout_seconds: $SUBAGENT_GENERAL_PURPOSE_TIMEOUT_SECONDS\n max_turns: $SUBAGENT_GENERAL_PURPOSE_MAX_TURNS\n bash:\n timeout_seconds: $SUBAGENT_BASH_TIMEOUT_SECONDS\n max_turns: $SUBAGENT_BASH_MAX_TURNS\n\n # =============================================\n # RUN EVENTS (message + tool-call telemetry)\n # =============================================\n # Upstream defaults this to \"memory\", which means every conversation's\n # full execution trace (subagent dispatches, tool invocations, token\n # counts, errors) is kept only in process RAM and lost on pod restart.\n # That makes it impossible to audit \"why did the lead burn 15 minutes\n # last night\" after the fact, and it makes the per-run token-usage\n # accounting in the UI reset on every restart.\n #\n # We already run sqlite for the user store + checkpointer, so flipping\n # run_events to the same backend is essentially free -- traces land in\n # the same deerflow.db file. Still bounded by max_trace_content (10 KiB\n # per individual trace blob) so a runaway tool output cannot bloat the\n # DB indefinitely.\n run_events:\n backend: db\n max_trace_content: 10240\n track_token_usage: true\n\n # =============================================\n # CIRCUIT BREAKER (LLM provider failure isolation)\n # =============================================\n # When the upstream model provider (Ollama, vLLM, OpenAI, ...) returns\n # 5 consecutive failures, fast-fail subsequent calls for 60 seconds\n # instead of letting LangGraph retry-loops hammer a dying provider.\n # Defaults below match upstream and work well for both local Ollama\n # (where 5 timeouts means the model is genuinely stuck) and cloud APIs\n # (where 5 failures usually means rate limit / outage). Pinned here so\n # the values are visible to operators rather than buried in code.\n circuit_breaker:\n failure_threshold: 5\n recovery_timeout_sec: 60\n\n extensions_config.json: |\n {\n \"mcpServers\": {},\n \"skills\": {}\n }\n\n # ============================================================================\n # MIGRATION SCRIPT: sqlite -> postgres database backend\n # ============================================================================\n # Mounted at /init-config/migrate_database_to_postgres.py and invoked by the\n # init-container in deployment.yaml. Idempotently rewrites the `database:`\n # block in an existing /config/config.yaml (the operator's persisted copy\n # on the appData hostPath) so chart upgrades flip the backend in place\n # without losing user-edited fields like `models:`.\n #\n # Why python instead of sed/awk: top-level YAML block detection has to\n # respect indentation, comment lines, and key order. A regex one-liner\n # gets it wrong on configs that operators have hand-edited, and the\n # failure mode (silently appending a duplicate `database:` block, then\n # PyYAML picks one of the two non-deterministically) is hard to debug.\n # The deer-flow image ships python3.12, so this is free.\n migrate_database_to_postgres.py: |\n \"\"\"Rewrite database:/checkpointer: blocks in config.yaml to use postgres.\n\n Usage: python3 migrate_database_to_postgres.py /path/to/config.yaml\n \"\"\"\n from __future__ import annotations\n\n import re\n import sys\n from pathlib import Path\n\n POSTGRES_BLOCK = (\n \"\\n\"\n \"# Auto-injected by init-container (chart >= 1.0.9). The Olares\\n\"\n \"# `middleware.postgres` provisioner exposes connection details via\\n\"\n \"# DATABASE_URL (asyncpg ORM) and CHECKPOINTER_DATABASE_URL (psycopg\\n\"\n \"# LangGraph saver with TCP keepalives). To switch back to sqlite,\\n\"\n \"# remove the postgres middleware block from OlaresManifest.yaml and\\n\"\n \"# edit this section by hand.\\n\"\n \"database:\\n\"\n \" backend: postgres\\n\"\n \" postgres_url: $DATABASE_URL\\n\"\n \" checkpointer_postgres_url: $CHECKPOINTER_DATABASE_URL\\n\"\n )\n\n\n def ensure_checkpointer_postgres_url(cfg: Path) -> None:\n \"\"\"Inject checkpointer_postgres_url for chart 1.0.11+ without full rewrite.\"\"\"\n text = cfg.read_text(encoding=\"utf-8\")\n if \"checkpointer_postgres_url:\" in text:\n return\n if \"backend: postgres\" not in text or \"postgres_url:\" not in text:\n return\n lines = text.splitlines(keepends=True)\n out: list[str] = []\n inserted = False\n for line in lines:\n out.append(line)\n if not inserted and re.match(r\"^[ \\t]+postgres_url:[ \\t]\", line):\n out.append(\" checkpointer_postgres_url: $CHECKPOINTER_DATABASE_URL\\n\")\n inserted = True\n if inserted:\n cfg.write_text(\"\".join(out), encoding=\"utf-8\")\n print(f\"migrate: injected checkpointer_postgres_url into {cfg}\")\n\n # Match a top-level YAML block whose key is `database` or the legacy\n # upstream `checkpointer`. The body is any consecutive run of indented\n # or blank lines following the key. Stops at the next top-level key\n # (a non-whitespace character at column 0) or end-of-file.\n BLOCK_RE = re.compile(\n r\"^(?:database|checkpointer):[ \\t]*\\n(?:[ \\t]+[^\\n]*\\n|[ \\t]*\\n)*\",\n re.MULTILINE,\n )\n\n\n def main(path: str) -> int:\n cfg = Path(path)\n if not cfg.exists():\n print(f\"migrate: {cfg} does not exist; nothing to do\")\n return 0\n\n original = cfg.read_text(encoding=\"utf-8\")\n # Drop ALL existing database:/checkpointer: blocks. There may be\n # more than one if a previous chart bug appended duplicates.\n stripped = BLOCK_RE.sub(\"\", original)\n # Trim trailing whitespace before re-appending so the final file\n # is not padded with blank lines after repeated runs.\n new = stripped.rstrip() + \"\\n\" + POSTGRES_BLOCK\n\n if new == original:\n print(f\"migrate: {cfg} already on postgres backend; no change\")\n ensure_checkpointer_postgres_url(cfg)\n return 0\n\n cfg.write_text(new, encoding=\"utf-8\")\n print(f\"migrate: rewrote {cfg} database block -> postgres\")\n ensure_checkpointer_postgres_url(cfg)\n return 0\n\n\n if __name__ == \"__main__\":\n sys.exit(main(sys.argv[1] if len(sys.argv) > 1 else \"/config/config.yaml\"))\n\n # ============================================================================\n # MIGRATION SCRIPT: legacy ephemeral SQLite -> postgres conversation history\n # ============================================================================\n # Mounted into the *gateway* container at /migrate (see deployment.yaml).\n # Run on demand by the operator AFTER:\n # 1. Upgrading to chart >= 1.1.4 (postgres backend + this script)\n # 2. Visiting /setup in the browser to create the first admin account\n #\n # kubectl -n <ns> exec -it deploy/deerflowv2-backend -c gateway -- \\\n # sh /migrate/migrate_legacy_sqlite.sh <admin_email>\n #\n # The script reads /data/deer-flow/checkpoints.db read-only, looks up the\n # admin user_id from the new PG `users` table, and writes every checkpoint\n # + pending write into PostgreSQL via AsyncPostgresSaver, stamping\n # metadata.user_id = <admin> so the migrated threads show up in the admin's\n # sidebar (langgraph_auth.add_owner_filter scopes the thread list to\n # metadata.user_id == ctx.user.identity).\n #\n # Idempotent: re-running upserts the same rows. Source SQLite is opened\n # read-only and never modified -- safe to run as many times as needed.\n # Mounted via items: so only these two files appear under /migrate, the\n # rest of init-config (config.yaml etc.) does not leak into the runtime\n # filesystem.\n migrate_legacy_sqlite.sh: |\n #!/bin/sh\n # Convenience wrapper around migrate_legacy_sqlite.py. Sets up PYTHONPATH\n # and uv so the script can import deerflow.* modules from the running\n # gateway image without the operator having to remember the venv layout.\n set -e\n\n if [ \"$#\" -lt 1 ]; then\n cat <<'EOF'\n Usage: sh /migrate/migrate_legacy_sqlite.sh <admin_email> [--src PATH] [--dry-run]\n\n Examples:\n sh /migrate/migrate_legacy_sqlite.sh admin@example.com\n sh /migrate/migrate_legacy_sqlite.sh admin@example.com --dry-run\n sh /migrate/migrate_legacy_sqlite.sh admin@example.com --src /data/deer-flow/db/checkpoints.db\n EOF\n exit 1\n fi\n\n ADMIN_EMAIL=\"$1\"\n shift\n\n cd /app/backend\n exec env PYTHONPATH=. uv run python /migrate/migrate_legacy_sqlite.py \\\n --admin-email \"$ADMIN_EMAIL\" \"$@\"\n\n migrate_legacy_sqlite.py: |\n \"\"\"Migrate legacy SQLite checkpointer data to the new PostgreSQL backend.\n\n Run inside the gateway container after upgrading to chart >= 1.1.4 and\n completing /setup (so an admin user exists in PG):\n\n kubectl -n <ns> exec -it deploy/deerflowv2-backend -c gateway -- \\\n sh /migrate/migrate_legacy_sqlite.sh <admin_email>\n\n Two-phase migration. Both phases run in one invocation:\n\n Phase 1 -- LangGraph checkpoints\n Opens /data/deer-flow/checkpoints.db read-only via AsyncSqliteSaver\n and copies every checkpoint + pending write into PostgreSQL via\n AsyncPostgresSaver, stamping metadata.user_id = <admin user_id> so\n langgraph_auth.add_owner_filter authorises the migrated threads.\n\n Phase 2 -- threads_meta rows (the sidebar list)\n DeerFlow's gateway lists conversations from a separate `threads_meta`\n SQL table -- distinct from LangGraph's checkpoints schema. Without\n a row here a migrated thread is invisible in the UI even though the\n underlying checkpoint exists. For each thread_id seen in phase 1,\n upsert a row with display_name = first user message (60 chars),\n user_id = admin, created/updated_at = first/last checkpoint ts.\n\n Idempotent on both phases: re-running upserts the same checkpoints and\n only inserts threads_meta rows that don't already exist (or backfills\n user_id / display_name on rows whose owner is NULL). The source SQLite\n file is opened read-only and never modified.\n \"\"\"\n from __future__ import annotations\n\n import argparse\n import asyncio\n import logging\n import os\n from collections import defaultdict\n from datetime import UTC, datetime\n from pathlib import Path\n from typing import Any\n\n logging.basicConfig(\n level=logging.INFO,\n format=\"%(asctime)s %(levelname)s %(name)s %(message)s\",\n )\n log = logging.getLogger(\"migrate\")\n\n DEFAULT_SQLITE = \"/data/deer-flow/checkpoints.db\"\n DISPLAY_NAME_MAX_LEN = 60\n\n\n # ------------------------------------------------------------------ helpers\n\n\n def _parse_iso(value: Any) -> datetime | None:\n \"\"\"Parse the LangGraph checkpoint `ts` field into an aware datetime.\n\n Checkpoints written by older LangGraph versions sometimes carry the\n `ts` as a millisecond integer; newer ones use ISO-8601 strings.\n Returns None when the value is missing or unparsable so the caller\n can fall back to `datetime.now(UTC)` rather than crashing the whole\n thread-meta backfill on one bad row.\n \"\"\"\n if not value:\n return None\n if isinstance(value, datetime):\n return value if value.tzinfo else value.replace(tzinfo=UTC)\n if isinstance(value, (int, float)):\n try:\n return datetime.fromtimestamp(float(value), UTC)\n except (OSError, OverflowError, ValueError):\n return None\n if isinstance(value, str):\n s = value.replace(\"Z\", \"+00:00\")\n try:\n dt = datetime.fromisoformat(s)\n except ValueError:\n return None\n return dt if dt.tzinfo else dt.replace(tzinfo=UTC)\n return None\n\n\n def _extract_first_user_text(checkpoint: dict) -> str | None:\n \"\"\"Find the first human/user message text in a checkpoint state.\n\n LangGraph deserialises message channels into LangChain Message\n objects (HumanMessage, AIMessage, ...) with `.type` and `.content`,\n but very old checkpoints may contain raw dicts instead. Handle\n both. Multimodal `content` (list of part dicts) is flattened to\n its concatenated text parts so an image-only first turn does not\n produce an empty title.\n \"\"\"\n channel_values = checkpoint.get(\"channel_values\") or {}\n messages = channel_values.get(\"messages\") or []\n for msg in messages:\n mtype = getattr(msg, \"type\", None) or (\n msg.get(\"type\") if isinstance(msg, dict) else None\n )\n if mtype not in {\"human\", \"user\"}:\n continue\n content = getattr(msg, \"content\", None)\n if content is None and isinstance(msg, dict):\n content = msg.get(\"content\")\n if isinstance(content, str):\n text = content.strip()\n elif isinstance(content, list):\n parts = []\n for part in content:\n if isinstance(part, dict) and part.get(\"type\") == \"text\":\n parts.append(str(part.get(\"text\", \"\")))\n elif isinstance(part, str):\n parts.append(part)\n text = \" \".join(p for p in parts if p).strip()\n else:\n text = \"\"\n if text:\n return text[:DISPLAY_NAME_MAX_LEN]\n return None\n\n\n # ----------------------------------------------------------------- phase 1\n\n\n async def _lookup_user_id(email: str) -> str:\n \"\"\"Resolve an admin email to its user_id row in the new PG users table.\"\"\"\n from sqlalchemy import select\n\n from deerflow.persistence.engine import get_session_factory\n from deerflow.persistence.user.model import UserRow\n\n sf = get_session_factory()\n if sf is None:\n raise SystemExit(\n \"session factory is None -- engine failed to initialise. \"\n \"Is DATABASE_URL set inside this pod?\"\n )\n async with sf() as session:\n row = (\n await session.execute(select(UserRow).where(UserRow.email == email))\n ).scalar_one_or_none()\n if row is None:\n raise SystemExit(\n f\"No user with email {email!r} in PG. Did you complete /setup?\"\n )\n return str(row.id)\n\n\n async def _migrate_checkpoints(\n src_path: str, target_user_id: str, dry_run: bool\n ) -> dict[str, dict]:\n \"\"\"Phase 1: copy every SQLite checkpoint into PG.\n\n Returns a per-thread summary {thread_id: {\"created\": dt, \"updated\": dt,\n \"display_name\": str|None}} so phase 2 can upsert threads_meta rows\n without re-reading the source file.\n \"\"\"\n from langgraph.checkpoint.postgres.aio import AsyncPostgresSaver\n from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver\n\n # file: URI with mode=ro guarantees we never write to the source DB,\n # even if AsyncSqliteSaver.setup() were to attempt a CREATE TABLE.\n src_uri = f\"file:{src_path}?mode=ro\"\n pg_url = (\n os.environ.get(\"CHECKPOINTER_DATABASE_URL\")\n or os.environ.get(\"DATABASE_URL\")\n )\n if not pg_url:\n raise SystemExit(\"CHECKPOINTER_DATABASE_URL / DATABASE_URL not set\")\n\n log.info(\"phase 1: LangGraph checkpoints\")\n log.info(\" source : %s (read-only)\", src_path)\n log.info(\" target : postgres (CHECKPOINTER_DATABASE_URL)\")\n log.info(\" user_id: %s\", target_user_id)\n if dry_run:\n log.info(\" DRY RUN -- nothing will be written to PG\")\n\n threads_info: dict[str, dict] = {}\n cp_count = 0\n pw_count = 0\n skipped = 0\n\n async with AsyncSqliteSaver.from_conn_string(src_uri) as src:\n async with AsyncPostgresSaver.from_conn_string(pg_url) as dst:\n if not dry_run:\n await dst.setup()\n\n # Passing config=None enumerates every checkpoint across every\n # thread. Iteration order is by checkpoint_id descending so the\n # *first* checkpoint we see for a given thread is the latest\n # one -- important for picking up the most recent user message\n # as the title fallback.\n async for tup in src.alist(None):\n try:\n config = dict(tup.config)\n config[\"configurable\"] = dict(config.get(\"configurable\", {}))\n thread_id = config[\"configurable\"].get(\"thread_id\", \"<no-thread>\")\n\n metadata = dict(tup.metadata or {})\n metadata[\"user_id\"] = target_user_id\n\n checkpoint = tup.checkpoint\n new_versions = dict(checkpoint.get(\"channel_versions\") or {})\n\n if dry_run:\n new_config = config\n else:\n new_config = await dst.aput(\n config, checkpoint, metadata, new_versions\n )\n\n cp_count += 1\n\n # Phase-2 summary -- updated on every checkpoint, but\n # `display_name` only gets *set* on the earliest user\n # message we encounter (which, given the descending\n # iteration, is updated until we hit step 0).\n ts = _parse_iso(checkpoint.get(\"ts\"))\n info = threads_info.setdefault(\n thread_id,\n {\"created\": ts, \"updated\": ts, \"display_name\": None},\n )\n if ts is not None:\n if info[\"created\"] is None or ts < info[\"created\"]:\n info[\"created\"] = ts\n if info[\"updated\"] is None or ts > info[\"updated\"]:\n info[\"updated\"] = ts\n text = _extract_first_user_text(checkpoint)\n if text:\n # Overwrite so we end up with the title from the\n # *earliest* checkpoint (chronologically the user's\n # opening question), not whichever step happened to\n # be enumerated first.\n info[\"display_name\"] = text\n\n if tup.pending_writes:\n grouped: dict[str, list[tuple[str, object]]] = defaultdict(list)\n for task_id, channel, value in tup.pending_writes:\n grouped[task_id].append((channel, value))\n for task_id, writes in grouped.items():\n if not dry_run:\n await dst.aput_writes(new_config, writes, task_id)\n pw_count += len(writes)\n\n if cp_count % 100 == 0:\n log.info(\n \" ...%d checkpoints, %d pending writes, %d threads so far\",\n cp_count, pw_count, len(threads_info),\n )\n except Exception as exc:\n # One bad row should not abort a 15 MB migration.\n skipped += 1\n log.warning(\" skipped checkpoint due to error: %s\", exc)\n\n log.info(\n \" done: %d checkpoints, %d pending writes, %d threads, %d skipped\",\n cp_count, pw_count, len(threads_info), skipped,\n )\n return threads_info\n\n\n # ----------------------------------------------------------------- phase 2\n\n\n async def _backfill_thread_meta(\n target_user_id: str,\n threads_info: dict[str, dict],\n dry_run: bool,\n ) -> None:\n \"\"\"Phase 2: insert/update one threads_meta row per migrated thread.\n\n Without this, /api/threads/search returns nothing for the migrated\n threads and the sidebar stays empty even though the checkpoints are\n in PG. Idempotent: existing rows whose user_id is None get adopted\n by the admin; existing rows already owned by someone are left alone.\n \"\"\"\n from deerflow.persistence.engine import get_session_factory\n from deerflow.persistence.thread_meta.model import ThreadMetaRow\n\n sf = get_session_factory()\n if sf is None:\n raise SystemExit(\"session factory is None\")\n\n log.info(\"phase 2: threads_meta backfill (sidebar entries)\")\n if dry_run:\n log.info(\" DRY RUN -- nothing will be written to PG\")\n for tid, info in list(threads_info.items())[:5]:\n log.info(\n \" would upsert thread=%s title=%r updated=%s\",\n tid,\n info.get(\"display_name\") or \"Untitled (migrated)\",\n info.get(\"updated\"),\n )\n log.info(\" ...and %d more thread(s)\", max(0, len(threads_info) - 5))\n return\n\n inserted = 0\n adopted = 0\n skipped = 0\n now = datetime.now(UTC)\n\n async with sf() as session:\n for thread_id, info in threads_info.items():\n try:\n existing = await session.get(ThreadMetaRow, thread_id)\n if existing is None:\n row = ThreadMetaRow(\n thread_id=thread_id,\n user_id=target_user_id,\n display_name=info.get(\"display_name\")\n or \"Untitled (migrated)\",\n status=\"idle\",\n metadata_json={\"migrated_from\": \"legacy_sqlite\"},\n created_at=info.get(\"created\") or now,\n updated_at=info.get(\"updated\") or now,\n )\n session.add(row)\n inserted += 1\n elif existing.user_id is None:\n # Pre-existing row from before auth was wired up.\n # Adopt it for the admin and fill in any missing\n # cosmetic fields.\n existing.user_id = target_user_id\n if not existing.display_name:\n existing.display_name = (\n info.get(\"display_name\") or \"Untitled (migrated)\"\n )\n existing.updated_at = info.get(\"updated\") or now\n adopted += 1\n else:\n # Already owned -- never silently steal a thread\n # that's been claimed by some other user (or even\n # the same admin, in which case the row is fine).\n skipped += 1\n except Exception as exc:\n log.warning(\n \" skipped thread_meta for %s due to error: %s\",\n thread_id, exc,\n )\n await session.commit()\n\n log.info(\n \" done: %d inserted, %d adopted (user_id was NULL), %d already owned\",\n inserted, adopted, skipped,\n )\n\n\n # --------------------------------------------------------------- driver\n\n\n async def _run(args: argparse.Namespace) -> None:\n from deerflow.config.app_config import get_app_config\n from deerflow.persistence.engine import close_engine, init_engine_from_config\n\n app_config = get_app_config()\n await init_engine_from_config(app_config.database)\n try:\n target_user_id = await _lookup_user_id(args.admin_email)\n log.info(\"resolved admin user_id: %s\", target_user_id)\n threads_info = await _migrate_checkpoints(\n args.src, target_user_id, args.dry_run\n )\n await _backfill_thread_meta(target_user_id, threads_info, args.dry_run)\n finally:\n await close_engine()\n\n log.info(\"=\" * 60)\n log.info(\"migration complete -- refresh the DeerFlow UI to see the\")\n log.info(\"migrated conversations in the admin sidebar.\")\n log.info(\"Safe to re-run if you upload a fresh sqlite dump.\")\n log.info(\"=\" * 60)\n\n\n def main() -> int:\n p = argparse.ArgumentParser(description=__doc__)\n p.add_argument(\n \"--admin-email\",\n required=True,\n help=\"Email of the admin user created via /setup\",\n )\n p.add_argument(\n \"--src\",\n default=DEFAULT_SQLITE,\n help=f\"Legacy SQLite path (default: {DEFAULT_SQLITE})\",\n )\n p.add_argument(\n \"--dry-run\",\n action=\"store_true\",\n help=\"Iterate the source DB but do not write anything to PG\",\n )\n args = p.parse_args()\n\n if not Path(args.src).exists():\n log.warning(\n \"source SQLite not found at %s -- nothing to migrate. \"\n \"Pass --src <path> if your file lives elsewhere.\",\n args.src,\n )\n return 0\n\n asyncio.run(_run(args))\n return 0\n\n\n if __name__ == \"__main__\":\n raise SystemExit(main())\n\n # ============================================================================\n # SKILL: image-ocr -- canonical OCR workflow for uploaded images\n # ============================================================================\n # deer-flow's <uploaded_files> middleware nudges the model toward read_file /\n # grep / glob (text-file affordances) regardless of whether the upload is\n # actually an image. Without further guidance the lead agent tends to either\n # (a) hallucinate the image contents, or (b) reach for a heavyweight OCR\n # stack like paddleocr / easyocr that downloads ~600 MB and blows past the\n # 600s subprocess.run timeout in LocalSandbox.execute_command.\n #\n # Skills are deer-flow's first-class extension point for this kind of fix:\n # any SKILL.md placed under <skills_root>/custom/<name>/ is auto-enabled\n # (extensions_config.is_skill_enabled defaults to True for the \"custom\"\n # category), and its `description` frontmatter is injected into the lead\n # agent system prompt as <available_skills>. When the user query matches\n # the description, the agent calls read_file on the SKILL.md to load the\n # full instructions, then follows them verbatim.\n #\n # The init-container copies this skill to /skills/custom/image-ocr/SKILL.md\n # on every pod start, so chart bumps push updated skill content. Operators\n # who want a different policy can edit the file directly, then bump the\n # `name` in the frontmatter so init does not overwrite it; or disable the\n # skill from the Olares extensions UI.\n image-ocr-skill.md: |\n ---\n name: image-ocr\n description: >-\n Extract text from an uploaded image. Trigger when the user asks to OCR\n an image, read text from a picture, recognize characters, transcribe a\n screenshot, identify what is written on a poster / receipt / business\n card / handwritten note, or extract any textual content from an image\n file. Common phrasings include \"OCR this image\", \"OCR it\", \"extract\n text from screenshot\", \"transcribe this image\", \"what does this image\n say\", \"read the text in this picture\", \"recognize the characters in\n this photo\", and equivalent requests in any other language (the model\n should match by intent, not literal wording). Trigger whenever a .png,\n .jpg, .jpeg, .webp, or .heic file is uploaded and the user wants the\n textual content rather than a visual description.\n ---\n\n # Image OCR\n\n Canonical, fast path for extracting text from an uploaded image. Follow\n this strictly to avoid the two failure modes operators have reported:\n\n 1. Hallucinating image contents instead of actually reading the file.\n 2. Picking heavyweight OCR stacks (paddleocr, easyocr, mmocr, rapidocr)\n that download hundreds of megabytes and time out the bash tool.\n\n ## Decision tree\n\n ### Step 1 -- Try `view_image` first (best when available)\n\n Look at your own tool list. If `view_image` is present, call it directly:\n\n ```\n view_image(image_path=\"/mnt/user-data/uploads/<filename>\")\n ```\n\n This injects the actual image bytes into your context as a vision input.\n Read the text directly from what you see and return it. No installation,\n no sandbox call, sub-second result, exact characters preserved.\n\n If `view_image` is not in your tool list (the selected model has no\n vision capability) or the call returns an error, fall through to Step 2.\n\n ### Step 2 -- pytesseract (only if Step 1 unavailable)\n\n Install `pytesseract` plus the `tesseract` binary and the Chinese +\n English language packs in one shot. Use these exact commands; do NOT\n substitute paddleocr, easyocr, mmocr, rapidocr, or any other OCR stack:\n they are 10x to 100x larger than tesseract and routinely exceed the\n bash tool's 600s wall-clock timeout.\n\n ```bash\n which tesseract >/dev/null 2>&1 || (apt-get update -qq && apt-get install -y -qq tesseract-ocr tesseract-ocr-chi-sim tesseract-ocr-chi-tra tesseract-ocr-eng > /dev/null)\n python -c \"import pytesseract\" 2>/dev/null || pip install -q pytesseract Pillow -i https://pypi.tuna.tsinghua.edu.cn/simple\n ```\n\n Then run OCR. The `lang` argument matters; without it Chinese text comes\n back as garbage:\n\n ```bash\n python - <<'PY'\n import pytesseract\n from PIL import Image\n text = pytesseract.image_to_string(\n Image.open('/mnt/user-data/uploads/<filename>'),\n lang='chi_sim+chi_tra+eng',\n )\n print(text)\n PY\n ```\n\n Total wall time on a fresh sandbox: roughly 30 to 60 seconds for the\n first install (apt fetches plus tesseract data files), under 2 seconds\n for subsequent calls in the same conversation since both apt and pip\n are cached inside the gateway pod for its lifetime.\n\n ## Constraints\n\n - Do not invent text. If the image is unreadable or OCR returns\n nothing, say so explicitly. Do not guess at the contents.\n - Do not save the OCR output as a file under `/mnt/user-data/outputs`\n unless the user explicitly asks for a file deliverable. Just\n include the extracted text inline in your response.\n - Do not chain multiple OCR libraries in a single turn. Pick one path\n (Step 1 OR Step 2) and stick with it. If Step 2 fails, report the\n failure to the user instead of falling back to a third option.\n - For multi-page PDFs that have already been converted to images, run\n Step 2 once per page and concatenate the results in page order.\n"}, "4ead52c67894de39a29bf92826fb839bf6b93fa2": {"t": 1785748957.9534316, "n": 28440, "h": "7aa9b2914eaddf5b", "c": "---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: deerflowv2-env-config\n namespace: {{ .Release.Namespace }}\ndata:\n # ──────────────────────────────────────────────────────────────────────────\n # System locale -- forces UTF-8 inside the slim Python container.\n # Upstream commit 82e7936d bakes these into the Dockerfile, but our v2.0.1\n # backend image was built before that commit. Without them, any path or\n # tool output containing non-ASCII characters (Chinese filenames, emoji\n # in markdown, etc.) raises UnicodeEncodeError when Python opens the file\n # or writes to stdout. Safe to leave on permanently.\n # ──────────────────────────────────────────────────────────────────────────\n LANG: \"C.UTF-8\"\n LC_ALL: \"C.UTF-8\"\n PYTHONIOENCODING: \"utf-8\"\n\n # ──────────────────────────────────────────────────────────────────────────\n # Web search & crawl provider keys.\n # Tavily / Jina / InfoQuest are the original three. Serper / Firecrawl /\n # Exa were added by upstream in 2026-Q2 (Serper PR #2630 in commit\n # 44ab21fc). Leave empty unless the corresponding tool is referenced in\n # config.yaml's `tools:` list -- empty values are fine, deer-flow only\n # raises if a $VAR reference resolves to nothing.\n # ──────────────────────────────────────────────────────────────────────────\n TAVILY_API_KEY: \"\"\n JINA_API_KEY: \"\"\n INFOQUEST_API_KEY: \"\"\n SERPER_API_KEY: \"\"\n FIRECRAWL_API_KEY: \"\"\n EXA_API_KEY: \"\"\n\n # ──────────────────────────────────────────────────────────────────────────\n # Model provider keys. Reference these via $VAR in config.yaml's `models:`\n # section. Empty values are safe; only models that actually use the key\n # will fail to register at startup.\n # ──────────────────────────────────────────────────────────────────────────\n OPENAI_API_KEY: \"\"\n DEEPSEEK_API_KEY: \"\"\n ANTHROPIC_API_KEY: \"\"\n GOOGLE_API_KEY: \"\"\n GEMINI_API_KEY: \"\"\n VOLCENGINE_API_KEY: \"\"\n NOVITA_API_KEY: \"\"\n MINIMAX_API_KEY: \"\"\n MOONSHOT_API_KEY: \"\"\n VLLM_API_KEY: \"\"\n\n # LangSmith tracing (set to \"true\" and fill LANGSMITH_API_KEY to enable)\n LANGCHAIN_TRACING_V2: \"false\"\n LANGSMITH_API_KEY: \"\"\n\n # ──────────────────────────────────────────────────────────────────────────\n # Upload limits. Default upstream caps are tuned for the docker-compose\n # demo (50 MiB per file, 100 MiB per request, 10 files); on Olares users\n # routinely drop in 4K screenshots, recordings, and multi-page PDFs that\n # exceed the per-file cap. Bumping the defaults to 100 MiB / 500 MiB / 20\n # is conservative for an at-home server -- raise further if you upload\n # large media. Sizes are bytes.\n # ──────────────────────────────────────────────────────────────────────────\n UPLOADS_MAX_FILES: \"20\"\n UPLOADS_MAX_FILE_SIZE: \"104857600\" # 100 MiB\n UPLOADS_MAX_TOTAL_SIZE: \"524288000\" # 500 MiB\n # ============================================================================\n # SUBAGENT LIMITS — tune to fix \"Recursion limit reached\" errors\n # ============================================================================\n # Each subagent (general-purpose, bash) runs as its own LangGraph instance.\n # `max_turns` caps how many LLM <-> tool roundtrips it can take before the\n # graph aborts with `GraphRecursionError`. `timeout_seconds` caps wall time.\n #\n # The values below are referenced by config.yaml as `$VAR`, resolved via\n # DeerFlow's `resolve_env_variables()` and coerced to int by Pydantic\n # (lax mode), so quoted strings here become real ints downstream.\n #\n # ─── WHEN TO ADJUST ─────────────────────────────────────────────────────────\n # Symptom → Action\n # \"Recursion limit of N reached\" in logs → raise the matching\n # *_MAX_TURNS\n # Subagent aborts with \"timed out\" → raise the matching\n # *_TIMEOUT_SECONDS\n # Local model burns CPU/$$ in obvious loops → lower *_MAX_TURNS to\n # surface failures fast\n #\n # ─── RECOMMENDED VALUES BY MODEL CLASS ──────────────────────────────────────\n # Model class gp_turns bash_turns\n # ────────────────────────────────────────────── ──────── ──────────\n # Strong frontier (Claude 4 / GPT-5 / Doubao 2) 150 80\n # Efficient MoE (Gemma 4 26B A4B) 200 100\n # Mid local (Qwen3 32B+ / DeepSeek V3.2 / Kimi 2.5) 300 150\n # Heavy local 30B-class (Qwen3.x 35B) ← default 500 300\n # Weak / heavily quantized (Qwen3 ≤ 14B, INT4) 1000 400\n #\n # gp_turns = SUBAGENT_GENERAL_PURPOSE_MAX_TURNS\n # bash_turns = SUBAGENT_BASH_MAX_TURNS\n # Image-baked builtin defaults: gp=500, bash=300 (already raised from\n # upstream's 100 / 60 in this fork to handle long delegated workflows).\n #\n # ─── HOW TO SIZE TIMEOUTS ───────────────────────────────────────────────────\n # Timeouts should be a safety net for hung/runaway processes — they should\n # NOT trigger before max_turns under normal operation, otherwise raising\n # max_turns has no effect. Rule of thumb:\n # timeout_seconds ≥ max_turns × realistic_seconds_per_turn\n #\n # Per-turn cost depends on model speed:\n # Strong cloud (Claude / GPT-5) 5-15 s/turn\n # Efficient MoE on Blackwell GPU 3-8 s/turn\n # 30B local on RTX 4090 Q4 15-30 s/turn\n # 30B local on CPU / iGPU 60-180 s/turn\n #\n # Defaults below assume 500 turns × 14 s/turn ≈ 2 h for general-purpose,\n # which covers cloud and mid-tier local GPUs. For pure-CPU or very slow\n # quantized local setups, push GENERAL_PURPOSE_TIMEOUT_SECONDS to 14400\n # (4 h) or more.\n #\n # ─── HOW TO APPLY CHANGES ───────────────────────────────────────────────────\n # 1. Edit the value below (or `kubectl edit configmap deerflowv2-env-config`).\n # 2. Restart the backend so the new env vars are picked up:\n # kubectl -n <namespace> rollout restart deploy/deerflowv2-backend\n # 3. Verify inside the pod:\n # kubectl -n <ns> exec deploy/deerflowv2-backend -c gateway -- \\\n # env | grep SUBAGENT_\n #\n # ─── CAVEATS ────────────────────────────────────────────────────────────────\n # - Values MUST stay quoted strings (e.g. \"500\", not 500). They are env vars.\n # - Do NOT delete these keys. `resolve_env_variables()` raises ValueError if\n # a `$VAR` reference can't be resolved, which crashes the pod on startup.\n # - For installs whose appData/config/config.yaml predates this chart (no\n # `subagents:` block, or hard-coded numbers): either edit it manually to\n # match the template below, OR delete it and let the init-container\n # regenerate from the template (this loses any custom `models:` you added,\n # so back it up first).\n # - Changes to this ConfigMap do NOT auto-restart the pod. The rollout\n # restart in step 2 is required.\n SUBAGENT_DEFAULT_TIMEOUT_SECONDS: \"1800\"\n # Global default max_turns. Per-agent values below override this, so it\n # only kicks in for any custom_agents the user defines without an explicit\n # max_turns override (e.g. a \"data-analysis\" subagent created via\n # subagents.custom_agents in config.yaml). Upstream builtin default is 50.\n SUBAGENT_DEFAULT_MAX_TURNS: \"200\"\n SUBAGENT_GENERAL_PURPOSE_MAX_TURNS: \"500\"\n SUBAGENT_GENERAL_PURPOSE_TIMEOUT_SECONDS: \"7200\"\n SUBAGENT_BASH_MAX_TURNS: \"300\"\n SUBAGENT_BASH_TIMEOUT_SECONDS: \"1800\"\n---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: deerflowv2-init-config\n namespace: {{ .Release.Namespace }}\ndata:\n config.yaml: |\n # Bumped to match upstream config.example.yaml (deer-flow main, May 2026).\n # The version number itself is purely a \"your config is outdated, run\n # config-upgrade\" warning trigger in deer-flow's startup log -- it does\n # not unlock or restrict any features.\n config_version: 8\n\n # Logger threshold for the `deerflow` and `app` loggers. Third-party\n # libraries (uvicorn, sqlalchemy, langgraph, ...) keep their own levels.\n # Use \"debug\" temporarily to surface tool selection / middleware decisions\n # when an agent misbehaves, then revert.\n log_level: info\n\n # Surface input/output/total token counts in the workspace UI when the\n # provider returns them. Helps spot a runaway Ultra session before it\n # eats a Tavily quota or burns local GPU time on a duplicate-dispatch.\n token_usage:\n enabled: true\n\n # =============================================\n # HOW TO ADD MODELS\n # =============================================\n # Edit this file at: appData/config/config.yaml\n #\n # Ollama example (use models you've pulled in Ollama):\n # - name: qwen3\n # display_name: Qwen3 (Ollama)\n # use: langchain_openai:ChatOpenAI\n # model: qwen3\n # api_key: ollama\n # base_url: $OLLAMA_OPENAI_BASE_URL\n # supports_thinking: true # enables Pro/Ultra modes (Qwen3+ supports thinking)\n # supports_vision: true # enables image input (only for vision models like Qwen2.5-VL)\n #\n # OpenAI example:\n # - name: gpt-4o\n # display_name: GPT-4o\n # use: langchain_openai:ChatOpenAI\n # model: gpt-4o\n # api_key: $OPENAI_API_KEY\n # supports_thinking: true\n # supports_vision: true\n #\n # Claude example:\n # - name: claude-sonnet\n # display_name: Claude 4 Sonnet\n # use: langchain_anthropic:ChatAnthropic\n # model: claude-sonnet-4-20250514\n # api_key: $ANTHROPIC_API_KEY\n # supports_thinking: true\n # supports_reasoning_effort: true # enables reasoning effort slider\n # supports_vision: true\n #\n # Key model capabilities:\n # supports_thinking: true/false - Pro/Ultra mode (thinking/planning before answering)\n # supports_reasoning_effort: true/false - reasoning effort slider (low/medium/high)\n # supports_vision: true/false - image/vision input support\n #\n # After editing, restart the app to apply changes.\n models: []\n\n tool_groups:\n - name: web\n - name: file:read\n - name: file:write\n - name: bash\n\n tools:\n - name: web_search\n group: web\n use: deerflow.community.tavily.tools:web_search_tool\n max_results: 5\n - name: web_fetch\n group: web\n use: deerflow.community.jina_ai.tools:web_fetch_tool\n timeout: 10\n - name: image_search\n group: web\n use: deerflow.community.image_search.tools:image_search_tool\n max_results: 5\n - name: ls\n group: file:read\n use: deerflow.sandbox.tools:ls_tool\n - name: read_file\n group: file:read\n use: deerflow.sandbox.tools:read_file_tool\n - name: write_file\n group: file:write\n use: deerflow.sandbox.tools:write_file_tool\n - name: str_replace\n group: file:write\n use: deerflow.sandbox.tools:str_replace_tool\n - name: bash\n group: bash\n use: deerflow.sandbox.tools:bash_tool\n\n tool_search:\n enabled: false\n\n # =============================================\n # UPLOADS\n # =============================================\n # All values are sourced from env vars in deerflowv2-env-config so\n # operators can resize without editing config.yaml. See that ConfigMap\n # for tuning notes. Sizes are bytes; the gateway also surfaces these\n # to the frontend so the file picker can reject too-large files before\n # they hit the wire.\n uploads:\n max_files: $UPLOADS_MAX_FILES\n max_file_size: $UPLOADS_MAX_FILE_SIZE\n max_total_size: $UPLOADS_MAX_TOTAL_SIZE\n # Office/PDF auto-conversion runs on the gateway host BEFORE the\n # sandbox isolates the file. Keep off unless uploads come from a\n # fully trusted source -- a malicious .docx can hit the parser before\n # any sandboxing applies. The image-ocr Skill handles user-driven\n # OCR explicitly via tesseract anyway.\n auto_convert_documents: false\n pdf_converter: auto\n\n sandbox:\n use: deerflow.sandbox.local:LocalSandboxProvider\n # Re-enable host bash for the LLM. Upstream PR #1547 (Security: Address\n # critical host-shell escape in LocalSandboxProvider) gates the `bash`\n # tool behind this flag, because LocalSandboxProvider executes commands\n # directly on the gateway pod's filesystem rather than inside an\n # isolated sandbox container. With the flag off, the LLM cannot\n # `pip install` or `apt-get install` runtime helpers (notably\n # pytesseract / tesseract-ocr for OCR), which silently breaks the\n # legacy \"ask the model to OCR an image\" workflow that worked before\n # we picked up the security patch.\n #\n # In a single-tenant Olares deployment the gateway pod is the trust\n # boundary anyway, so re-enabling this restores prior behavior without\n # widening exposure beyond what existed before. If you ever expose this\n # instance to untrusted users, switch to AioSandboxProvider (per-thread\n # container) instead and flip this back to false.\n allow_host_bash: true\n\n skills:\n container_path: /mnt/skills\n\n title:\n enabled: true\n max_words: 6\n max_chars: 60\n model_name: null\n\n summarization:\n enabled: true\n model_name: null\n trigger:\n - type: tokens\n value: 15564\n keep:\n type: messages\n value: 10\n trim_tokens_to_summarize: 15564\n summary_prompt: null\n # Recently-loaded skill files (e.g. the image-ocr SKILL.md the agent\n # just opened) are exempt from summarization, otherwise the agent\n # would lose its instructions mid-task. These are upstream defaults\n # pinned here so a future deer-flow update changing the defaults\n # cannot silently degrade the image-ocr workflow.\n preserve_recent_skill_count: 5\n preserve_recent_skill_tokens: 25000\n preserve_recent_skill_tokens_per_skill: 5000\n skill_file_read_tool_names:\n - read_file\n - read\n - view\n - cat\n\n memory:\n enabled: true\n storage_path: memory.json\n debounce_seconds: 30\n model_name: null\n max_facts: 100\n fact_confidence_threshold: 0.7\n injection_enabled: true\n max_injection_tokens: 2000\n\n # =============================================\n # PERSISTENCE (users, threads metadata, checkpoints, run events)\n # =============================================\n # The unified `database:` block drives EVERYTHING that needs durable\n # storage: gateway user store (users + sessions + tokens), thread\n # metadata, the LangGraph checkpointer, and the run-event journal\n # (token usage, tool-call traces). All routed to a single postgres\n # database provisioned automatically by the Olares postgres\n # middleware (declared in OlaresManifest.yaml as `middleware.postgres`).\n #\n # Connection strings are built from .Values.postgres.* in deployment.yaml:\n # DATABASE_URL → SQLAlchemy ORM (asyncpg) — **no** libpq-only params\n # CHECKPOINTER_DATABASE_URL → LangGraph checkpointer (psycopg) — TCP keepalives OK\n # deer-flow resolves both via AppConfig.resolve_env_variables() at startup.\n #\n # To switch back to sqlite (single-tenant, no postgres dependency):\n # 1. Remove the `middleware.postgres` block from OlaresManifest.yaml\n # 2. Remove the DATABASE_URL env entries from deployment.yaml\n # 3. Replace the block below with:\n # database:\n # backend: sqlite\n # sqlite_dir: /data/deer-flow/db\n # 4. Recreate the appData/config/config.yaml or run the migration\n # script in reverse.\n database:\n backend: postgres\n postgres_url: $DATABASE_URL\n checkpointer_postgres_url: $CHECKPOINTER_DATABASE_URL\n\n # =============================================\n # SUBAGENT LIMITS\n # =============================================\n # All values resolved from env vars defined in the deerflowv2-env-config\n # ConfigMap — see that ConfigMap's comments for tuning guidance,\n # recommended values per model class, and how to apply changes.\n # Tune the env vars there, NOT the literals below.\n subagents:\n timeout_seconds: $SUBAGENT_DEFAULT_TIMEOUT_SECONDS\n max_turns: $SUBAGENT_DEFAULT_MAX_TURNS\n agents:\n general-purpose:\n timeout_seconds: $SUBAGENT_GENERAL_PURPOSE_TIMEOUT_SECONDS\n max_turns: $SUBAGENT_GENERAL_PURPOSE_MAX_TURNS\n bash:\n timeout_seconds: $SUBAGENT_BASH_TIMEOUT_SECONDS\n max_turns: $SUBAGENT_BASH_MAX_TURNS\n\n # =============================================\n # RUN EVENTS (message + tool-call telemetry)\n # =============================================\n # Upstream defaults this to \"memory\", which means every conversation's\n # full execution trace (subagent dispatches, tool invocations, token\n # counts, errors) is kept only in process RAM and lost on pod restart.\n # That makes it impossible to audit \"why did the lead burn 15 minutes\n # last night\" after the fact, and it makes the per-run token-usage\n # accounting in the UI reset on every restart.\n #\n # We already run sqlite for the user store + checkpointer, so flipping\n # run_events to the same backend is essentially free -- traces land in\n # the same deerflow.db file. Still bounded by max_trace_content (10 KiB\n # per individual trace blob) so a runaway tool output cannot bloat the\n # DB indefinitely.\n run_events:\n backend: db\n max_trace_content: 10240\n track_token_usage: true\n\n # =============================================\n # CIRCUIT BREAKER (LLM provider failure isolation)\n # =============================================\n # When the upstream model provider (Ollama, vLLM, OpenAI, ...) returns\n # 5 consecutive failures, fast-fail subsequent calls for 60 seconds\n # instead of letting LangGraph retry-loops hammer a dying provider.\n # Defaults below match upstream and work well for both local Ollama\n # (where 5 timeouts means the model is genuinely stuck) and cloud APIs\n # (where 5 failures usually means rate limit / outage). Pinned here so\n # the values are visible to operators rather than buried in code.\n circuit_breaker:\n failure_threshold: 5\n recovery_timeout_sec: 60\n\n extensions_config.json: |\n {\n \"mcpServers\": {},\n \"skills\": {}\n }\n\n # ============================================================================\n # MIGRATION SCRIPT: sqlite -> postgres database backend\n # ============================================================================\n # Mounted at /init-config/migrate_database_to_postgres.py and invoked by the\n # init-container in deployment.yaml. Idempotently rewrites the `database:`\n # block in an existing /config/config.yaml (the operator's persisted copy\n # on the appData hostPath) so chart upgrades flip the backend in place\n # without losing user-edited fields like `models:`.\n #\n # Why python instead of sed/awk: top-level YAML block detection has to\n # respect indentation, comment lines, and key order. A regex one-liner\n # gets it wrong on configs that operators have hand-edited, and the\n # failure mode (silently appending a duplicate `database:` block, then\n # PyYAML picks one of the two non-deterministically) is hard to debug.\n # The deer-flow image ships python3.12, so this is free.\n migrate_database_to_postgres.py: |\n \"\"\"Rewrite database:/checkpointer: blocks in config.yaml to use postgres.\n\n Usage: python3 migrate_database_to_postgres.py /path/to/config.yaml\n \"\"\"\n from __future__ import annotations\n\n import re\n import sys\n from pathlib import Path\n\n POSTGRES_BLOCK = (\n \"\\n\"\n \"# Auto-injected by init-container (chart >= 1.0.9). The Olares\\n\"\n \"# `middleware.postgres` provisioner exposes connection details via\\n\"\n \"# DATABASE_URL (asyncpg ORM) and CHECKPOINTER_DATABASE_URL (psycopg\\n\"\n \"# LangGraph saver with TCP keepalives). To switch back to sqlite,\\n\"\n \"# remove the postgres middleware block from OlaresManifest.yaml and\\n\"\n \"# edit this section by hand.\\n\"\n \"database:\\n\"\n \" backend: postgres\\n\"\n \" postgres_url: $DATABASE_URL\\n\"\n \" checkpointer_postgres_url: $CHECKPOINTER_DATABASE_URL\\n\"\n )\n\n\n def ensure_checkpointer_postgres_url(cfg: Path) -> None:\n \"\"\"Inject checkpointer_postgres_url for chart 1.0.11+ without full rewrite.\"\"\"\n text = cfg.read_text(encoding=\"utf-8\")\n if \"checkpointer_postgres_url:\" in text:\n return\n if \"backend: postgres\" not in text or \"postgres_url:\" not in text:\n return\n lines = text.splitlines(keepends=True)\n out: list[str] = []\n inserted = False\n for line in lines:\n out.append(line)\n if not inserted and re.match(r\"^[ \\t]+postgres_url:[ \\t]\", line):\n out.append(\" checkpointer_postgres_url: $CHECKPOINTER_DATABASE_URL\\n\")\n inserted = True\n if inserted:\n cfg.write_text(\"\".join(out), encoding=\"utf-8\")\n print(f\"migrate: injected checkpointer_postgres_url into {cfg}\")\n\n # Match a top-level YAML block whose key is `database` or the legacy\n # upstream `checkpointer`. The body is any consecutive run of indented\n # or blank lines following the key. Stops at the next top-level key\n # (a non-whitespace character at column 0) or end-of-file.\n BLOCK_RE = re.compile(\n r\"^(?:database|checkpointer):[ \\t]*\\n(?:[ \\t]+[^\\n]*\\n|[ \\t]*\\n)*\",\n re.MULTILINE,\n )\n\n\n def main(path: str) -> int:\n cfg = Path(path)\n if not cfg.exists():\n print(f\"migrate: {cfg} does not exist; nothing to do\")\n return 0\n\n original = cfg.read_text(encoding=\"utf-8\")\n # Drop ALL existing database:/checkpointer: blocks. There may be\n # more than one if a previous chart bug appended duplicates.\n stripped = BLOCK_RE.sub(\"\", original)\n # Trim trailing whitespace before re-appending so the final file\n # is not padded with blank lines after repeated runs.\n new = stripped.rstrip() + \"\\n\" + POSTGRES_BLOCK\n\n if new == original:\n print(f\"migrate: {cfg} already on postgres backend; no change\")\n ensure_checkpointer_postgres_url(cfg)\n return 0\n\n cfg.write_text(new, encoding=\"utf-8\")\n print(f\"migrate: rewrote {cfg} database block -> postgres\")\n ensure_checkpointer_postgres_url(cfg)\n return 0\n\n\n if __name__ == \"__main__\":\n sys.exit(main(sys.argv[1] if len(sys.argv) > 1 else \"/config/config.yaml\"))\n\n # ============================================================================\n # SKILL: image-ocr -- canonical OCR workflow for uploaded images\n # ============================================================================\n # deer-flow's <uploaded_files> middleware nudges the model toward read_file /\n # grep / glob (text-file affordances) regardless of whether the upload is\n # actually an image. Without further guidance the lead agent tends to either\n # (a) hallucinate the image contents, or (b) reach for a heavyweight OCR\n # stack like paddleocr / easyocr that downloads ~600 MB and blows past the\n # 600s subprocess.run timeout in LocalSandbox.execute_command.\n #\n # Skills are deer-flow's first-class extension point for this kind of fix:\n # any SKILL.md placed under <skills_root>/custom/<name>/ is auto-enabled\n # (extensions_config.is_skill_enabled defaults to True for the \"custom\"\n # category), and its `description` frontmatter is injected into the lead\n # agent system prompt as <available_skills>. When the user query matches\n # the description, the agent calls read_file on the SKILL.md to load the\n # full instructions, then follows them verbatim.\n #\n # The init-container copies this skill to /skills/custom/image-ocr/SKILL.md\n # on every pod start, so chart bumps push updated skill content. Operators\n # who want a different policy can edit the file directly, then bump the\n # `name` in the frontmatter so init does not overwrite it; or disable the\n # skill from the Olares extensions UI.\n image-ocr-skill.md: |\n ---\n name: image-ocr\n description: >-\n Extract text from an uploaded image. Trigger when the user asks to OCR\n an image, read text from a picture, recognize characters, transcribe a\n screenshot, identify what is written on a poster / receipt / business\n card / handwritten note, or extract any textual content from an image\n file. Common phrasings include \"OCR this image\", \"OCR it\", \"extract\n text from screenshot\", \"transcribe this image\", \"what does this image\n say\", \"read the text in this picture\", \"recognize the characters in\n this photo\", and equivalent requests in any other language (the model\n should match by intent, not literal wording). Trigger whenever a .png,\n .jpg, .jpeg, .webp, or .heic file is uploaded and the user wants the\n textual content rather than a visual description.\n ---\n\n # Image OCR\n\n Canonical, fast path for extracting text from an uploaded image. Follow\n this strictly to avoid the two failure modes operators have reported:\n\n 1. Hallucinating image contents instead of actually reading the file.\n 2. Picking heavyweight OCR stacks (paddleocr, easyocr, mmocr, rapidocr)\n that download hundreds of megabytes and time out the bash tool.\n\n ## Decision tree\n\n ### Step 1 -- Try `view_image` first (best when available)\n\n Look at your own tool list. If `view_image` is present, call it directly:\n\n ```\n view_image(image_path=\"/mnt/user-data/uploads/<filename>\")\n ```\n\n This injects the actual image bytes into your context as a vision input.\n Read the text directly from what you see and return it. No installation,\n no sandbox call, sub-second result, exact characters preserved.\n\n If `view_image` is not in your tool list (the selected model has no\n vision capability) or the call returns an error, fall through to Step 2.\n\n ### Step 2 -- pytesseract (only if Step 1 unavailable)\n\n Install `pytesseract` plus the `tesseract` binary and the Chinese +\n English language packs in one shot. Use these exact commands; do NOT\n substitute paddleocr, easyocr, mmocr, rapidocr, or any other OCR stack:\n they are 10x to 100x larger than tesseract and routinely exceed the\n bash tool's 600s wall-clock timeout.\n\n ```bash\n which tesseract >/dev/null 2>&1 || (apt-get update -qq && apt-get install -y -qq tesseract-ocr tesseract-ocr-chi-sim tesseract-ocr-chi-tra tesseract-ocr-eng > /dev/null)\n python -c \"import pytesseract\" 2>/dev/null || pip install -q pytesseract Pillow -i https://pypi.tuna.tsinghua.edu.cn/simple\n ```\n\n Then run OCR. The `lang` argument matters; without it Chinese text comes\n back as garbage:\n\n ```bash\n python - <<'PY'\n import pytesseract\n from PIL import Image\n text = pytesseract.image_to_string(\n Image.open('/mnt/user-data/uploads/<filename>'),\n lang='chi_sim+chi_tra+eng',\n )\n print(text)\n PY\n ```\n\n Total wall time on a fresh sandbox: roughly 30 to 60 seconds for the\n first install (apt fetches plus tesseract data files), under 2 seconds\n for subsequent calls in the same conversation since both apt and pip\n are cached inside the gateway pod for its lifetime.\n\n ## Constraints\n\n - Do not invent text. If the image is unreadable or OCR returns\n nothing, say so explicitly. Do not guess at the contents.\n - Do not save the OCR output as a file under `/mnt/user-data/outputs`\n unless the user explicitly asks for a file deliverable. Just\n include the extracted text inline in your response.\n - Do not chain multiple OCR libraries in a single turn. Pick one path\n (Step 1 OR Step 2) and stick with it. If Step 2 fails, report the\n failure to the user instead of falling back to a third option.\n - For multi-page PDFs that have already been converted to images, run\n Step 2 once per page and concatenate the results in page order.\n"}, "72e8891470804377c1232f8dc3a06c45862f5328": {"t": 1785748959.0308185, "n": 4247, "h": "9ea8cd167601dce1", "c": "---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: deerflowv2-env-config\n namespace: {{ .Release.Namespace }}\ndata:\n # Search & Crawl tools\n TAVILY_API_KEY: \"\"\n JINA_API_KEY: \"\"\n INFOQUEST_API_KEY: \"\"\n # Model API keys (set the ones you use in config.yaml models section)\n OPENAI_API_KEY: \"\"\n DEEPSEEK_API_KEY: \"\"\n ANTHROPIC_API_KEY: \"\"\n GOOGLE_API_KEY: \"\"\n VOLCENGINE_API_KEY: \"\"\n NOVITA_API_KEY: \"\"\n MINIMAX_API_KEY: \"\"\n MOONSHOT_API_KEY: \"\"\n # LangSmith tracing (set to \"true\" and fill LANGSMITH_API_KEY to enable)\n LANGCHAIN_TRACING_V2: \"false\"\n LANGSMITH_API_KEY: \"\"\n---\napiVersion: v1\nkind: ConfigMap\nmetadata:\n name: deerflowv2-init-config\n namespace: {{ .Release.Namespace }}\ndata:\n config.yaml: |\n config_version: 2\n\n # =============================================\n # HOW TO ADD MODELS\n # =============================================\n # Edit this file at: appData/config/config.yaml\n #\n # Ollama example (use models you've pulled in Ollama):\n # - name: qwen3\n # display_name: Qwen3 (Ollama)\n # use: langchain_openai:ChatOpenAI\n # model: qwen3\n # api_key: ollama\n # base_url: $OLLAMA_OPENAI_BASE_URL\n # supports_thinking: true # enables Pro/Ultra modes (Qwen3+ supports thinking)\n # supports_vision: true # enables image input (only for vision models like Qwen2.5-VL)\n #\n # OpenAI example:\n # - name: gpt-4o\n # display_name: GPT-4o\n # use: langchain_openai:ChatOpenAI\n # model: gpt-4o\n # api_key: $OPENAI_API_KEY\n # supports_thinking: true\n # supports_vision: true\n #\n # Claude example:\n # - name: claude-sonnet\n # display_name: Claude 4 Sonnet\n # use: langchain_anthropic:ChatAnthropic\n # model: claude-sonnet-4-20250514\n # api_key: $ANTHROPIC_API_KEY\n # supports_thinking: true\n # supports_reasoning_effort: true # enables reasoning effort slider\n # supports_vision: true\n #\n # Key model capabilities:\n # supports_thinking: true/false - Pro/Ultra mode (thinking/planning before answering)\n # supports_reasoning_effort: true/false - reasoning effort slider (low/medium/high)\n # supports_vision: true/false - image/vision input support\n #\n # After editing, restart the app to apply changes.\n models: []\n\n tool_groups:\n - name: web\n - name: file:read\n - name: file:write\n - name: bash\n\n tools:\n - name: web_search\n group: web\n use: deerflow.community.tavily.tools:web_search_tool\n max_results: 5\n - name: web_fetch\n group: web\n use: deerflow.community.jina_ai.tools:web_fetch_tool\n timeout: 10\n - name: image_search\n group: web\n use: deerflow.community.image_search.tools:image_search_tool\n max_results: 5\n - name: ls\n group: file:read\n use: deerflow.sandbox.tools:ls_tool\n - name: read_file\n group: file:read\n use: deerflow.sandbox.tools:read_file_tool\n - name: write_file\n group: file:write\n use: deerflow.sandbox.tools:write_file_tool\n - name: str_replace\n group: file:write\n use: deerflow.sandbox.tools:str_replace_tool\n - name: bash\n group: bash\n use: deerflow.sandbox.tools:bash_tool\n\n tool_search:\n enabled: false\n\n sandbox:\n use: deerflow.sandbox.local:LocalSandboxProvider\n\n skills:\n container_path: /mnt/skills\n\n title:\n enabled: true\n max_words: 6\n max_chars: 60\n model_name: null\n\n summarization:\n enabled: true\n model_name: null\n trigger:\n - type: tokens\n value: 15564\n keep:\n type: messages\n value: 10\n trim_tokens_to_summarize: 15564\n summary_prompt: null\n\n memory:\n enabled: true\n storage_path: memory.json\n debounce_seconds: 30\n model_name: null\n max_facts: 100\n fact_confidence_threshold: 0.7\n injection_enabled: true\n max_injection_tokens: 2000\n\n checkpointer:\n type: sqlite\n connection_string: checkpoints.db\n\n extensions_config.json: |\n {\n \"mcpServers\": {},\n \"skills\": {}\n }\n"}}, "tradingagents/OlaresManifest.yaml": {"51afc2791a78507a03913fe1006ee41919d53a00": {"t": 1786068217.6854503, "n": 5822, "h": "d82be33d96956ad6", "c": "---\nolaresManifest.version: '0.12.0'\nolaresManifest.type: app\napiVersion: 'v3'\n\nworkloadReplicas:\n tradingagentsingress: 1\n terminal: 1\n tradingagents: 1\nmetadata:\n name: tradingagents\n icon: https://app.cdn.olares.com/appstore/tradingagents/icon.png\n description: Multi-Agents LLM Financial Trading Framework\n appid: tradingagents\n title: TradingAgents\n version: \"1.0.7\"\n categories:\n - Productivity_v112\n\nentrances:\n - name: tradingagentsTerminal\n port: 8081\n host: tradingagentsingress\n title: TradingAgents\n icon: https://app.cdn.olares.com/appstore/tradingagents/icon.png\n authLevel: private\n openMethod: window\n - name: tradingagents\n port: 8080\n host: tradingagentsingress\n title: TradingAgents Info\n invisible: true\n icon: https://app.cdn.olares.com/appstore/tradingagents/icon.png\n authLevel: private\n openMethod: window\n\nspec:\n upgradeDescription: |\n Switch the reverse-proxy image from Bitnami OpenResty (`beclab/aboveos-bitnami-openresty:1.25.3-2`) to the official OpenResty image `openresty/openresty:1.29.2.5-bookworm-fat`.\n\n Align nginx config mounts and log paths with the official image layout (`/etc/nginx/conf.d/default.conf`, `/usr/local/openresty/nginx/logs/...`), and drop Bitnami-specific `OPENRESTY_CONF_FILE` wiring.\n versionName: \"0.2.5\"\n fullDescription: |\n TradingAgents is a multi-agent trading framework that mirrors the dynamics of real-world trading firms. By deploying specialized LLM-powered agents: from fundamental analysts, sentiment experts, and technical analysts, to trader, risk management team, the platform collaboratively evaluates market conditions and informs trading decisions. Moreover, these agents engage in dynamic discussions to pinpoint the optimal strategy.\n\n > TradingAgents framework is designed for research purposes. Trading performance may vary based on many factors, including the chosen backbone language models, model temperature, trading periods, the quality of data, and other non-deterministic factors. It is not intended as financial, investment, or trading advice.\n\n Our framework decomposes complex trading tasks into specialized roles. This ensures the system achieves a robust, scalable approach to market analysis and decision-making.\n\n **Analyst Team**\n - Fundamentals Analyst: Evaluates company financials and performance metrics, identifying intrinsic values and potential red flags.\n - Sentiment Analyst: Aggregates news headlines, StockTwits, and Reddit chatter into a single sentiment read to gauge short-term market mood.\n - News Analyst: Monitors global news and macroeconomic indicators, interpreting the impact of events on market conditions.\n - Technical Analyst: Utilizes technical indicators (like MACD and RSI) to detect trading patterns and forecast price movements.\n\n **Researcher Team**\n - Comprises both bullish and bearish researchers who critically assess the insights provided by the Analyst Team. Through structured debates, they balance potential gains against inherent risks.\n\n **Trader Agent**\n - Composes reports from the analysts and researchers to make informed trading decisions. It determines the timing and magnitude of trades based on comprehensive market insights.\n\n **Risk Management and Portfolio Manager**\n - Continuously evaluates portfolio risk by assessing market volatility, liquidity, and other risk factors. The risk management team evaluates and adjusts trading strategies, providing assessment reports to the Portfolio Manager for final decision.\n The Portfolio Manager approves/rejects the transaction proposal. If approved, the order will be sent to the simulated exchange and executed.\n\n developer: TauricResearch\n sourceCode: https://github.com/TauricResearch/TradingAgents\n submitter: Olares\n locale:\n - en-US\n - zh-CN\n requiredMemory: 1Gi\n limitedMemory: 4Gi\n requiredDisk: 256Mi\n limitedDisk: 2Gi\n requiredCpu: 500m\n limitedCpu: \"3\"\n supportArch:\n - amd64\n - arm64\n\npermission:\n appData: true\n\n# Set keys for the provider you use in the CLI (DeepSeek -> DEEPSEEK_API_KEY, OpenAI -> OPENAI_API_KEY, etc.).\nenvs:\n - envName: OLLAMA_BASE_URL\n required: false\n applyOnChange: true\n type: url\n editable: true\n description: Ollama base URL\n - envName: DEEPSEEK_API_KEY\n required: false\n applyOnChange: true\n type: string\n editable: true\n description: DeepSeek API key when using provider DeepSeek\n - envName: OPENAI_API_KEY\n required: false\n applyOnChange: true\n type: string\n editable: true\n description: OpenAI API key when using provider OpenAI\n - envName: ANTHROPIC_API_KEY\n required: false\n applyOnChange: true\n type: string\n editable: true\n description: Anthropic API key when using Claude\n - envName: GOOGLE_API_KEY\n required: false\n applyOnChange: true\n type: string\n editable: true\n description: Google AI API key for Gemini\n - envName: DASHSCOPE_API_KEY\n required: false\n applyOnChange: true\n type: string\n editable: true\n description: Alibaba DashScope for Qwen\n - envName: ZHIPU_API_KEY\n required: false\n applyOnChange: true\n type: string\n editable: true\n description: Zhipu GLM API key\n - envName: OPENROUTER_API_KEY\n required: false\n applyOnChange: true\n type: string\n editable: true\n description: OpenRouter API key\n - envName: XAI_API_KEY\n required: false\n applyOnChange: true\n type: string\n editable: true\n description: xAI API key\n - envName: ALPHA_VANTAGE_API_KEY\n required: false\n applyOnChange: true\n type: string\n editable: true\n description: Optional Alpha Vantage key for market data\n\noptions:\n apiTimeout: 0\n dependencies:\n - name: olares\n type: system\n version: '>=1.12.6-0'\n"}}}}