1 line
222 KiB
JSON
1 line
222 KiB
JSON
{"repo": "abu-shotai/ai-video-remix", "files": {"README.md": {"7ca00a40057e92da07fc40a8e3eae4c7038a8387": {"t": 1785674033.9933276, "n": 8754, "h": "572107b3bed9801b", "c": "# AI Video Remix\n\n> AI-driven video remix generator — semantic video search + LLM planning + Remotion rendering.\n>\n> Requires [ShotAI](https://www.shotai.io) — local video asset management and footage search for Mac & Windows.\n>\n> 中文文档: [README.zh.md](README.zh.md)\n\nGenerate styled video compositions from your local video footage library using natural language. [ShotAI](https://www.shotai.io) handles shot-level indexing and semantic search; this tool handles the planning, music, and rendering.\n\n---\n\n## Demo\n\n> Hong Kong Cyberpunk Night — generated from local video footage with ShotAI + Remotion, no manual editing.\n\n[](https://www.youtube.com/watch?v=mibbqDf6uQY)\n\n---\n\n## Use as a Claude Skill\n\nThis repo ships a ready-to-install [Claude Agent Skill](https://support.claude.com/en/articles/12512176-what-are-skills) in the [`skill/`](skill/) directory.\n\n**Install in Claude Code:**\n```bash\n/plugin install ai-video-remix@abu-ShotAI/ai-video-remix#skill\n```\n\nOr point Claude Code settings to the local `skill/` folder.\n\nOnce installed, just describe what you want:\n> *\"Make a travel vlog from my library\"*\n> *\"Create a cyberpunk city highlight reel\"*\n> *\"Sports highlight from last weekend's footage\"*\n\n---\n\n## Prerequisites\n\n| Tool | Purpose | Install |\n|------|---------|---------|\n| [ShotAI](https://www.shotai.io) | AI video asset management + semantic footage search — provides the MCP server this tool queries | [Download for Mac / Windows](https://www.shotai.io) |\n| ffmpeg | Clip extraction and keyframe analysis | Mac: `brew install ffmpeg` / Win: `winget install ffmpeg` |\n| yt-dlp | Auto background music from YouTube | Mac: `brew install yt-dlp` / Win: `winget install yt-dlp` |\n| Node.js 18+ | Runtime | Mac: `brew install node` / Win: `winget install OpenJS.NodeJS` |\n\n### ShotAI Setup\n\n1. Download and open [ShotAI](https://www.shotai.io), add your video footage folders to a collection\n2. Wait for indexing (shot detection + semantic embeddings — automatic, takes a few minutes for large libraries)\n3. **Settings → MCP Server → Enable**\n4. Note your **MCP URL** (default: `http://127.0.0.1:23817`) and **MCP Token**\n\n---\n\n## Quick Start\n\n```bash\ngit clone https://github.com/abu-ShotAI/ai-video-remix.git\ncd ai-video-editor\nnpm install\ncp .env.example .env # fill in SHOTAI_URL, SHOTAI_TOKEN, and optionally AGENT_PROVIDER\n```\n\n```bash\n# Nature documentary (no LLM required)\nAGENT_PROVIDER=none npx tsx src/skill/cli.ts \"自然野生动物纪录片\" --composition NatureWild\n\n# Sports highlight reel (generic — works with any sport footage)\nnpx tsx src/skill/cli.ts \"世界杯足球精彩时刻混剪\" --composition SportsHighlight\n\n# Travel vlog with English captions\nnpx tsx src/skill/cli.ts \"Japan and Paris travel highlights\" --composition TravelVlog --lang en\n\n# Cyberpunk city night cuts\nnpx tsx src/skill/cli.ts \"香港赛博朋克夜景混剪\" --composition CyberpunkCity\n\n# With local music file\nnpx tsx src/skill/cli.ts \"scenic alpine journey\" --composition SwitzerlandScenic --bgm ./music/alpine.mp3\n```\n\n---\n\n## Pipeline\n\nHow AI Video Remix turns a text prompt into a finished video:\n\n```\nUser prompt\n │\n ▼\n1. parseIntent — LLM extracts theme, selects composition, optionally overrides music style\n2. refineQueries — LLM rewrites per-slot search terms to match library content\n3. pickShots — ShotAI semantic search across your video footage library; scored by similarity + duration + mood\n4. resolveMusic — yt-dlp YouTube search+download, or local --bgm file\n5. extractClip — ffmpeg trims each shot to an independent .mp4\n6. annotateClips — LLM assigns per-clip visual params (tone, kenBurns, dramatic, caption)\n7. File Server — HTTP server serves clips to the Remotion renderer\n8. Remotion render — Final MP4 composed and rendered\n```\n\n---\n\n## CLI Reference\n\n```bash\nnpx tsx src/skill/cli.ts \"<request>\" [options]\n\nOptions:\n --composition <id> Force a specific composition (skip LLM selection)\n --bgm <path> Local MP3 path (skip YouTube search)\n --lang <zh|en> Caption language — zh (default) or en\n --output <dir> Output directory (default: ./output)\n --probe Scan library first; LLM plans slots from actual content\n```\n\n---\n\n## Compositions\n\n| ID | Style | Best For |\n|----|-------|----------|\n| `CyberpunkCity` | Cyberpunk night | Neon city, night scenes, sci-fi |\n| `TravelVlog` | Travel vlog | Multi-city travel with location cards |\n| `MoodDriven` | Mood-driven cuts | Emotional fast/slow montage |\n| `NatureWild` | BBC nature doc | Wildlife, landscapes, nature footage |\n| `SwitzerlandScenic` | Alpine scenic | Mountain travel with elegant captions |\n| `SportsHighlight` | ESPN sports | Goal/action highlights with captions |\n\n---\n\n## Modes\n\n**Standard mode** (default) — LLM picks the composition and generates search queries from registry templates.\n\n**Probe mode** (`--probe`) — Scans the library first (video names, shot samples, mood/scene tags), then LLM builds custom slots tailored to what actually exists. Use this when:\n- Library content is unknown or varied\n- User wants \"best of my library\"\n- Standard queries return low-quality shots\n\n---\n\n## Configuration\n\nEdit `.env` (copy from `.env.example`):\n\n```env\n# ── LLM Agent ────────────────────────────────────────────────────────────────\nAGENT_PROVIDER=claude # claude | openai | openai-compat | none\nANTHROPIC_API_KEY=sk-ant-... # required when AGENT_PROVIDER=claude\nOPENAI_API_KEY=sk-... # required when AGENT_PROVIDER=openai\nOPENAI_COMPAT_BASE_URL=https://... # required when AGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=claude-sonnet-4-6 # override default model\n\n# ── ShotAI ───────────────────────────────────────────────────────────────────\nSHOTAI_URL=http://127.0.0.1:23817\nSHOTAI_TOKEN=<your-token>\n\n# ── Music ────────────────────────────────────────────────────────────────────\nBGM_PATH=/path/to/music.mp3 # permanent local BGM default\n\n# ── Quality ──────────────────────────────────────────────────────────────────\nMIN_SCORE=0.5 # shot quality threshold 0–1 (recommended: 0.5)\n```\n\n### LLM Providers\n\n<details>\n<summary>Claude (Anthropic)</summary>\n\n```env\nAGENT_PROVIDER=claude\nANTHROPIC_API_KEY=sk-ant-...\nAGENT_MODEL=claude-sonnet-4-6\n```\n</details>\n\n<details>\n<summary>OpenAI</summary>\n\n```env\nAGENT_PROVIDER=openai\nOPENAI_API_KEY=sk-...\nAGENT_MODEL=gpt-4o\n```\n</details>\n\n<details>\n<summary>OpenRouter (recommended for multi-provider access)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://openrouter.ai/api/v1\nOPENAI_COMPAT_API_KEY=sk-or-v1-...\nAGENT_MODEL=deepseek/deepseek-chat-v3-0324\n```\n</details>\n\n<details>\n<summary>Ollama (local, no API key needed)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=http://localhost:11434/v1\nOPENAI_COMPAT_API_KEY=ollama\nAGENT_MODEL=llama3.1\n```\n</details>\n\n<details>\n<summary>DeepSeek (direct)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://api.deepseek.com/v1\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=deepseek-chat\n```\n</details>\n\n<details>\n<summary>No LLM (heuristic fallback)</summary>\n\n```env\nAGENT_PROVIDER=none\n```\nKeyword-based composition selection + registry default queries. No API key required.\n</details>\n\n---\n\n## Troubleshooting\n\n### Clip boundary flicker (1–2 frame flash at cuts)\n\nAn 80ms head/tail trim is applied automatically (`TRIM = 0.08`). If it persists, increase `TRIM` to `0.12` or `0.15` in `src/skill/orchestrator.ts`.\n\n### Red flash in CyberpunkCity\n\n`GlitchFlicker` triggers on very short clips. Set `MIN_SCORE=0.5` in `.env` to keep short clips out of the pipeline.\n\n### Low-quality or off-topic shots\n\n1. Raise `MIN_SCORE` (try `0.5` → `0.7`)\n2. Use `--probe` mode — LLM sees your actual library before picking queries\n3. Force `--composition <id>` to a composition whose slots match your content\n\n### Music download fails\n\n```bash\npip install -U yt-dlp # update yt-dlp (requires 2026.03.03+)\n# or use a local file:\nnpx tsx src/skill/cli.ts \"...\" --bgm /path/to/music.mp3\n```\n\nIf yt-dlp reports `n challenge solving failed`, it needs the EJS remote solver. This is handled automatically with `--remote-components ejs:github` (already set in the code).\n\n---\n\n## Performance\n\n| Step | Typical time (M-series Mac) |\n|------|-----------------------------|\n| Remotion render (60s video) | 30–90s |\n| ShotAI search per slot | 1–3s |\n| ffmpeg clip extraction | ~0.5s per clip |\n\n---\n\n## Adding a New Composition\n\nSee [references/composition-guide.md](references/composition-guide.md) for step-by-step instructions on adding a new Remotion visual style + registry entry.\n"}, "793195161979da426d9218655f738605d93d6654": {"t": 1785763892.0118115, "n": 8754, "h": "572107b3bed9801b", "c": "# AI Video Remix\n\n> AI-driven video remix generator — semantic video search + LLM planning + Remotion rendering.\n>\n> Requires [ShotAI](https://www.shotai.io) — local video asset management and footage search for Mac & Windows.\n>\n> 中文文档: [README.zh.md](README.zh.md)\n\nGenerate styled video compositions from your local video footage library using natural language. [ShotAI](https://www.shotai.io) handles shot-level indexing and semantic search; this tool handles the planning, music, and rendering.\n\n---\n\n## Demo\n\n> Hong Kong Cyberpunk Night — generated from local video footage with ShotAI + Remotion, no manual editing.\n\n[](https://www.youtube.com/watch?v=mibbqDf6uQY)\n\n---\n\n## Use as a Claude Skill\n\nThis repo ships a ready-to-install [Claude Agent Skill](https://support.claude.com/en/articles/12512176-what-are-skills) in the [`skill/`](skill/) directory.\n\n**Install in Claude Code:**\n```bash\n/plugin install ai-video-remix@abu-ShotAI/ai-video-remix#skill\n```\n\nOr point Claude Code settings to the local `skill/` folder.\n\nOnce installed, just describe what you want:\n> *\"Make a travel vlog from my library\"*\n> *\"Create a cyberpunk city highlight reel\"*\n> *\"Sports highlight from last weekend's footage\"*\n\n---\n\n## Prerequisites\n\n| Tool | Purpose | Install |\n|------|---------|---------|\n| [ShotAI](https://www.shotai.io) | AI video asset management + semantic footage search — provides the MCP server this tool queries | [Download for Mac / Windows](https://www.shotai.io) |\n| ffmpeg | Clip extraction and keyframe analysis | Mac: `brew install ffmpeg` / Win: `winget install ffmpeg` |\n| yt-dlp | Auto background music from YouTube | Mac: `brew install yt-dlp` / Win: `winget install yt-dlp` |\n| Node.js 18+ | Runtime | Mac: `brew install node` / Win: `winget install OpenJS.NodeJS` |\n\n### ShotAI Setup\n\n1. Download and open [ShotAI](https://www.shotai.io), add your video footage folders to a collection\n2. Wait for indexing (shot detection + semantic embeddings — automatic, takes a few minutes for large libraries)\n3. **Settings → MCP Server → Enable**\n4. Note your **MCP URL** (default: `http://127.0.0.1:23817`) and **MCP Token**\n\n---\n\n## Quick Start\n\n```bash\ngit clone https://github.com/abu-ShotAI/ai-video-remix.git\ncd ai-video-editor\nnpm install\ncp .env.example .env # fill in SHOTAI_URL, SHOTAI_TOKEN, and optionally AGENT_PROVIDER\n```\n\n```bash\n# Nature documentary (no LLM required)\nAGENT_PROVIDER=none npx tsx src/skill/cli.ts \"自然野生动物纪录片\" --composition NatureWild\n\n# Sports highlight reel (generic — works with any sport footage)\nnpx tsx src/skill/cli.ts \"世界杯足球精彩时刻混剪\" --composition SportsHighlight\n\n# Travel vlog with English captions\nnpx tsx src/skill/cli.ts \"Japan and Paris travel highlights\" --composition TravelVlog --lang en\n\n# Cyberpunk city night cuts\nnpx tsx src/skill/cli.ts \"香港赛博朋克夜景混剪\" --composition CyberpunkCity\n\n# With local music file\nnpx tsx src/skill/cli.ts \"scenic alpine journey\" --composition SwitzerlandScenic --bgm ./music/alpine.mp3\n```\n\n---\n\n## Pipeline\n\nHow AI Video Remix turns a text prompt into a finished video:\n\n```\nUser prompt\n │\n ▼\n1. parseIntent — LLM extracts theme, selects composition, optionally overrides music style\n2. refineQueries — LLM rewrites per-slot search terms to match library content\n3. pickShots — ShotAI semantic search across your video footage library; scored by similarity + duration + mood\n4. resolveMusic — yt-dlp YouTube search+download, or local --bgm file\n5. extractClip — ffmpeg trims each shot to an independent .mp4\n6. annotateClips — LLM assigns per-clip visual params (tone, kenBurns, dramatic, caption)\n7. File Server — HTTP server serves clips to the Remotion renderer\n8. Remotion render — Final MP4 composed and rendered\n```\n\n---\n\n## CLI Reference\n\n```bash\nnpx tsx src/skill/cli.ts \"<request>\" [options]\n\nOptions:\n --composition <id> Force a specific composition (skip LLM selection)\n --bgm <path> Local MP3 path (skip YouTube search)\n --lang <zh|en> Caption language — zh (default) or en\n --output <dir> Output directory (default: ./output)\n --probe Scan library first; LLM plans slots from actual content\n```\n\n---\n\n## Compositions\n\n| ID | Style | Best For |\n|----|-------|----------|\n| `CyberpunkCity` | Cyberpunk night | Neon city, night scenes, sci-fi |\n| `TravelVlog` | Travel vlog | Multi-city travel with location cards |\n| `MoodDriven` | Mood-driven cuts | Emotional fast/slow montage |\n| `NatureWild` | BBC nature doc | Wildlife, landscapes, nature footage |\n| `SwitzerlandScenic` | Alpine scenic | Mountain travel with elegant captions |\n| `SportsHighlight` | ESPN sports | Goal/action highlights with captions |\n\n---\n\n## Modes\n\n**Standard mode** (default) — LLM picks the composition and generates search queries from registry templates.\n\n**Probe mode** (`--probe`) — Scans the library first (video names, shot samples, mood/scene tags), then LLM builds custom slots tailored to what actually exists. Use this when:\n- Library content is unknown or varied\n- User wants \"best of my library\"\n- Standard queries return low-quality shots\n\n---\n\n## Configuration\n\nEdit `.env` (copy from `.env.example`):\n\n```env\n# ── LLM Agent ────────────────────────────────────────────────────────────────\nAGENT_PROVIDER=claude # claude | openai | openai-compat | none\nANTHROPIC_API_KEY=sk-ant-... # required when AGENT_PROVIDER=claude\nOPENAI_API_KEY=sk-... # required when AGENT_PROVIDER=openai\nOPENAI_COMPAT_BASE_URL=https://... # required when AGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=claude-sonnet-4-6 # override default model\n\n# ── ShotAI ───────────────────────────────────────────────────────────────────\nSHOTAI_URL=http://127.0.0.1:23817\nSHOTAI_TOKEN=<your-token>\n\n# ── Music ────────────────────────────────────────────────────────────────────\nBGM_PATH=/path/to/music.mp3 # permanent local BGM default\n\n# ── Quality ──────────────────────────────────────────────────────────────────\nMIN_SCORE=0.5 # shot quality threshold 0–1 (recommended: 0.5)\n```\n\n### LLM Providers\n\n<details>\n<summary>Claude (Anthropic)</summary>\n\n```env\nAGENT_PROVIDER=claude\nANTHROPIC_API_KEY=sk-ant-...\nAGENT_MODEL=claude-sonnet-4-6\n```\n</details>\n\n<details>\n<summary>OpenAI</summary>\n\n```env\nAGENT_PROVIDER=openai\nOPENAI_API_KEY=sk-...\nAGENT_MODEL=gpt-4o\n```\n</details>\n\n<details>\n<summary>OpenRouter (recommended for multi-provider access)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://openrouter.ai/api/v1\nOPENAI_COMPAT_API_KEY=sk-or-v1-...\nAGENT_MODEL=deepseek/deepseek-chat-v3-0324\n```\n</details>\n\n<details>\n<summary>Ollama (local, no API key needed)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=http://localhost:11434/v1\nOPENAI_COMPAT_API_KEY=ollama\nAGENT_MODEL=llama3.1\n```\n</details>\n\n<details>\n<summary>DeepSeek (direct)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://api.deepseek.com/v1\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=deepseek-chat\n```\n</details>\n\n<details>\n<summary>No LLM (heuristic fallback)</summary>\n\n```env\nAGENT_PROVIDER=none\n```\nKeyword-based composition selection + registry default queries. No API key required.\n</details>\n\n---\n\n## Troubleshooting\n\n### Clip boundary flicker (1–2 frame flash at cuts)\n\nAn 80ms head/tail trim is applied automatically (`TRIM = 0.08`). If it persists, increase `TRIM` to `0.12` or `0.15` in `src/skill/orchestrator.ts`.\n\n### Red flash in CyberpunkCity\n\n`GlitchFlicker` triggers on very short clips. Set `MIN_SCORE=0.5` in `.env` to keep short clips out of the pipeline.\n\n### Low-quality or off-topic shots\n\n1. Raise `MIN_SCORE` (try `0.5` → `0.7`)\n2. Use `--probe` mode — LLM sees your actual library before picking queries\n3. Force `--composition <id>` to a composition whose slots match your content\n\n### Music download fails\n\n```bash\npip install -U yt-dlp # update yt-dlp (requires 2026.03.03+)\n# or use a local file:\nnpx tsx src/skill/cli.ts \"...\" --bgm /path/to/music.mp3\n```\n\nIf yt-dlp reports `n challenge solving failed`, it needs the EJS remote solver. This is handled automatically with `--remote-components ejs:github` (already set in the code).\n\n---\n\n## Performance\n\n| Step | Typical time (M-series Mac) |\n|------|-----------------------------|\n| Remotion render (60s video) | 30–90s |\n| ShotAI search per slot | 1–3s |\n| ffmpeg clip extraction | ~0.5s per clip |\n\n---\n\n## Adding a New Composition\n\nSee [references/composition-guide.md](references/composition-guide.md) for step-by-step instructions on adding a new Remotion visual style + registry entry.\n"}, "cc66ef0e67f0ea0325b7d5a642bbcf0cd6e21b4b": {"t": 1785763893.4152427, "n": 8619, "h": "18291d76ac8aab31", "c": "# AI Video Remix\n\n> AI-driven video remix generator — semantic video search + LLM planning + Remotion rendering.\n>\n> Requires [ShotAI](https://www.shotai.io) — local video asset management and footage search for Mac.\n>\n> 中文文档: [README.zh.md](README.zh.md)\n\nGenerate styled video compositions from your local video footage library using natural language. [ShotAI](https://www.shotai.io) handles shot-level indexing and semantic search; this tool handles the planning, music, and rendering.\n\n---\n\n## Demo\n\n> Hong Kong Cyberpunk Night — generated from local video footage with ShotAI + Remotion, no manual editing.\n\n[](https://www.youtube.com/watch?v=mibbqDf6uQY)\n\n---\n\n## Use as a Claude Skill\n\nThis repo ships a ready-to-install [Claude Agent Skill](https://support.claude.com/en/articles/12512176-what-are-skills) in the [`skill/`](skill/) directory.\n\n**Install in Claude Code:**\n```bash\n/plugin install ai-video-remix@abu-ShotAI/ai-video-remix#skill\n```\n\nOr point Claude Code settings to the local `skill/` folder.\n\nOnce installed, just describe what you want:\n> *\"Make a travel vlog from my library\"*\n> *\"Create a cyberpunk city highlight reel\"*\n> *\"Sports highlight from last weekend's footage\"*\n\n---\n\n## Prerequisites\n\n| Tool | Purpose | Install |\n|------|---------|---------|\n| [ShotAI](https://www.shotai.io) | AI video asset management + semantic footage search — provides the MCP server this tool queries | [Download for Mac](https://www.shotai.io) |\n| ffmpeg | Clip extraction and keyframe analysis | `brew install ffmpeg` |\n| yt-dlp | Auto background music from YouTube | `brew install yt-dlp` |\n| Node.js 18+ | Runtime | `brew install node` |\n\n### ShotAI Setup\n\n1. Download and open [ShotAI](https://www.shotai.io), add your video footage folders to a collection\n2. Wait for indexing (shot detection + semantic embeddings — automatic, takes a few minutes for large libraries)\n3. **Settings → MCP Server → Enable**\n4. Note your **MCP URL** (default: `http://127.0.0.1:23817`) and **MCP Token**\n\n---\n\n## Quick Start\n\n```bash\ngit clone https://github.com/abu-ShotAI/ai-video-remix.git\ncd ai-video-editor\nnpm install\ncp .env.example .env # fill in SHOTAI_URL, SHOTAI_TOKEN, and optionally AGENT_PROVIDER\n```\n\n```bash\n# Nature documentary (no LLM required)\nAGENT_PROVIDER=none npx tsx src/skill/cli.ts \"自然野生动物纪录片\" --composition NatureWild\n\n# Sports highlight reel (generic — works with any sport footage)\nnpx tsx src/skill/cli.ts \"世界杯足球精彩时刻混剪\" --composition SportsHighlight\n\n# Travel vlog with English captions\nnpx tsx src/skill/cli.ts \"Japan and Paris travel highlights\" --composition TravelVlog --lang en\n\n# Cyberpunk city night cuts\nnpx tsx src/skill/cli.ts \"香港赛博朋克夜景混剪\" --composition CyberpunkCity\n\n# With local music file\nnpx tsx src/skill/cli.ts \"scenic alpine journey\" --composition SwitzerlandScenic --bgm ./music/alpine.mp3\n```\n\n---\n\n## Pipeline\n\nHow AI Video Remix turns a text prompt into a finished video:\n\n```\nUser prompt\n │\n ▼\n1. parseIntent — LLM extracts theme, selects composition, optionally overrides music style\n2. refineQueries — LLM rewrites per-slot search terms to match library content\n3. pickShots — ShotAI semantic search across your video footage library; scored by similarity + duration + mood\n4. resolveMusic — yt-dlp YouTube search+download, or local --bgm file\n5. extractClip — ffmpeg trims each shot to an independent .mp4\n6. annotateClips — LLM assigns per-clip visual params (tone, kenBurns, dramatic, caption)\n7. File Server — HTTP server serves clips to the Remotion renderer\n8. Remotion render — Final MP4 composed and rendered\n```\n\n---\n\n## CLI Reference\n\n```bash\nnpx tsx src/skill/cli.ts \"<request>\" [options]\n\nOptions:\n --composition <id> Force a specific composition (skip LLM selection)\n --bgm <path> Local MP3 path (skip YouTube search)\n --lang <zh|en> Caption language — zh (default) or en\n --output <dir> Output directory (default: ./output)\n --probe Scan library first; LLM plans slots from actual content\n```\n\n---\n\n## Compositions\n\n| ID | Style | Best For |\n|----|-------|----------|\n| `CyberpunkCity` | Cyberpunk night | Neon city, night scenes, sci-fi |\n| `TravelVlog` | Travel vlog | Multi-city travel with location cards |\n| `MoodDriven` | Mood-driven cuts | Emotional fast/slow montage |\n| `NatureWild` | BBC nature doc | Wildlife, landscapes, nature footage |\n| `SwitzerlandScenic` | Alpine scenic | Mountain travel with elegant captions |\n| `SportsHighlight` | ESPN sports | Goal/action highlights with captions |\n\n---\n\n## Modes\n\n**Standard mode** (default) — LLM picks the composition and generates search queries from registry templates.\n\n**Probe mode** (`--probe`) — Scans the library first (video names, shot samples, mood/scene tags), then LLM builds custom slots tailored to what actually exists. Use this when:\n- Library content is unknown or varied\n- User wants \"best of my library\"\n- Standard queries return low-quality shots\n\n---\n\n## Configuration\n\nEdit `.env` (copy from `.env.example`):\n\n```env\n# ── LLM Agent ────────────────────────────────────────────────────────────────\nAGENT_PROVIDER=claude # claude | openai | openai-compat | none\nANTHROPIC_API_KEY=sk-ant-... # required when AGENT_PROVIDER=claude\nOPENAI_API_KEY=sk-... # required when AGENT_PROVIDER=openai\nOPENAI_COMPAT_BASE_URL=https://... # required when AGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=claude-sonnet-4-6 # override default model\n\n# ── ShotAI ───────────────────────────────────────────────────────────────────\nSHOTAI_URL=http://127.0.0.1:23817\nSHOTAI_TOKEN=<your-token>\n\n# ── Music ────────────────────────────────────────────────────────────────────\nBGM_PATH=/path/to/music.mp3 # permanent local BGM default\n\n# ── Quality ──────────────────────────────────────────────────────────────────\nMIN_SCORE=0.5 # shot quality threshold 0–1 (recommended: 0.5)\n```\n\n### LLM Providers\n\n<details>\n<summary>Claude (Anthropic)</summary>\n\n```env\nAGENT_PROVIDER=claude\nANTHROPIC_API_KEY=sk-ant-...\nAGENT_MODEL=claude-sonnet-4-6\n```\n</details>\n\n<details>\n<summary>OpenAI</summary>\n\n```env\nAGENT_PROVIDER=openai\nOPENAI_API_KEY=sk-...\nAGENT_MODEL=gpt-4o\n```\n</details>\n\n<details>\n<summary>OpenRouter (recommended for multi-provider access)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://openrouter.ai/api/v1\nOPENAI_COMPAT_API_KEY=sk-or-v1-...\nAGENT_MODEL=deepseek/deepseek-chat-v3-0324\n```\n</details>\n\n<details>\n<summary>Ollama (local, no API key needed)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=http://localhost:11434/v1\nOPENAI_COMPAT_API_KEY=ollama\nAGENT_MODEL=llama3.1\n```\n</details>\n\n<details>\n<summary>DeepSeek (direct)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://api.deepseek.com/v1\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=deepseek-chat\n```\n</details>\n\n<details>\n<summary>No LLM (heuristic fallback)</summary>\n\n```env\nAGENT_PROVIDER=none\n```\nKeyword-based composition selection + registry default queries. No API key required.\n</details>\n\n---\n\n## Troubleshooting\n\n### Clip boundary flicker (1–2 frame flash at cuts)\n\nAn 80ms head/tail trim is applied automatically (`TRIM = 0.08`). If it persists, increase `TRIM` to `0.12` or `0.15` in `src/skill/orchestrator.ts`.\n\n### Red flash in CyberpunkCity\n\n`GlitchFlicker` triggers on very short clips. Set `MIN_SCORE=0.5` in `.env` to keep short clips out of the pipeline.\n\n### Low-quality or off-topic shots\n\n1. Raise `MIN_SCORE` (try `0.5` → `0.7`)\n2. Use `--probe` mode — LLM sees your actual library before picking queries\n3. Force `--composition <id>` to a composition whose slots match your content\n\n### Music download fails\n\n```bash\npip install -U yt-dlp # update yt-dlp (requires 2026.03.03+)\n# or use a local file:\nnpx tsx src/skill/cli.ts \"...\" --bgm /path/to/music.mp3\n```\n\nIf yt-dlp reports `n challenge solving failed`, it needs the EJS remote solver. This is handled automatically with `--remote-components ejs:github` (already set in the code).\n\n---\n\n## Performance\n\n| Step | Typical time (M-series Mac) |\n|------|-----------------------------|\n| Remotion render (60s video) | 30–90s |\n| ShotAI search per slot | 1–3s |\n| ffmpeg clip extraction | ~0.5s per clip |\n\n---\n\n## Adding a New Composition\n\nSee [references/composition-guide.md](references/composition-guide.md) for step-by-step instructions on adding a new Remotion visual style + registry entry.\n"}, "b191e862ec41f82418777f7c9191b4546c6cabff": {"t": 1785763894.6798491, "n": 8146, "h": "564b356f26ceb05f", "c": "# AI Video Remix\n\n> AI-driven video remix generator — semantic search + LLM planning + Remotion rendering.\n>\n> 中文文档: [README.zh.md](README.zh.md)\n\nGenerate styled video compositions from your local ShotAI video library using natural language.\n\n---\n\n## Demo\n\n> Hong Kong Cyberpunk Night — generated with ShotAI MCP + Remotion, no manual editing.\n\n[](https://www.youtube.com/watch?v=mibbqDf6uQY)\n\n---\n\n## Use as a Claude Skill\n\nThis repo ships a ready-to-install [Claude Agent Skill](https://support.claude.com/en/articles/12512176-what-are-skills) in the [`skill/`](skill/) directory.\n\n**Install in Claude Code:**\n```bash\n/plugin install ai-video-remix@abu-ShotAI/ai-video-remix#skill\n```\n\nOr point Claude Code settings to the local `skill/` folder.\n\nOnce installed, just describe what you want:\n> *\"Make a travel vlog from my library\"*\n> *\"Create a cyberpunk city highlight reel\"*\n> *\"Sports highlight from last weekend's footage\"*\n\n---\n\n## Prerequisites\n\n| Tool | Purpose | Install |\n|------|---------|---------|\n| [ShotAI](https://www.shotai.io) | Local video asset management + semantic shot search (MCP server) | Download from website |\n| ffmpeg | Clip extraction and keyframe analysis | `brew install ffmpeg` |\n| yt-dlp | Auto background music from YouTube | `brew install yt-dlp` |\n| Node.js 18+ | Runtime | `brew install node` |\n\n### ShotAI Setup\n\n1. Download and open ShotAI, add your video files/folders to a collection\n2. Wait for indexing (shot detection + embeddings — takes a few minutes)\n3. **Settings → MCP Server → Enable**\n4. Note your **MCP URL** (default: `http://127.0.0.1:23817`) and **MCP Token**\n\n---\n\n## Quick Start\n\n```bash\ngit clone https://github.com/abu-ShotAI/ai-video-remix.git\ncd ai-video-editor\nnpm install\ncp .env.example .env # fill in SHOTAI_URL, SHOTAI_TOKEN, and optionally AGENT_PROVIDER\n```\n\n```bash\n# Nature documentary (no LLM required)\nAGENT_PROVIDER=none npx tsx src/skill/cli.ts \"自然野生动物纪录片\" --composition NatureWild\n\n# Sports highlight reel (generic — works with any sport footage)\nnpx tsx src/skill/cli.ts \"世界杯足球精彩时刻混剪\" --composition SportsHighlight\n\n# Travel vlog with English captions\nnpx tsx src/skill/cli.ts \"Japan and Paris travel highlights\" --composition TravelVlog --lang en\n\n# Cyberpunk city night cuts\nnpx tsx src/skill/cli.ts \"香港赛博朋克夜景混剪\" --composition CyberpunkCity\n\n# With local music file\nnpx tsx src/skill/cli.ts \"scenic alpine journey\" --composition SwitzerlandScenic --bgm ./music/alpine.mp3\n```\n\n---\n\n## Pipeline\n\n```\nUser prompt\n │\n ▼\n1. parseIntent — LLM extracts theme, selects composition, optionally overrides music style\n2. refineQueries — LLM rewrites per-slot search terms to match library content\n3. pickShots — ShotAI semantic search per slot; scored by similarity + duration + mood\n4. resolveMusic — yt-dlp YouTube search+download, or local --bgm file\n5. extractClip — ffmpeg trims each shot to an independent .mp4\n6. annotateClips — LLM assigns per-clip visual params (tone, kenBurns, dramatic, caption)\n7. File Server — HTTP server serves clips to the Remotion renderer\n8. Remotion render — Final MP4 composed and rendered\n```\n\n---\n\n## CLI Reference\n\n```bash\nnpx tsx src/skill/cli.ts \"<request>\" [options]\n\nOptions:\n --composition <id> Force a specific composition (skip LLM selection)\n --bgm <path> Local MP3 path (skip YouTube search)\n --lang <zh|en> Caption language — zh (default) or en\n --output <dir> Output directory (default: ./output)\n --probe Scan library first; LLM plans slots from actual content\n```\n\n---\n\n## Compositions\n\n| ID | Style | Best For |\n|----|-------|----------|\n| `CyberpunkCity` | Cyberpunk night | Neon city, night scenes, sci-fi |\n| `TravelVlog` | Travel vlog | Multi-city travel with location cards |\n| `MoodDriven` | Mood-driven cuts | Emotional fast/slow montage |\n| `NatureWild` | BBC nature doc | Wildlife, landscapes, nature footage |\n| `SwitzerlandScenic` | Alpine scenic | Mountain travel with elegant captions |\n| `SportsHighlight` | ESPN sports | Goal/action highlights with captions |\n\n---\n\n## Modes\n\n**Standard mode** (default) — LLM picks the composition and generates search queries from registry templates.\n\n**Probe mode** (`--probe`) — Scans the library first (video names, shot samples, mood/scene tags), then LLM builds custom slots tailored to what actually exists. Use this when:\n- Library content is unknown or varied\n- User wants \"best of my library\"\n- Standard queries return low-quality shots\n\n---\n\n## Configuration\n\nEdit `.env` (copy from `.env.example`):\n\n```env\n# ── LLM Agent ────────────────────────────────────────────────────────────────\nAGENT_PROVIDER=claude # claude | openai | openai-compat | none\nANTHROPIC_API_KEY=sk-ant-... # required when AGENT_PROVIDER=claude\nOPENAI_API_KEY=sk-... # required when AGENT_PROVIDER=openai\nOPENAI_COMPAT_BASE_URL=https://... # required when AGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=claude-sonnet-4-6 # override default model\n\n# ── ShotAI ───────────────────────────────────────────────────────────────────\nSHOTAI_URL=http://127.0.0.1:23817\nSHOTAI_TOKEN=<your-token>\n\n# ── Music ────────────────────────────────────────────────────────────────────\nBGM_PATH=/path/to/music.mp3 # permanent local BGM default\n\n# ── Quality ──────────────────────────────────────────────────────────────────\nMIN_SCORE=0.5 # shot quality threshold 0–1 (recommended: 0.5)\n```\n\n### LLM Providers\n\n<details>\n<summary>Claude (Anthropic)</summary>\n\n```env\nAGENT_PROVIDER=claude\nANTHROPIC_API_KEY=sk-ant-...\nAGENT_MODEL=claude-sonnet-4-6\n```\n</details>\n\n<details>\n<summary>OpenAI</summary>\n\n```env\nAGENT_PROVIDER=openai\nOPENAI_API_KEY=sk-...\nAGENT_MODEL=gpt-4o\n```\n</details>\n\n<details>\n<summary>OpenRouter (recommended for multi-provider access)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://openrouter.ai/api/v1\nOPENAI_COMPAT_API_KEY=sk-or-v1-...\nAGENT_MODEL=deepseek/deepseek-chat-v3-0324\n```\n</details>\n\n<details>\n<summary>Ollama (local, no API key needed)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=http://localhost:11434/v1\nOPENAI_COMPAT_API_KEY=ollama\nAGENT_MODEL=llama3.1\n```\n</details>\n\n<details>\n<summary>DeepSeek (direct)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://api.deepseek.com/v1\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=deepseek-chat\n```\n</details>\n\n<details>\n<summary>No LLM (heuristic fallback)</summary>\n\n```env\nAGENT_PROVIDER=none\n```\nKeyword-based composition selection + registry default queries. No API key required.\n</details>\n\n---\n\n## Troubleshooting\n\n### Clip boundary flicker (1–2 frame flash at cuts)\n\nAn 80ms head/tail trim is applied automatically (`TRIM = 0.08`). If it persists, increase `TRIM` to `0.12` or `0.15` in `src/skill/orchestrator.ts`.\n\n### Red flash in CyberpunkCity\n\n`GlitchFlicker` triggers on very short clips. Set `MIN_SCORE=0.5` in `.env` to keep short clips out of the pipeline.\n\n### Low-quality or off-topic shots\n\n1. Raise `MIN_SCORE` (try `0.5` → `0.7`)\n2. Use `--probe` mode — LLM sees your actual library before picking queries\n3. Force `--composition <id>` to a composition whose slots match your content\n\n### Music download fails\n\n```bash\npip install -U yt-dlp # update yt-dlp (requires 2026.03.03+)\n# or use a local file:\nnpx tsx src/skill/cli.ts \"...\" --bgm /path/to/music.mp3\n```\n\nIf yt-dlp reports `n challenge solving failed`, it needs the EJS remote solver. This is handled automatically with `--remote-components ejs:github` (already set in the code).\n\n---\n\n## Performance\n\n| Step | Typical time (M-series Mac) |\n|------|-----------------------------|\n| Remotion render (60s video) | 30–90s |\n| ShotAI search per slot | 1–3s |\n| ffmpeg clip extraction | ~0.5s per clip |\n\n---\n\n## Adding a New Composition\n\nSee [references/composition-guide.md](references/composition-guide.md) for step-by-step instructions on adding a new Remotion visual style + registry entry.\n"}, "a02111d30d8e6b54962dc2a1a93a301ae38b5e74": {"t": 1785763897.3131554, "n": 7892, "h": "c21557ebb56a8a37", "c": "# AI Video Remix\n\n> AI-driven video remix generator — semantic search + LLM planning + Remotion rendering.\n>\n> 中文文档: [README.zh.md](README.zh.md)\n\nGenerate styled video compositions from your local ShotAI video library using natural language.\n\n---\n\n## Use as a Claude Skill\n\nThis repo ships a ready-to-install [Claude Agent Skill](https://support.claude.com/en/articles/12512176-what-are-skills) in the [`skill/`](skill/) directory.\n\n**Install in Claude Code:**\n```bash\n/plugin install ai-video-remix@abu-ShotAI/ai-video-remix#skill\n```\n\nOr point Claude Code settings to the local `skill/` folder.\n\nOnce installed, just describe what you want:\n> *\"Make a travel vlog from my library\"*\n> *\"Create a cyberpunk city highlight reel\"*\n> *\"Sports highlight from last weekend's footage\"*\n\n---\n\n## Prerequisites\n\n| Tool | Purpose | Install |\n|------|---------|---------|\n| [ShotAI](https://www.shotai.io) | Local video asset management + semantic shot search (MCP server) | Download from website |\n| ffmpeg | Clip extraction and keyframe analysis | `brew install ffmpeg` |\n| yt-dlp | Auto background music from YouTube | `brew install yt-dlp` |\n| Node.js 18+ | Runtime | `brew install node` |\n\n### ShotAI Setup\n\n1. Download and open ShotAI, add your video files/folders to a collection\n2. Wait for indexing (shot detection + embeddings — takes a few minutes)\n3. **Settings → MCP Server → Enable**\n4. Note your **MCP URL** (default: `http://127.0.0.1:23817`) and **MCP Token**\n\n---\n\n## Quick Start\n\n```bash\ngit clone https://github.com/abu-ShotAI/ai-video-remix.git\ncd ai-video-editor\nnpm install\ncp .env.example .env # fill in SHOTAI_URL, SHOTAI_TOKEN, and optionally AGENT_PROVIDER\n```\n\n```bash\n# Nature documentary (no LLM required)\nAGENT_PROVIDER=none npx tsx src/skill/cli.ts \"自然野生动物纪录片\" --composition NatureWild\n\n# Sports highlight reel (generic — works with any sport footage)\nnpx tsx src/skill/cli.ts \"世界杯足球精彩时刻混剪\" --composition SportsHighlight\n\n# Travel vlog with English captions\nnpx tsx src/skill/cli.ts \"Japan and Paris travel highlights\" --composition TravelVlog --lang en\n\n# Cyberpunk city night cuts\nnpx tsx src/skill/cli.ts \"香港赛博朋克夜景混剪\" --composition CyberpunkCity\n\n# With local music file\nnpx tsx src/skill/cli.ts \"scenic alpine journey\" --composition SwitzerlandScenic --bgm ./music/alpine.mp3\n```\n\n---\n\n## Pipeline\n\n```\nUser prompt\n │\n ▼\n1. parseIntent — LLM extracts theme, selects composition, optionally overrides music style\n2. refineQueries — LLM rewrites per-slot search terms to match library content\n3. pickShots — ShotAI semantic search per slot; scored by similarity + duration + mood\n4. resolveMusic — yt-dlp YouTube search+download, or local --bgm file\n5. extractClip — ffmpeg trims each shot to an independent .mp4\n6. annotateClips — LLM assigns per-clip visual params (tone, kenBurns, dramatic, caption)\n7. File Server — HTTP server serves clips to the Remotion renderer\n8. Remotion render — Final MP4 composed and rendered\n```\n\n---\n\n## CLI Reference\n\n```bash\nnpx tsx src/skill/cli.ts \"<request>\" [options]\n\nOptions:\n --composition <id> Force a specific composition (skip LLM selection)\n --bgm <path> Local MP3 path (skip YouTube search)\n --lang <zh|en> Caption language — zh (default) or en\n --output <dir> Output directory (default: ./output)\n --probe Scan library first; LLM plans slots from actual content\n```\n\n---\n\n## Compositions\n\n| ID | Style | Best For |\n|----|-------|----------|\n| `CyberpunkCity` | Cyberpunk night | Neon city, night scenes, sci-fi |\n| `TravelVlog` | Travel vlog | Multi-city travel with location cards |\n| `MoodDriven` | Mood-driven cuts | Emotional fast/slow montage |\n| `NatureWild` | BBC nature doc | Wildlife, landscapes, nature footage |\n| `SwitzerlandScenic` | Alpine scenic | Mountain travel with elegant captions |\n| `SportsHighlight` | ESPN sports | Goal/action highlights with captions |\n\n---\n\n## Modes\n\n**Standard mode** (default) — LLM picks the composition and generates search queries from registry templates.\n\n**Probe mode** (`--probe`) — Scans the library first (video names, shot samples, mood/scene tags), then LLM builds custom slots tailored to what actually exists. Use this when:\n- Library content is unknown or varied\n- User wants \"best of my library\"\n- Standard queries return low-quality shots\n\n---\n\n## Configuration\n\nEdit `.env` (copy from `.env.example`):\n\n```env\n# ── LLM Agent ────────────────────────────────────────────────────────────────\nAGENT_PROVIDER=claude # claude | openai | openai-compat | none\nANTHROPIC_API_KEY=sk-ant-... # required when AGENT_PROVIDER=claude\nOPENAI_API_KEY=sk-... # required when AGENT_PROVIDER=openai\nOPENAI_COMPAT_BASE_URL=https://... # required when AGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=claude-sonnet-4-6 # override default model\n\n# ── ShotAI ───────────────────────────────────────────────────────────────────\nSHOTAI_URL=http://127.0.0.1:23817\nSHOTAI_TOKEN=<your-token>\n\n# ── Music ────────────────────────────────────────────────────────────────────\nBGM_PATH=/path/to/music.mp3 # permanent local BGM default\n\n# ── Quality ──────────────────────────────────────────────────────────────────\nMIN_SCORE=0.5 # shot quality threshold 0–1 (recommended: 0.5)\n```\n\n### LLM Providers\n\n<details>\n<summary>Claude (Anthropic)</summary>\n\n```env\nAGENT_PROVIDER=claude\nANTHROPIC_API_KEY=sk-ant-...\nAGENT_MODEL=claude-sonnet-4-6\n```\n</details>\n\n<details>\n<summary>OpenAI</summary>\n\n```env\nAGENT_PROVIDER=openai\nOPENAI_API_KEY=sk-...\nAGENT_MODEL=gpt-4o\n```\n</details>\n\n<details>\n<summary>OpenRouter (recommended for multi-provider access)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://openrouter.ai/api/v1\nOPENAI_COMPAT_API_KEY=sk-or-v1-...\nAGENT_MODEL=deepseek/deepseek-chat-v3-0324\n```\n</details>\n\n<details>\n<summary>Ollama (local, no API key needed)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=http://localhost:11434/v1\nOPENAI_COMPAT_API_KEY=ollama\nAGENT_MODEL=llama3.1\n```\n</details>\n\n<details>\n<summary>DeepSeek (direct)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://api.deepseek.com/v1\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=deepseek-chat\n```\n</details>\n\n<details>\n<summary>No LLM (heuristic fallback)</summary>\n\n```env\nAGENT_PROVIDER=none\n```\nKeyword-based composition selection + registry default queries. No API key required.\n</details>\n\n---\n\n## Troubleshooting\n\n### Clip boundary flicker (1–2 frame flash at cuts)\n\nAn 80ms head/tail trim is applied automatically (`TRIM = 0.08`). If it persists, increase `TRIM` to `0.12` or `0.15` in `src/skill/orchestrator.ts`.\n\n### Red flash in CyberpunkCity\n\n`GlitchFlicker` triggers on very short clips. Set `MIN_SCORE=0.5` in `.env` to keep short clips out of the pipeline.\n\n### Low-quality or off-topic shots\n\n1. Raise `MIN_SCORE` (try `0.5` → `0.7`)\n2. Use `--probe` mode — LLM sees your actual library before picking queries\n3. Force `--composition <id>` to a composition whose slots match your content\n\n### Music download fails\n\n```bash\npip install -U yt-dlp # update yt-dlp (requires 2026.03.03+)\n# or use a local file:\nnpx tsx src/skill/cli.ts \"...\" --bgm /path/to/music.mp3\n```\n\nIf yt-dlp reports `n challenge solving failed`, it needs the EJS remote solver. This is handled automatically with `--remote-components ejs:github` (already set in the code).\n\n---\n\n## Performance\n\n| Step | Typical time (M-series Mac) |\n|------|-----------------------------|\n| Remotion render (60s video) | 30–90s |\n| ShotAI search per slot | 1–3s |\n| ffmpeg clip extraction | ~0.5s per clip |\n\n---\n\n## Adding a New Composition\n\nSee [references/composition-guide.md](references/composition-guide.md) for step-by-step instructions on adding a new Remotion visual style + registry entry.\n"}, "4e99c304047a3a0f81fb95c2a5027b3177f31124": {"t": 1785763901.4408574, "n": 7547, "h": "67578c67ca4b6072", "c": "# AI Video Remix\n\n> AI-driven video remix generator — semantic search + LLM planning + Remotion rendering.\n>\n> 中文文档: [README.zh.md](README.zh.md)\n\nGenerate styled video compositions from your local ShotAI video library using natural language.\n\n---\n\n## Use as a Claude Skill\n\nThis repo ships a ready-to-install [Claude Agent Skill](https://support.claude.com/en/articles/12512176-what-are-skills) in the [`skill/`](skill/) directory.\n\n**Install in Claude Code:**\n```bash\n/plugin install ai-video-remix@abu-ShotAI/ai-video-remix#skill\n```\n\nOr point Claude Code settings to the local `skill/` folder.\n\nOnce installed, just describe what you want:\n> *\"Make a travel vlog from my library\"*\n> *\"Create a cyberpunk city highlight reel\"*\n> *\"Sports highlight from last weekend's footage\"*\n\n---\n\n## Prerequisites\n\n| Tool | Purpose | Install |\n|------|---------|---------|\n| [ShotAI](https://www.shotai.io) | Local video asset management + semantic shot search (MCP server) | Download from website |\n| ffmpeg | Clip extraction and keyframe analysis | `brew install ffmpeg` |\n| yt-dlp | Auto background music from YouTube | `brew install yt-dlp` |\n| Node.js 18+ | Runtime | `brew install node` |\n\n### ShotAI Setup\n\n1. Download and open ShotAI, add your video files/folders to a collection\n2. Wait for indexing (shot detection + embeddings — takes a few minutes)\n3. **Settings → MCP Server → Enable**\n4. Note your **MCP URL** (default: `http://127.0.0.1:23817`) and **MCP Token**\n\n---\n\n## Quick Start\n\n```bash\ngit clone https://github.com/abu-ShotAI/ai-video-remix.git\ncd ai-video-editor\nnpm install\ncp .env.example .env # fill in SHOTAI_URL, SHOTAI_TOKEN, and optionally AGENT_PROVIDER\n```\n\n```bash\n# Travel vlog from your library\nnpx tsx src/skill/cli.ts \"make a travel vlog from my library\"\n\n# Sports highlight reel\nnpx tsx src/skill/cli.ts \"create a sports highlight from last weekend\"\n\n# Cyberpunk city night cuts\nnpx tsx src/skill/cli.ts \"cyberpunk city vibes, neon nights\"\n\n# Nature documentary style\nnpx tsx src/skill/cli.ts \"BBC nature doc style with my wildlife footage\"\n\n# With explicit composition + local music\nnpx tsx src/skill/cli.ts \"scenic alpine journey\" --composition SwitzerlandScenic --bgm ./music/alpine.mp3\n```\n\n---\n\n## Pipeline\n\n```\nUser prompt\n │\n ▼\n1. parseIntent — LLM extracts theme, selects composition, optionally overrides music style\n2. refineQueries — LLM rewrites per-slot search terms to match library content\n3. pickShots — ShotAI semantic search per slot; scored by similarity + duration + mood\n4. resolveMusic — yt-dlp YouTube search+download, or local --bgm file\n5. extractClip — ffmpeg trims each shot to an independent .mp4\n6. annotateClips — LLM assigns per-clip visual params (tone, kenBurns, dramatic, caption)\n7. File Server — HTTP server serves clips to the Remotion renderer\n8. Remotion render — Final MP4 composed and rendered\n```\n\n---\n\n## CLI Reference\n\n```bash\nnpx tsx src/skill/cli.ts \"<request>\" [options]\n\nOptions:\n --composition <id> Force a specific composition (skip LLM selection)\n --bgm <path> Local MP3 path (skip YouTube search)\n --output <dir> Output directory (default: ./output)\n --probe Scan library first; LLM plans slots from actual content\n```\n\n---\n\n## Compositions\n\n| ID | Style | Best For |\n|----|-------|----------|\n| `CyberpunkCity` | Cyberpunk night | Neon city, night scenes, sci-fi |\n| `TravelVlog` | Travel vlog | Multi-city travel with location cards |\n| `MoodDriven` | Mood-driven cuts | Emotional fast/slow montage |\n| `NatureWild` | BBC nature doc | Wildlife, landscapes, nature footage |\n| `SwitzerlandScenic` | Alpine scenic | Mountain travel with elegant captions |\n| `SportsHighlight` | ESPN sports | Goal/action highlights with captions |\n\n---\n\n## Modes\n\n**Standard mode** (default) — LLM picks the composition and generates search queries from registry templates.\n\n**Probe mode** (`--probe`) — Scans the library first (video names, shot samples, mood/scene tags), then LLM builds custom slots tailored to what actually exists. Use this when:\n- Library content is unknown or varied\n- User wants \"best of my library\"\n- Standard queries return low-quality shots\n\n---\n\n## Configuration\n\nEdit `.env` (copy from `.env.example`):\n\n```env\n# ── LLM Agent ────────────────────────────────────────────────────────────────\nAGENT_PROVIDER=claude # claude | openai | openai-compat | none\nANTHROPIC_API_KEY=sk-ant-... # required when AGENT_PROVIDER=claude\nOPENAI_API_KEY=sk-... # required when AGENT_PROVIDER=openai\nOPENAI_COMPAT_BASE_URL=https://... # required when AGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=claude-sonnet-4-6 # override default model\n\n# ── ShotAI ───────────────────────────────────────────────────────────────────\nSHOTAI_URL=http://127.0.0.1:23817\nSHOTAI_TOKEN=<your-token>\n\n# ── Music ────────────────────────────────────────────────────────────────────\nBGM_PATH=/path/to/music.mp3 # permanent local BGM default\n\n# ── Quality ──────────────────────────────────────────────────────────────────\nMIN_SCORE=0.5 # shot quality threshold 0–1 (recommended: 0.5)\n```\n\n### LLM Providers\n\n<details>\n<summary>Claude (Anthropic)</summary>\n\n```env\nAGENT_PROVIDER=claude\nANTHROPIC_API_KEY=sk-ant-...\nAGENT_MODEL=claude-sonnet-4-6\n```\n</details>\n\n<details>\n<summary>OpenAI</summary>\n\n```env\nAGENT_PROVIDER=openai\nOPENAI_API_KEY=sk-...\nAGENT_MODEL=gpt-4o\n```\n</details>\n\n<details>\n<summary>OpenRouter (recommended for multi-provider access)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://openrouter.ai/api/v1\nOPENAI_COMPAT_API_KEY=sk-or-v1-...\nAGENT_MODEL=deepseek/deepseek-chat-v3-0324\n```\n</details>\n\n<details>\n<summary>Ollama (local, no API key needed)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=http://localhost:11434/v1\nOPENAI_COMPAT_API_KEY=ollama\nAGENT_MODEL=llama3.1\n```\n</details>\n\n<details>\n<summary>DeepSeek (direct)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://api.deepseek.com/v1\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=deepseek-chat\n```\n</details>\n\n<details>\n<summary>No LLM (heuristic fallback)</summary>\n\n```env\nAGENT_PROVIDER=none\n```\nKeyword-based composition selection + registry default queries. No API key required.\n</details>\n\n---\n\n## Troubleshooting\n\n### Clip boundary flicker (1–2 frame flash at cuts)\n\nAn 80ms head/tail trim is applied automatically (`TRIM = 0.08`). If it persists, increase `TRIM` to `0.12` or `0.15` in `src/skill/orchestrator.ts`.\n\n### Red flash in CyberpunkCity\n\n`GlitchFlicker` triggers on very short clips. Set `MIN_SCORE=0.5` in `.env` to keep short clips out of the pipeline.\n\n### Low-quality or off-topic shots\n\n1. Raise `MIN_SCORE` (try `0.5` → `0.7`)\n2. Use `--probe` mode — LLM sees your actual library before picking queries\n3. Force `--composition <id>` to a composition whose slots match your content\n\n### Music download fails\n\n```bash\npip install -U yt-dlp # update yt-dlp\n# or use a local file:\nnpx tsx src/skill/cli.ts \"...\" --bgm /path/to/music.mp3\n```\n\n---\n\n## Performance\n\n| Step | Typical time (M-series Mac) |\n|------|-----------------------------|\n| Remotion render (60s video) | 30–90s |\n| ShotAI search per slot | 1–3s |\n| ffmpeg clip extraction | ~0.5s per clip |\n\n---\n\n## Adding a New Composition\n\nSee [references/composition-guide.md](references/composition-guide.md) for step-by-step instructions on adding a new Remotion visual style + registry entry.\n"}, "4660c844c4c23a822e531c5b63cc6c63b788b6f1": {"t": 1785763902.85176, "n": 7009, "h": "45640c4738ae1e85", "c": "# AI Video Remix\n\n> AI-driven video remix generator — semantic search + LLM planning + Remotion rendering.\n>\n> 中文文档: [README.zh.md](README.zh.md)\n\nGenerate styled video compositions from your local ShotAI video library using natural language.\n\n---\n\n## Prerequisites\n\n| Tool | Purpose | Install |\n|------|---------|---------|\n| [ShotAI](https://www.shotai.io) | Local video asset management + semantic shot search (MCP server) | Download from website |\n| ffmpeg | Clip extraction and keyframe analysis | `brew install ffmpeg` |\n| yt-dlp | Auto background music from YouTube | `brew install yt-dlp` |\n| Node.js 18+ | Runtime | `brew install node` |\n\n### ShotAI Setup\n\n1. Download and open ShotAI, add your video files/folders to a collection\n2. Wait for indexing (shot detection + embeddings — takes a few minutes)\n3. **Settings → MCP Server → Enable**\n4. Note your **MCP URL** (default: `http://127.0.0.1:23817`) and **MCP Token**\n\n---\n\n## Quick Start\n\n```bash\ngit clone https://github.com/abu-ShotAI/ai-video-remix.git\ncd ai-video-editor\nnpm install\ncp .env.example .env # fill in SHOTAI_URL, SHOTAI_TOKEN, and optionally AGENT_PROVIDER\n```\n\n```bash\n# Travel vlog from your library\nnpx tsx src/skill/cli.ts \"make a travel vlog from my library\"\n\n# Sports highlight reel\nnpx tsx src/skill/cli.ts \"create a sports highlight from last weekend\"\n\n# Cyberpunk city night cuts\nnpx tsx src/skill/cli.ts \"cyberpunk city vibes, neon nights\"\n\n# Nature documentary style\nnpx tsx src/skill/cli.ts \"BBC nature doc style with my wildlife footage\"\n\n# With explicit composition + local music\nnpx tsx src/skill/cli.ts \"scenic alpine journey\" --composition SwitzerlandScenic --bgm ./music/alpine.mp3\n```\n\n---\n\n## Pipeline\n\n```\nUser prompt\n │\n ▼\n1. parseIntent — LLM extracts theme, selects composition, optionally overrides music style\n2. refineQueries — LLM rewrites per-slot search terms to match library content\n3. pickShots — ShotAI semantic search per slot; scored by similarity + duration + mood\n4. resolveMusic — yt-dlp YouTube search+download, or local --bgm file\n5. extractClip — ffmpeg trims each shot to an independent .mp4\n6. annotateClips — LLM assigns per-clip visual params (tone, kenBurns, dramatic, caption)\n7. File Server — HTTP server serves clips to the Remotion renderer\n8. Remotion render — Final MP4 composed and rendered\n```\n\n---\n\n## CLI Reference\n\n```bash\nnpx tsx src/skill/cli.ts \"<request>\" [options]\n\nOptions:\n --composition <id> Force a specific composition (skip LLM selection)\n --bgm <path> Local MP3 path (skip YouTube search)\n --output <dir> Output directory (default: ./output)\n --probe Scan library first; LLM plans slots from actual content\n```\n\n---\n\n## Compositions\n\n| ID | Style | Best For |\n|----|-------|----------|\n| `CyberpunkCity` | Cyberpunk night | Neon city, night scenes, sci-fi |\n| `TravelVlog` | Travel vlog | Multi-city travel with location cards |\n| `MoodDriven` | Mood-driven cuts | Emotional fast/slow montage |\n| `NatureWild` | BBC nature doc | Wildlife, landscapes, nature footage |\n| `SwitzerlandScenic` | Alpine scenic | Mountain travel with elegant captions |\n| `SportsHighlight` | ESPN sports | Goal/action highlights with captions |\n\n---\n\n## Modes\n\n**Standard mode** (default) — LLM picks the composition and generates search queries from registry templates.\n\n**Probe mode** (`--probe`) — Scans the library first (video names, shot samples, mood/scene tags), then LLM builds custom slots tailored to what actually exists. Use this when:\n- Library content is unknown or varied\n- User wants \"best of my library\"\n- Standard queries return low-quality shots\n\n---\n\n## Configuration\n\nEdit `.env` (copy from `.env.example`):\n\n```env\n# ── LLM Agent ────────────────────────────────────────────────────────────────\nAGENT_PROVIDER=claude # claude | openai | openai-compat | none\nANTHROPIC_API_KEY=sk-ant-... # required when AGENT_PROVIDER=claude\nOPENAI_API_KEY=sk-... # required when AGENT_PROVIDER=openai\nOPENAI_COMPAT_BASE_URL=https://... # required when AGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=claude-sonnet-4-6 # override default model\n\n# ── ShotAI ───────────────────────────────────────────────────────────────────\nSHOTAI_URL=http://127.0.0.1:23817\nSHOTAI_TOKEN=<your-token>\n\n# ── Music ────────────────────────────────────────────────────────────────────\nBGM_PATH=/path/to/music.mp3 # permanent local BGM default\n\n# ── Quality ──────────────────────────────────────────────────────────────────\nMIN_SCORE=0.5 # shot quality threshold 0–1 (recommended: 0.5)\n```\n\n### LLM Providers\n\n<details>\n<summary>Claude (Anthropic)</summary>\n\n```env\nAGENT_PROVIDER=claude\nANTHROPIC_API_KEY=sk-ant-...\nAGENT_MODEL=claude-sonnet-4-6\n```\n</details>\n\n<details>\n<summary>OpenAI</summary>\n\n```env\nAGENT_PROVIDER=openai\nOPENAI_API_KEY=sk-...\nAGENT_MODEL=gpt-4o\n```\n</details>\n\n<details>\n<summary>OpenRouter (recommended for multi-provider access)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://openrouter.ai/api/v1\nOPENAI_COMPAT_API_KEY=sk-or-v1-...\nAGENT_MODEL=deepseek/deepseek-chat-v3-0324\n```\n</details>\n\n<details>\n<summary>Ollama (local, no API key needed)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=http://localhost:11434/v1\nOPENAI_COMPAT_API_KEY=ollama\nAGENT_MODEL=llama3.1\n```\n</details>\n\n<details>\n<summary>DeepSeek (direct)</summary>\n\n```env\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://api.deepseek.com/v1\nOPENAI_COMPAT_API_KEY=sk-...\nAGENT_MODEL=deepseek-chat\n```\n</details>\n\n<details>\n<summary>No LLM (heuristic fallback)</summary>\n\n```env\nAGENT_PROVIDER=none\n```\nKeyword-based composition selection + registry default queries. No API key required.\n</details>\n\n---\n\n## Troubleshooting\n\n### Clip boundary flicker (1–2 frame flash at cuts)\n\nAn 80ms head/tail trim is applied automatically (`TRIM = 0.08`). If it persists, increase `TRIM` to `0.12` or `0.15` in `src/skill/orchestrator.ts`.\n\n### Red flash in CyberpunkCity\n\n`GlitchFlicker` triggers on very short clips. Set `MIN_SCORE=0.5` in `.env` to keep short clips out of the pipeline.\n\n### Low-quality or off-topic shots\n\n1. Raise `MIN_SCORE` (try `0.5` → `0.7`)\n2. Use `--probe` mode — LLM sees your actual library before picking queries\n3. Force `--composition <id>` to a composition whose slots match your content\n\n### Music download fails\n\n```bash\npip install -U yt-dlp # update yt-dlp\n# or use a local file:\nnpx tsx src/skill/cli.ts \"...\" --bgm /path/to/music.mp3\n```\n\n---\n\n## Performance\n\n| Step | Typical time (M-series Mac) |\n|------|-----------------------------|\n| Remotion render (60s video) | 30–90s |\n| ShotAI search per slot | 1–3s |\n| ffmpeg clip extraction | ~0.5s per clip |\n\n---\n\n## Adding a New Composition\n\nSee [references/composition-guide.md](references/composition-guide.md) for step-by-step instructions on adding a new Remotion visual style + registry entry.\n"}, "04b7b88c0f7b09f650d6169af3cb2f3c1b4ac2d6": {"t": 1785763904.2784088, "n": 3841, "h": "a6f55e50af6df72b", "c": "# AI Video Remix\n\n> AI-powered video remix generator — natural language → styled video composition\n\nUses **ShotAI** (semantic shot search) + **LLM planning** + **Remotion** (React-based rendering) to turn your local video library into polished, themed video cuts.\n\n---\n\n## Features\n\n- **Natural language input**: \"帮我做一个赛博朋克香港夜景混剪\" → full video\n- **6 built-in compositions**: CyberpunkCity, TravelVlog, MoodDriven, NatureWild, SwitzerlandScenic, SportsHighlight\n- **LLM-driven shot planning**: Claude / GPT-4o / DeepSeek / Ollama all supported\n- **Heuristic fallback**: works with `AGENT_PROVIDER=none`, no API key needed\n- **Auto BGM**: yt-dlp YouTube search, or bring your own MP3\n- **ShotAI integration**: shot-level semantic search over your local video library\n- **Probe mode**: scans library first, generates queries tailored to actual content\n\n## Prerequisites\n\n| Dependency | Required | Notes |\n|---|---|---|\n| [ShotAI](https://www.shotai.io) | ✅ Yes | Local video asset manager, provides MCP server |\n| ffmpeg | ✅ Yes | Clip extraction |\n| Node.js 18+ | ✅ Yes | Runtime |\n| yt-dlp | For auto music | YouTube BGM download |\n| LLM API key | Optional | Falls back to heuristic mode if absent |\n\n## Quick Start\n\n```bash\ngit clone https://github.com/abu-ShotAI/ai-video-remix.git\ncd ai-video-remix\nnpm install\ncp .env.example .env # edit with your ShotAI token + optional LLM key\n```\n\n```bash\n# Heuristic mode (no LLM key needed)\nAGENT_PROVIDER=none npx tsx src/skill/cli.ts \"travel vlog\" --composition TravelVlog\n\n# With LLM (Claude)\nnpx tsx src/skill/cli.ts \"帮我做一个赛博朋克香港混剪\"\n\n# Bring your own music\nnpx tsx src/skill/cli.ts \"nature montage\" --bgm ~/music/ambient.mp3\n\n# Probe mode: scan library first\nnpx tsx src/skill/cli.ts \"best of my library\" --probe\n```\n\nOutput video: `./output/<composition>-<timestamp>.mp4`\n\n## Compositions\n\n| ID | Style | Best For |\n|----|-------|----------|\n| `CyberpunkCity` | 赛博朋克夜景 | Neon city, night scenes, sci-fi |\n| `TravelVlog` | 旅行 Vlog | Multi-city travel with location cards |\n| `MoodDriven` | 情绪驱动混剪 | Fast/slow emotion cuts |\n| `NatureWild` | 自然野生动物 | BBC nature documentary style |\n| `SwitzerlandScenic` | 瑞士风光 | Alpine scenic travel |\n| `SportsHighlight` | 体育集锦 | ESPN-style sports cuts |\n\n## Configuration\n\nCopy `.env.example` to `.env` and fill in:\n\n```env\n# Required\nSHOTAI_URL=http://127.0.0.1:23817\nSHOTAI_TOKEN=<your-token>\n\n# Optional: LLM provider (default: none / heuristic)\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://openrouter.ai/api/v1\nOPENAI_COMPAT_API_KEY=sk-or-v1-...\nAGENT_MODEL=deepseek/deepseek-chat-v3-0324\n\n# Quality\nMIN_SCORE=0.5\n```\n\nSee [docs/config.md](docs/config.md) for all options and LLM provider examples (Claude, OpenAI, OpenRouter, Ollama, DeepSeek).\n\n## ShotAI Setup\n\n1. Download at [https://www.shotai.io](https://www.shotai.io)\n2. Add your video folders and wait for indexing\n3. Enable MCP server: **Settings → MCP Server → Enable**\n4. Copy the MCP URL and token to `.env`\n\n## Project Structure\n\n```\nsrc/\n skill/\n cli.ts # Entry point\n orchestrator.ts # Pipeline: shots → clips → render\n agent.ts # LLM backends (Claude, OpenAI, OpenRouter, heuristic)\n registry.ts # Composition metadata + shot slots\n mcp.ts # ShotAI MCP client\n music.ts # BGM resolution (yt-dlp / local)\n remotion/\n compositions/ # Remotion React components per style\n Root.tsx # Composition registry\n```\n\n## Adding a New Composition\n\nSee [docs/composition-guide.md](docs/composition-guide.md).\n\n## Known Issues & Tuning\n\nSee [docs/tuning.md](docs/tuning.md) for solutions to boundary flicker, flash artifacts, and low-quality shots.\n\n## License\n\nMIT — see [LICENSE](LICENSE)\n\n---\n\nBuilt with [ShotAI](https://www.shotai.io) · [Remotion](https://www.remotion.dev) · [Seeknetic](https://seeknetic.com)\n"}, "61a3b14c6e0e4286fad5a005c1bd097caa6a9903": {"t": 1785763905.723627, "n": 3842, "h": "a8e3226d02abc33b", "c": "# AI Video Remix\n\n> AI-powered video remix generator — natural language → styled video composition\n\nUses **ShotAI** (semantic shot search) + **LLM planning** + **Remotion** (React-based rendering) to turn your local video library into polished, themed video cuts.\n\n---\n\n## Features\n\n- **Natural language input**: \"帮我做一个赛博朋克香港夜景混剪\" → full video\n- **6 built-in compositions**: CyberpunkCity, TravelVlog, MoodDriven, NatureWild, SwitzerlandScenic, SportsHighlight\n- **LLM-driven shot planning**: Claude / GPT-4o / DeepSeek / Ollama all supported\n- **Heuristic fallback**: works with `AGENT_PROVIDER=none`, no API key needed\n- **Auto BGM**: yt-dlp YouTube search, or bring your own MP3\n- **ShotAI integration**: shot-level semantic search over your local video library\n- **Probe mode**: scans library first, generates queries tailored to actual content\n\n## Prerequisites\n\n| Dependency | Required | Notes |\n|---|---|---|\n| [ShotAI](https://www.shotai.io) | ✅ Yes | Local video asset manager, provides MCP server |\n| ffmpeg | ✅ Yes | Clip extraction |\n| Node.js 18+ | ✅ Yes | Runtime |\n| yt-dlp | For auto music | YouTube BGM download |\n| LLM API key | Optional | Falls back to heuristic mode if absent |\n\n## Quick Start\n\n```bash\ngit clone https://github.com/seeknetic/ai-video-editor.git\ncd ai-video-editor\nnpm install\ncp .env.example .env # edit with your ShotAI token + optional LLM key\n```\n\n```bash\n# Heuristic mode (no LLM key needed)\nAGENT_PROVIDER=none npx tsx src/skill/cli.ts \"travel vlog\" --composition TravelVlog\n\n# With LLM (Claude)\nnpx tsx src/skill/cli.ts \"帮我做一个赛博朋克香港混剪\"\n\n# Bring your own music\nnpx tsx src/skill/cli.ts \"nature montage\" --bgm ~/music/ambient.mp3\n\n# Probe mode: scan library first\nnpx tsx src/skill/cli.ts \"best of my library\" --probe\n```\n\nOutput video: `./output/<composition>-<timestamp>.mp4`\n\n## Compositions\n\n| ID | Style | Best For |\n|----|-------|----------|\n| `CyberpunkCity` | 赛博朋克夜景 | Neon city, night scenes, sci-fi |\n| `TravelVlog` | 旅行 Vlog | Multi-city travel with location cards |\n| `MoodDriven` | 情绪驱动混剪 | Fast/slow emotion cuts |\n| `NatureWild` | 自然野生动物 | BBC nature documentary style |\n| `SwitzerlandScenic` | 瑞士风光 | Alpine scenic travel |\n| `SportsHighlight` | 体育集锦 | ESPN-style sports cuts |\n\n## Configuration\n\nCopy `.env.example` to `.env` and fill in:\n\n```env\n# Required\nSHOTAI_URL=http://127.0.0.1:23817\nSHOTAI_TOKEN=<your-token>\n\n# Optional: LLM provider (default: none / heuristic)\nAGENT_PROVIDER=openai-compat\nOPENAI_COMPAT_BASE_URL=https://openrouter.ai/api/v1\nOPENAI_COMPAT_API_KEY=sk-or-v1-...\nAGENT_MODEL=deepseek/deepseek-chat-v3-0324\n\n# Quality\nMIN_SCORE=0.5\n```\n\nSee [docs/config.md](docs/config.md) for all options and LLM provider examples (Claude, OpenAI, OpenRouter, Ollama, DeepSeek).\n\n## ShotAI Setup\n\n1. Download at [https://www.shotai.io](https://www.shotai.io)\n2. Add your video folders and wait for indexing\n3. Enable MCP server: **Settings → MCP Server → Enable**\n4. Copy the MCP URL and token to `.env`\n\n## Project Structure\n\n```\nsrc/\n skill/\n cli.ts # Entry point\n orchestrator.ts # Pipeline: shots → clips → render\n agent.ts # LLM backends (Claude, OpenAI, OpenRouter, heuristic)\n registry.ts # Composition metadata + shot slots\n mcp.ts # ShotAI MCP client\n music.ts # BGM resolution (yt-dlp / local)\n remotion/\n compositions/ # Remotion React components per style\n Root.tsx # Composition registry\n```\n\n## Adding a New Composition\n\nSee [docs/composition-guide.md](docs/composition-guide.md).\n\n## Known Issues & Tuning\n\nSee [docs/tuning.md](docs/tuning.md) for solutions to boundary flicker, flash artifacts, and low-quality shots.\n\n## License\n\nMIT — see [LICENSE](LICENSE)\n\n---\n\nBuilt with [ShotAI](https://www.shotai.io) · [Remotion](https://www.remotion.dev) · [Seeknetic](https://seeknetic.com)\n"}}, "src/skill/agent.ts": {"7ca00a40057e92da07fc40a8e3eae4c7038a8387": {"t": 1785746722.442526, "n": 19615, "h": "c18113e3db4f29a8", "c": "/**\n * Agent Layer — LLM-driven intent parsing and shot selection\n *\n * Architecture:\n * - LLMBackend: thin interface for any LLM provider (one method: callLLM)\n * - BaseAgent: all prompt logic lives here, calls this.backend.callLLM()\n * - Provider backends: AnthropicBackend, OpenAIBackend, OpenAICompatBackend\n * - HeuristicAgent: keyword fallback, no API key required\n *\n * Provider selection (set AGENT_PROVIDER in .env):\n * 'claude' — Anthropic API (ANTHROPIC_API_KEY)\n * 'openai' — OpenAI API (OPENAI_API_KEY)\n * 'openai-compat' — Any OpenAI-compatible endpoint (OPENAI_COMPAT_BASE_URL + OPENAI_COMPAT_API_KEY)\n * Works with: Ollama, vLLM, Together, Groq, DeepSeek, Azure OpenAI, etc.\n * 'none' — Heuristic fallback (no API key required)\n */\nimport { REGISTRY, CompositionMeta } from './registry';\nimport { analyzeKeyframe } from './brightness';\n\nexport interface ParsedIntent {\n /** Main theme / title for the video */\n theme: string;\n /** Which composition to use */\n composition: CompositionMeta;\n /** Per-slot search query overrides (optional — falls back to registry defaults) */\n queryOverrides?: string[];\n /** Music style description override (optional) */\n musicStyleOverride?: string;\n}\n\n/**\n * Per-clip annotation generated by the LLM based on the actual shot summary.\n * These override or supplement the static `extra` fields from the registry.\n *\n * Fields are composition-aware: only relevant fields are used per composition.\n * - tone: 'warm' | 'cool' — color grade direction (SwitzerlandScenic)\n * - dramatic: boolean — slow-motion / emphasis clip (SportsHighlight)\n * - transition: 'fade' | 'slide' — per-clip transition override (future use)\n * - kenBurns: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right' (NatureWild)\n * - caption: string — override narrative caption if LLM improves it\n * - textBg: Record of region→'light'|'dark' — keyframe brightness analysis result.\n * Keys match TEXT_REGIONS in brightness.ts (e.g. \"NatureWild.caption\").\n * 'light' means the video background is bright → overlay needed.\n * Set by HeuristicAgent via keyframe pixel analysis; LLM agents can\n * also populate this from vision analysis.\n */\nexport interface ClipAnnotation {\n tone?: 'warm' | 'cool';\n dramatic?: boolean;\n transition?: 'fade' | 'slide';\n kenBurns?: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right';\n caption?: string;\n /** Per-region brightness result from keyframe analysis */\n textBg?: Record<string, 'light' | 'dark'>;\n}\n\n// ─── LLM Backend interface ────────────────────────────────────────────────────\n\n/**\n * Minimal interface any LLM provider must implement.\n * All prompt logic lives in BaseAgent; backends only handle HTTP transport.\n */\nexport interface LLMBackend {\n callLLM(system: string, user: string): Promise<string>;\n}\n\n// ─── Anthropic backend ────────────────────────────────────────────────────────\n\nexport class AnthropicBackend implements LLMBackend {\n constructor(\n private apiKey: string,\n private model = 'claude-sonnet-4-6',\n ) {}\n\n async callLLM(system: string, user: string): Promise<string> {\n const res = await fetch('https://api.anthropic.com/v1/messages', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'x-api-key': this.apiKey,\n 'anthropic-version': '2023-06-01',\n },\n body: JSON.stringify({\n model: this.model,\n max_tokens: 1024,\n system,\n messages: [{ role: 'user', content: user }],\n }),\n });\n if (!res.ok) throw new Error(`Anthropic API error ${res.status}: ${await res.text()}`);\n const data = await res.json() as any;\n return data.content[0].text as string;\n }\n}\n\n// ─── OpenAI-compatible backend (OpenAI, Ollama, vLLM, Groq, DeepSeek, …) ─────\n\nexport class OpenAICompatBackend implements LLMBackend {\n private baseUrl: string;\n\n constructor(\n private apiKey: string,\n private model: string,\n /** Base URL — defaults to OpenAI. Override for Ollama/vLLM/etc. */\n baseUrl?: string,\n ) {\n const base = (baseUrl ?? 'https://api.openai.com').replace(/\\/$/, '');\n // If caller already includes /v1 (e.g. OpenRouter), don't double-append it\n this.baseUrl = base.endsWith('/v1') ? base : `${base}/v1`;\n }\n\n async callLLM(system: string, user: string): Promise<string> {\n let lastError: Error | undefined;\n for (let attempt = 1; attempt <= 3; attempt++) {\n const res = await fetch(`${this.baseUrl}/chat/completions`, {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'Authorization': `Bearer ${this.apiKey}`,\n },\n body: JSON.stringify({\n model: this.model,\n messages: [\n { role: 'system', content: system },\n { role: 'user', content: user },\n ],\n max_tokens: 1024,\n }),\n });\n if (res.ok) {\n const data = await res.json() as any;\n return data.choices[0].message.content as string;\n }\n const body = await res.text();\n lastError = new Error(`LLM API error ${res.status}: ${body}`);\n if (res.status < 500) break; // 4xx errors won't be fixed by retry\n if (attempt < 3) console.warn(` ⚠ LLM API ${res.status}, retrying (${attempt}/3)...`);\n }\n throw lastError!;\n }\n}\n\n// ─── AgentProvider interface ──────────────────────────────────────────────────\n\n/** Compact summary of a user's media library, passed to LLM for dynamic planning. */\nexport interface LibrarySummary {\n videos: Array<{\n name: string;\n duration: number;\n shotCount: number;\n /** Up to 20 shot summaries sampled from the video */\n sampleSummaries: string[];\n /** Unique mood tags found across sampled shots */\n moods: string[];\n /** Unique scene tags found across sampled shots */\n scenes: string[];\n }>;\n}\n\n/** A dynamically generated slot list produced by LLM after seeing the library. */\nexport interface DynamicPlan {\n /** Theme/title for the video */\n theme: string;\n /** Which registered composition to use as the Remotion template */\n compositionId: string;\n /** Music style description */\n musicStyle: string;\n /** Slot list tailored to what actually exists in the library */\n slots: Array<{\n query: string;\n mood?: string;\n /** Any extra metadata the LLM wants to attach (caption, location, etc.) */\n extra?: Record<string, unknown>;\n }>;\n}\n\nexport interface AgentProvider {\n parseIntent(userRequest: string): Promise<ParsedIntent>;\n refineQueries(slots: Array<{ defaultQuery: string; mood?: string }>, theme: string): Promise<string[]>;\n /**\n * Given the actual shot summaries returned by ShotAI, produce per-clip\n * visual annotations that drive Remotion special-effects decisions.\n *\n * @param clips The resolved clip list (summary, optional keyframePath, slot extra, optional visual analysis)\n * @param compositionId Which composition is being rendered (determines which fields matter)\n */\n annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown>; visualAnalysis?: string }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]>;\n /**\n * After seeing what's actually in the user's library, generate a tailored plan:\n * which composition to use and what slots to search for.\n */\n planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan>;\n}\n\n// ─── BaseAgent — all prompt logic lives here ──────────────────────────────────\n\nclass BaseAgent implements AgentProvider {\n constructor(private backend: LLMBackend, private lang: 'zh' | 'en' = 'zh') {}\n\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const langInstr = this.lang === 'en'\n ? 'Output the theme in English.'\n : '主题用中文输出。';\n\n const system = `你是一个视频混剪规划助手。根据用户的混剪需求，选择最合适的合成风格并提取主题。${langInstr}\n只输出原始 JSON（不要 markdown 代码块，不要任何说明），格式如下：\n{\n \"theme\": \"视频主题名称（简短，用做片头片尾标题）\",\n \"compositionId\": \"CyberpunkCity | TravelVlog | MoodDriven\",\n \"musicStyleOverride\": \"（可选）英文音乐风格描述，留空则用默认\"\n}`;\n\n const user = `可选合成风格：\n${compositionList}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`Agent parseIntent: invalid JSON response: ${raw}`);\n }\n\n const meta = REGISTRY.find(m => m.id === parsed.compositionId) ?? REGISTRY[0];\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n composition: meta,\n musicStyleOverride: parsed.musicStyleOverride || undefined,\n };\n }\n\n async refineQueries(\n slots: Array<{ defaultQuery: string; mood?: string }>,\n theme: string,\n ): Promise<string[]> {\n const slotList = slots.map((s, i) =>\n ` ${i + 1}. default: \"${s.defaultQuery}\" mood: ${s.mood ?? 'any'}`\n ).join('\\n');\n\n const system = `You are a video shot search query writer. Rewrite each slot into an effective semantic search query.\n\nRules:\n- Write a descriptive English sentence or rich phrase (10–20 words) describing exactly what the camera captures\n- Include: subject, action, environment, lighting/time of day — the more specific the visual, the better\n- Use real-world vocabulary (e.g. \"Eiffel Tower\", \"snow-capped Alps\", \"soccer player sprinting\") — never genre labels like \"cyberpunk\", \"cinematic\", or \"bokeh\"\n- If the theme names a real place or event, anchor your queries to that location/event\n- Avoid negations (\"not indoors\") — describe the positive target instead\n- Output ONLY a raw JSON array — no explanation, no markdown fences, no preamble`;\n\n const user = `Theme: ${theme}\nSlots:\n${slotList}\n\n[\"query1\",\"query2\",...]`;\n\n const raw = await this.backend.callLLM(system, user);\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== slots.length) {\n throw new Error(`Expected array of length ${slots.length}`);\n }\n return arr as string[];\n } catch (e) {\n console.warn(` ⚠ Agent refineQueries failed (${(e as Error).message}), using defaults`);\n return slots.map(s => s.defaultQuery);\n }\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown>; visualAnalysis?: string }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n const fieldHints: Record<string, string> = {\n SwitzerlandScenic: '\"tone\": \"warm\"|\"cool\" — warm for sunshine/meadow/sunset, cool for snow/glacier/water',\n SportsHighlight: '\"dramatic\": true|false — true for goals/key moments/emotional close-ups, false for regular play',\n NatureWild: '\"kenBurns\": \"zoom-in\"|\"zoom-out\"|\"pan-left\"|\"pan-right\" — choose based on subject (animal approaching=zoom-in, wide landscape=zoom-out, moving subject=pan)',\n };\n const relevantFields = fieldHints[compositionId] ?? '\"tone\": \"warm\"|\"cool\"';\n\n const clipList = clips.map((c, i) => {\n const visualPart = c.visualAnalysis ? `\\n visual_analysis: \"${c.visualAnalysis.slice(0, 200)}\"` : '';\n return ` ${i + 1}. summary: \"${c.summary}\"${visualPart}`;\n }).join('\\n');\n\n const system = `你是一个视频剪辑视效助手。根据每个镜头的语义描述和视觉分析，为每个镜头生成精准的视觉特效注解。\n只输出原始 JSON 数组（不要 markdown 代码块，不要任何说明），每个元素只包含相关字段（不相关字段省略）。\n当前合成需要的字段：${relevantFields}\n\ncaption 字段写作规则（重要）：\n- 必须基于 visual_analysis（如果有），描述这个镜头实际发生的内容\n- 文字必须服务整个视频的叙事弧，而非单纯描述这一个镜头的内容\n- ${this.lang === 'en' ? '**Language: ENGLISH only. Maximum 30 characters. Do NOT use Chinese.**' : '**语言：必须用中文，禁止英文。最多10个字，超过会被硬截断显示省略号。**'}\n- 风格：简洁有力，像纪录片旁白或诗句，传递情绪而非描述细节\n- 反例（太长/太平）：\"一只黑豹在非洲热带雨林的茂密植被中悄悄地穿行觅食\" → 正例：\"暗夜行者，无声的猎手\"\n输出格式示例：[{\"tone\":\"cool\"},{\"tone\":\"warm\",\"dramatic\":true},...]\n元素数量必须与输入完全相同。`;\n\n const user = `合成ID：${compositionId}\n镜头列表（共 ${clips.length} 个）：\n${clipList}\n\n请为每个镜头生成视效注解，输出 JSON 数组。`;\n\n const raw = await this.backend.callLLM(system, user);\n let annotations: ClipAnnotation[];\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== clips.length) {\n throw new Error(`Expected array of length ${clips.length}`);\n }\n annotations = arr as ClipAnnotation[];\n } catch (e) {\n console.warn(` ⚠ annotateClips failed (${(e as Error).message}), skipping annotations`);\n annotations = clips.map(() => ({}));\n }\n\n // Augment with keyframe brightness analysis (fixed algorithm, no LLM)\n for (let i = 0; i < clips.length; i++) {\n const kf = clips[i].keyframePath;\n if (kf) annotations[i].textBg = analyzeKeyframe(kf, compositionId);\n }\n return annotations;\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const libraryDesc = library.videos.map(v =>\n ` 【${v.name}】时长${Math.round(v.duration / 60)}分钟，${v.shotCount}个镜头\\n` +\n ` 情绪: ${v.moods.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 场景: ${v.scenes.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 镜头样本: ${v.sampleSummaries.slice(0, 8).map(s => `\"${s.slice(0, 60)}\"`).join(' / ')}`\n ).join('\\n\\n');\n\n const system = `你是一个视频混剪规划师。用户有一个视频素材库，你需要：\n1. 仔细阅读库里每个视频的镜头样本描述（sampleSummaries）\n2. 根据用户需求，选择最合适的合成模板\n3. 生成专门针对这个库内容的搜索槽位 — 每条 query 必须用英文写成10-20词的具体视觉描述句\n\nQuery 写作规则（非常重要）：\n- 使用真实的视觉描述，例如 \"aerial view of Hong Kong skyscrapers at night with illuminated windows\"\n- 直接借用库里样本描述中出现的词汇和场景，命中率更高\n- 禁止使用风格标签（cyberpunk, cinematic, bokeh）或消极描述（not indoors）\n- 如遇地名或运动项目，直接写进查询（\"Paris Eiffel Tower aerial\", \"soccer player kicking ball on green field\"）\n\n${this.lang === 'en' ? 'Output the theme field in English.' : '主题 theme 字段用中文输出。'}\n只输出原始 JSON（不要 markdown 代码块，不要任何说明）：\n{\n \"theme\": \"视频主题（简短标题）\",\n \"compositionId\": \"选择的合成ID\",\n \"musicStyle\": \"英文音乐风格描述\",\n \"slots\": [\n { \"query\": \"10-20 word English visual description\", \"mood\": \"calm|fast|slow|urban\", \"extra\": {} },\n ...\n ]\n}\nslots数量建议6-10个，与选择的合成模板匹配。`;\n\n const user = `可用合成模板：\n${compositionList}\n\n用户素材库内容：\n${libraryDesc}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`planFromLibrary: invalid JSON response: ${raw}`);\n }\n\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n compositionId: parsed.compositionId ?? REGISTRY[0].id,\n musicStyle: parsed.musicStyle ?? 'cinematic ambient',\n slots: Array.isArray(parsed.slots) ? parsed.slots : [],\n };\n }\n}\n\n// ─── Heuristic fallback (no API key) ──────────────────────────────────────────\n\nclass HeuristicAgent implements AgentProvider {\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const req = userRequest.toLowerCase();\n let meta: CompositionMeta;\n if (req.includes('赛博') || req.includes('cyberpunk') || req.includes('霓虹') || req.includes('夜景')) {\n meta = REGISTRY.find(m => m.id === 'CyberpunkCity')!;\n } else if (req.includes('旅行') || req.includes('vlog') || req.includes('城市名') || req.includes('地标')) {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n } else {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n }\n const theme = userRequest.split(/[,，。\\n]/)[0].slice(0, 20).trim();\n console.log(` ℹ HeuristicAgent: no API key — selected \"${meta.label}\" for \"${theme}\"`);\n return { theme, composition: meta };\n }\n\n async refineQueries(slots: Array<{ defaultQuery: string }>, _theme: string): Promise<string[]> {\n return slots.map(s => s.defaultQuery);\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; visualAnalysis?: string }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n return clips.map(c => {\n const ann: ClipAnnotation = {};\n if (c.keyframePath) ann.textBg = analyzeKeyframe(c.keyframePath, compositionId);\n return ann;\n });\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n // Heuristic: reuse parseIntent to pick a composition, then use registry defaults as slots\n const intent = await this.parseIntent(userRequest);\n return {\n theme: intent.theme,\n compositionId: intent.composition.id,\n musicStyle: intent.musicStyleOverride ?? intent.composition.musicStyle,\n slots: intent.composition.shotSlots.map(s => ({\n query: s.query,\n mood: s.mood,\n extra: s.extra,\n })),\n };\n }\n}\n\n// ─── Factory ──────────────────────────────────────────────────────────────────\n\nexport type AgentProviderName = 'claude' | 'openai' | 'openai-compat' | 'none';\n\nexport interface CreateAgentOptions {\n provider: AgentProviderName;\n apiKey?: string;\n model?: string;\n /** Only for 'openai-compat': base URL of the compatible endpoint */\n baseUrl?: string;\n /** Output language for theme titles and LLM-generated captions ('zh' | 'en', default 'zh') */\n lang?: 'zh' | 'en';\n}\n\n/**\n * createAgent — instantiate the right backend and wrap it in BaseAgent.\n *\n * Examples:\n * createAgent({ provider: 'claude', apiKey: '...' })\n * createAgent({ provider: 'openai', apiKey: '...', model: 'gpt-4o' })\n * createAgent({ provider: 'openai-compat', apiKey: '...', model: 'deepseek-chat', baseUrl: 'https://api.deepseek.com' })\n * createAgent({ provider: 'openai-compat', apiKey: 'ollama', model: 'llama3', baseUrl: 'http://localhost:11434' })\n * createAgent({ provider: 'none' }) // keyword heuristics only\n */\nexport function createAgent(opts: CreateAgentOptions): AgentProvider;\n/** @deprecated Pass an options object instead */\nexport function createAgent(provider: AgentProviderName, apiKey?: string, model?: string): AgentProvider;\nexport function createAgent(\n optsOrProvider: CreateAgentOptions | AgentProviderName,\n apiKey?: string,\n model?: string,\n): AgentProvider {\n const opts: CreateAgentOptions =\n typeof optsOrProvider === 'string'\n ? { provider: optsOrProvider as AgentProviderName, apiKey, model }\n : optsOrProvider;\n\n if (opts.provider === 'claude') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=claude but ANTHROPIC_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new AnthropicBackend(opts.apiKey, opts.model ?? 'claude-sonnet-4-6'), opts.lang);\n }\n\n if (opts.provider === 'openai') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=openai but OPENAI_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'gpt-4o'), opts.lang);\n }\n\n if (opts.provider === 'openai-compat') {\n if (!opts.apiKey || !opts.baseUrl) {\n console.warn(' ⚠ openai-compat requires apiKey + baseUrl — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'default', opts.baseUrl), opts.lang);\n }\n\n return new HeuristicAgent();\n}\n"}, "cd2c9e3dc6bace90262eefce2d5b6800212a64aa": {"t": 1785763892.1371598, "n": 19615, "h": "c18113e3db4f29a8", "c": "/**\n * Agent Layer — LLM-driven intent parsing and shot selection\n *\n * Architecture:\n * - LLMBackend: thin interface for any LLM provider (one method: callLLM)\n * - BaseAgent: all prompt logic lives here, calls this.backend.callLLM()\n * - Provider backends: AnthropicBackend, OpenAIBackend, OpenAICompatBackend\n * - HeuristicAgent: keyword fallback, no API key required\n *\n * Provider selection (set AGENT_PROVIDER in .env):\n * 'claude' — Anthropic API (ANTHROPIC_API_KEY)\n * 'openai' — OpenAI API (OPENAI_API_KEY)\n * 'openai-compat' — Any OpenAI-compatible endpoint (OPENAI_COMPAT_BASE_URL + OPENAI_COMPAT_API_KEY)\n * Works with: Ollama, vLLM, Together, Groq, DeepSeek, Azure OpenAI, etc.\n * 'none' — Heuristic fallback (no API key required)\n */\nimport { REGISTRY, CompositionMeta } from './registry';\nimport { analyzeKeyframe } from './brightness';\n\nexport interface ParsedIntent {\n /** Main theme / title for the video */\n theme: string;\n /** Which composition to use */\n composition: CompositionMeta;\n /** Per-slot search query overrides (optional — falls back to registry defaults) */\n queryOverrides?: string[];\n /** Music style description override (optional) */\n musicStyleOverride?: string;\n}\n\n/**\n * Per-clip annotation generated by the LLM based on the actual shot summary.\n * These override or supplement the static `extra` fields from the registry.\n *\n * Fields are composition-aware: only relevant fields are used per composition.\n * - tone: 'warm' | 'cool' — color grade direction (SwitzerlandScenic)\n * - dramatic: boolean — slow-motion / emphasis clip (SportsHighlight)\n * - transition: 'fade' | 'slide' — per-clip transition override (future use)\n * - kenBurns: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right' (NatureWild)\n * - caption: string — override narrative caption if LLM improves it\n * - textBg: Record of region→'light'|'dark' — keyframe brightness analysis result.\n * Keys match TEXT_REGIONS in brightness.ts (e.g. \"NatureWild.caption\").\n * 'light' means the video background is bright → overlay needed.\n * Set by HeuristicAgent via keyframe pixel analysis; LLM agents can\n * also populate this from vision analysis.\n */\nexport interface ClipAnnotation {\n tone?: 'warm' | 'cool';\n dramatic?: boolean;\n transition?: 'fade' | 'slide';\n kenBurns?: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right';\n caption?: string;\n /** Per-region brightness result from keyframe analysis */\n textBg?: Record<string, 'light' | 'dark'>;\n}\n\n// ─── LLM Backend interface ────────────────────────────────────────────────────\n\n/**\n * Minimal interface any LLM provider must implement.\n * All prompt logic lives in BaseAgent; backends only handle HTTP transport.\n */\nexport interface LLMBackend {\n callLLM(system: string, user: string): Promise<string>;\n}\n\n// ─── Anthropic backend ────────────────────────────────────────────────────────\n\nexport class AnthropicBackend implements LLMBackend {\n constructor(\n private apiKey: string,\n private model = 'claude-sonnet-4-6',\n ) {}\n\n async callLLM(system: string, user: string): Promise<string> {\n const res = await fetch('https://api.anthropic.com/v1/messages', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'x-api-key': this.apiKey,\n 'anthropic-version': '2023-06-01',\n },\n body: JSON.stringify({\n model: this.model,\n max_tokens: 1024,\n system,\n messages: [{ role: 'user', content: user }],\n }),\n });\n if (!res.ok) throw new Error(`Anthropic API error ${res.status}: ${await res.text()}`);\n const data = await res.json() as any;\n return data.content[0].text as string;\n }\n}\n\n// ─── OpenAI-compatible backend (OpenAI, Ollama, vLLM, Groq, DeepSeek, …) ─────\n\nexport class OpenAICompatBackend implements LLMBackend {\n private baseUrl: string;\n\n constructor(\n private apiKey: string,\n private model: string,\n /** Base URL — defaults to OpenAI. Override for Ollama/vLLM/etc. */\n baseUrl?: string,\n ) {\n const base = (baseUrl ?? 'https://api.openai.com').replace(/\\/$/, '');\n // If caller already includes /v1 (e.g. OpenRouter), don't double-append it\n this.baseUrl = base.endsWith('/v1') ? base : `${base}/v1`;\n }\n\n async callLLM(system: string, user: string): Promise<string> {\n let lastError: Error | undefined;\n for (let attempt = 1; attempt <= 3; attempt++) {\n const res = await fetch(`${this.baseUrl}/chat/completions`, {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'Authorization': `Bearer ${this.apiKey}`,\n },\n body: JSON.stringify({\n model: this.model,\n messages: [\n { role: 'system', content: system },\n { role: 'user', content: user },\n ],\n max_tokens: 1024,\n }),\n });\n if (res.ok) {\n const data = await res.json() as any;\n return data.choices[0].message.content as string;\n }\n const body = await res.text();\n lastError = new Error(`LLM API error ${res.status}: ${body}`);\n if (res.status < 500) break; // 4xx errors won't be fixed by retry\n if (attempt < 3) console.warn(` ⚠ LLM API ${res.status}, retrying (${attempt}/3)...`);\n }\n throw lastError!;\n }\n}\n\n// ─── AgentProvider interface ──────────────────────────────────────────────────\n\n/** Compact summary of a user's media library, passed to LLM for dynamic planning. */\nexport interface LibrarySummary {\n videos: Array<{\n name: string;\n duration: number;\n shotCount: number;\n /** Up to 20 shot summaries sampled from the video */\n sampleSummaries: string[];\n /** Unique mood tags found across sampled shots */\n moods: string[];\n /** Unique scene tags found across sampled shots */\n scenes: string[];\n }>;\n}\n\n/** A dynamically generated slot list produced by LLM after seeing the library. */\nexport interface DynamicPlan {\n /** Theme/title for the video */\n theme: string;\n /** Which registered composition to use as the Remotion template */\n compositionId: string;\n /** Music style description */\n musicStyle: string;\n /** Slot list tailored to what actually exists in the library */\n slots: Array<{\n query: string;\n mood?: string;\n /** Any extra metadata the LLM wants to attach (caption, location, etc.) */\n extra?: Record<string, unknown>;\n }>;\n}\n\nexport interface AgentProvider {\n parseIntent(userRequest: string): Promise<ParsedIntent>;\n refineQueries(slots: Array<{ defaultQuery: string; mood?: string }>, theme: string): Promise<string[]>;\n /**\n * Given the actual shot summaries returned by ShotAI, produce per-clip\n * visual annotations that drive Remotion special-effects decisions.\n *\n * @param clips The resolved clip list (summary, optional keyframePath, slot extra, optional visual analysis)\n * @param compositionId Which composition is being rendered (determines which fields matter)\n */\n annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown>; visualAnalysis?: string }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]>;\n /**\n * After seeing what's actually in the user's library, generate a tailored plan:\n * which composition to use and what slots to search for.\n */\n planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan>;\n}\n\n// ─── BaseAgent — all prompt logic lives here ──────────────────────────────────\n\nclass BaseAgent implements AgentProvider {\n constructor(private backend: LLMBackend, private lang: 'zh' | 'en' = 'zh') {}\n\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const langInstr = this.lang === 'en'\n ? 'Output the theme in English.'\n : '主题用中文输出。';\n\n const system = `你是一个视频混剪规划助手。根据用户的混剪需求，选择最合适的合成风格并提取主题。${langInstr}\n只输出原始 JSON（不要 markdown 代码块，不要任何说明），格式如下：\n{\n \"theme\": \"视频主题名称（简短，用做片头片尾标题）\",\n \"compositionId\": \"CyberpunkCity | TravelVlog | MoodDriven\",\n \"musicStyleOverride\": \"（可选）英文音乐风格描述，留空则用默认\"\n}`;\n\n const user = `可选合成风格：\n${compositionList}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`Agent parseIntent: invalid JSON response: ${raw}`);\n }\n\n const meta = REGISTRY.find(m => m.id === parsed.compositionId) ?? REGISTRY[0];\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n composition: meta,\n musicStyleOverride: parsed.musicStyleOverride || undefined,\n };\n }\n\n async refineQueries(\n slots: Array<{ defaultQuery: string; mood?: string }>,\n theme: string,\n ): Promise<string[]> {\n const slotList = slots.map((s, i) =>\n ` ${i + 1}. default: \"${s.defaultQuery}\" mood: ${s.mood ?? 'any'}`\n ).join('\\n');\n\n const system = `You are a video shot search query writer. Rewrite each slot into an effective semantic search query.\n\nRules:\n- Write a descriptive English sentence or rich phrase (10–20 words) describing exactly what the camera captures\n- Include: subject, action, environment, lighting/time of day — the more specific the visual, the better\n- Use real-world vocabulary (e.g. \"Eiffel Tower\", \"snow-capped Alps\", \"soccer player sprinting\") — never genre labels like \"cyberpunk\", \"cinematic\", or \"bokeh\"\n- If the theme names a real place or event, anchor your queries to that location/event\n- Avoid negations (\"not indoors\") — describe the positive target instead\n- Output ONLY a raw JSON array — no explanation, no markdown fences, no preamble`;\n\n const user = `Theme: ${theme}\nSlots:\n${slotList}\n\n[\"query1\",\"query2\",...]`;\n\n const raw = await this.backend.callLLM(system, user);\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== slots.length) {\n throw new Error(`Expected array of length ${slots.length}`);\n }\n return arr as string[];\n } catch (e) {\n console.warn(` ⚠ Agent refineQueries failed (${(e as Error).message}), using defaults`);\n return slots.map(s => s.defaultQuery);\n }\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown>; visualAnalysis?: string }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n const fieldHints: Record<string, string> = {\n SwitzerlandScenic: '\"tone\": \"warm\"|\"cool\" — warm for sunshine/meadow/sunset, cool for snow/glacier/water',\n SportsHighlight: '\"dramatic\": true|false — true for goals/key moments/emotional close-ups, false for regular play',\n NatureWild: '\"kenBurns\": \"zoom-in\"|\"zoom-out\"|\"pan-left\"|\"pan-right\" — choose based on subject (animal approaching=zoom-in, wide landscape=zoom-out, moving subject=pan)',\n };\n const relevantFields = fieldHints[compositionId] ?? '\"tone\": \"warm\"|\"cool\"';\n\n const clipList = clips.map((c, i) => {\n const visualPart = c.visualAnalysis ? `\\n visual_analysis: \"${c.visualAnalysis.slice(0, 200)}\"` : '';\n return ` ${i + 1}. summary: \"${c.summary}\"${visualPart}`;\n }).join('\\n');\n\n const system = `你是一个视频剪辑视效助手。根据每个镜头的语义描述和视觉分析，为每个镜头生成精准的视觉特效注解。\n只输出原始 JSON 数组（不要 markdown 代码块，不要任何说明），每个元素只包含相关字段（不相关字段省略）。\n当前合成需要的字段：${relevantFields}\n\ncaption 字段写作规则（重要）：\n- 必须基于 visual_analysis（如果有），描述这个镜头实际发生的内容\n- 文字必须服务整个视频的叙事弧，而非单纯描述这一个镜头的内容\n- ${this.lang === 'en' ? '**Language: ENGLISH only. Maximum 30 characters. Do NOT use Chinese.**' : '**语言：必须用中文，禁止英文。最多10个字，超过会被硬截断显示省略号。**'}\n- 风格：简洁有力，像纪录片旁白或诗句，传递情绪而非描述细节\n- 反例（太长/太平）：\"一只黑豹在非洲热带雨林的茂密植被中悄悄地穿行觅食\" → 正例：\"暗夜行者，无声的猎手\"\n输出格式示例：[{\"tone\":\"cool\"},{\"tone\":\"warm\",\"dramatic\":true},...]\n元素数量必须与输入完全相同。`;\n\n const user = `合成ID：${compositionId}\n镜头列表（共 ${clips.length} 个）：\n${clipList}\n\n请为每个镜头生成视效注解，输出 JSON 数组。`;\n\n const raw = await this.backend.callLLM(system, user);\n let annotations: ClipAnnotation[];\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== clips.length) {\n throw new Error(`Expected array of length ${clips.length}`);\n }\n annotations = arr as ClipAnnotation[];\n } catch (e) {\n console.warn(` ⚠ annotateClips failed (${(e as Error).message}), skipping annotations`);\n annotations = clips.map(() => ({}));\n }\n\n // Augment with keyframe brightness analysis (fixed algorithm, no LLM)\n for (let i = 0; i < clips.length; i++) {\n const kf = clips[i].keyframePath;\n if (kf) annotations[i].textBg = analyzeKeyframe(kf, compositionId);\n }\n return annotations;\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const libraryDesc = library.videos.map(v =>\n ` 【${v.name}】时长${Math.round(v.duration / 60)}分钟，${v.shotCount}个镜头\\n` +\n ` 情绪: ${v.moods.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 场景: ${v.scenes.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 镜头样本: ${v.sampleSummaries.slice(0, 8).map(s => `\"${s.slice(0, 60)}\"`).join(' / ')}`\n ).join('\\n\\n');\n\n const system = `你是一个视频混剪规划师。用户有一个视频素材库，你需要：\n1. 仔细阅读库里每个视频的镜头样本描述（sampleSummaries）\n2. 根据用户需求，选择最合适的合成模板\n3. 生成专门针对这个库内容的搜索槽位 — 每条 query 必须用英文写成10-20词的具体视觉描述句\n\nQuery 写作规则（非常重要）：\n- 使用真实的视觉描述，例如 \"aerial view of Hong Kong skyscrapers at night with illuminated windows\"\n- 直接借用库里样本描述中出现的词汇和场景，命中率更高\n- 禁止使用风格标签（cyberpunk, cinematic, bokeh）或消极描述（not indoors）\n- 如遇地名或运动项目，直接写进查询（\"Paris Eiffel Tower aerial\", \"soccer player kicking ball on green field\"）\n\n${this.lang === 'en' ? 'Output the theme field in English.' : '主题 theme 字段用中文输出。'}\n只输出原始 JSON（不要 markdown 代码块，不要任何说明）：\n{\n \"theme\": \"视频主题（简短标题）\",\n \"compositionId\": \"选择的合成ID\",\n \"musicStyle\": \"英文音乐风格描述\",\n \"slots\": [\n { \"query\": \"10-20 word English visual description\", \"mood\": \"calm|fast|slow|urban\", \"extra\": {} },\n ...\n ]\n}\nslots数量建议6-10个，与选择的合成模板匹配。`;\n\n const user = `可用合成模板：\n${compositionList}\n\n用户素材库内容：\n${libraryDesc}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`planFromLibrary: invalid JSON response: ${raw}`);\n }\n\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n compositionId: parsed.compositionId ?? REGISTRY[0].id,\n musicStyle: parsed.musicStyle ?? 'cinematic ambient',\n slots: Array.isArray(parsed.slots) ? parsed.slots : [],\n };\n }\n}\n\n// ─── Heuristic fallback (no API key) ──────────────────────────────────────────\n\nclass HeuristicAgent implements AgentProvider {\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const req = userRequest.toLowerCase();\n let meta: CompositionMeta;\n if (req.includes('赛博') || req.includes('cyberpunk') || req.includes('霓虹') || req.includes('夜景')) {\n meta = REGISTRY.find(m => m.id === 'CyberpunkCity')!;\n } else if (req.includes('旅行') || req.includes('vlog') || req.includes('城市名') || req.includes('地标')) {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n } else {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n }\n const theme = userRequest.split(/[,，。\\n]/)[0].slice(0, 20).trim();\n console.log(` ℹ HeuristicAgent: no API key — selected \"${meta.label}\" for \"${theme}\"`);\n return { theme, composition: meta };\n }\n\n async refineQueries(slots: Array<{ defaultQuery: string }>, _theme: string): Promise<string[]> {\n return slots.map(s => s.defaultQuery);\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; visualAnalysis?: string }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n return clips.map(c => {\n const ann: ClipAnnotation = {};\n if (c.keyframePath) ann.textBg = analyzeKeyframe(c.keyframePath, compositionId);\n return ann;\n });\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n // Heuristic: reuse parseIntent to pick a composition, then use registry defaults as slots\n const intent = await this.parseIntent(userRequest);\n return {\n theme: intent.theme,\n compositionId: intent.composition.id,\n musicStyle: intent.musicStyleOverride ?? intent.composition.musicStyle,\n slots: intent.composition.shotSlots.map(s => ({\n query: s.query,\n mood: s.mood,\n extra: s.extra,\n })),\n };\n }\n}\n\n// ─── Factory ──────────────────────────────────────────────────────────────────\n\nexport type AgentProviderName = 'claude' | 'openai' | 'openai-compat' | 'none';\n\nexport interface CreateAgentOptions {\n provider: AgentProviderName;\n apiKey?: string;\n model?: string;\n /** Only for 'openai-compat': base URL of the compatible endpoint */\n baseUrl?: string;\n /** Output language for theme titles and LLM-generated captions ('zh' | 'en', default 'zh') */\n lang?: 'zh' | 'en';\n}\n\n/**\n * createAgent — instantiate the right backend and wrap it in BaseAgent.\n *\n * Examples:\n * createAgent({ provider: 'claude', apiKey: '...' })\n * createAgent({ provider: 'openai', apiKey: '...', model: 'gpt-4o' })\n * createAgent({ provider: 'openai-compat', apiKey: '...', model: 'deepseek-chat', baseUrl: 'https://api.deepseek.com' })\n * createAgent({ provider: 'openai-compat', apiKey: 'ollama', model: 'llama3', baseUrl: 'http://localhost:11434' })\n * createAgent({ provider: 'none' }) // keyword heuristics only\n */\nexport function createAgent(opts: CreateAgentOptions): AgentProvider;\n/** @deprecated Pass an options object instead */\nexport function createAgent(provider: AgentProviderName, apiKey?: string, model?: string): AgentProvider;\nexport function createAgent(\n optsOrProvider: CreateAgentOptions | AgentProviderName,\n apiKey?: string,\n model?: string,\n): AgentProvider {\n const opts: CreateAgentOptions =\n typeof optsOrProvider === 'string'\n ? { provider: optsOrProvider as AgentProviderName, apiKey, model }\n : optsOrProvider;\n\n if (opts.provider === 'claude') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=claude but ANTHROPIC_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new AnthropicBackend(opts.apiKey, opts.model ?? 'claude-sonnet-4-6'), opts.lang);\n }\n\n if (opts.provider === 'openai') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=openai but OPENAI_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'gpt-4o'), opts.lang);\n }\n\n if (opts.provider === 'openai-compat') {\n if (!opts.apiKey || !opts.baseUrl) {\n console.warn(' ⚠ openai-compat requires apiKey + baseUrl — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'default', opts.baseUrl), opts.lang);\n }\n\n return new HeuristicAgent();\n}\n"}, "a02111d30d8e6b54962dc2a1a93a301ae38b5e74": {"t": 1785763893.8272986, "n": 19217, "h": "c3e471e3b6984098", "c": "/**\n * Agent Layer — LLM-driven intent parsing and shot selection\n *\n * Architecture:\n * - LLMBackend: thin interface for any LLM provider (one method: callLLM)\n * - BaseAgent: all prompt logic lives here, calls this.backend.callLLM()\n * - Provider backends: AnthropicBackend, OpenAIBackend, OpenAICompatBackend\n * - HeuristicAgent: keyword fallback, no API key required\n *\n * Provider selection (set AGENT_PROVIDER in .env):\n * 'claude' — Anthropic API (ANTHROPIC_API_KEY)\n * 'openai' — OpenAI API (OPENAI_API_KEY)\n * 'openai-compat' — Any OpenAI-compatible endpoint (OPENAI_COMPAT_BASE_URL + OPENAI_COMPAT_API_KEY)\n * Works with: Ollama, vLLM, Together, Groq, DeepSeek, Azure OpenAI, etc.\n * 'none' — Heuristic fallback (no API key required)\n */\nimport { REGISTRY, CompositionMeta } from './registry';\nimport { analyzeKeyframe } from './brightness';\n\nexport interface ParsedIntent {\n /** Main theme / title for the video */\n theme: string;\n /** Which composition to use */\n composition: CompositionMeta;\n /** Per-slot search query overrides (optional — falls back to registry defaults) */\n queryOverrides?: string[];\n /** Music style description override (optional) */\n musicStyleOverride?: string;\n}\n\n/**\n * Per-clip annotation generated by the LLM based on the actual shot summary.\n * These override or supplement the static `extra` fields from the registry.\n *\n * Fields are composition-aware: only relevant fields are used per composition.\n * - tone: 'warm' | 'cool' — color grade direction (SwitzerlandScenic)\n * - dramatic: boolean — slow-motion / emphasis clip (SportsHighlight)\n * - transition: 'fade' | 'slide' — per-clip transition override (future use)\n * - kenBurns: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right' (NatureWild)\n * - caption: string — override narrative caption if LLM improves it\n * - textBg: Record of region→'light'|'dark' — keyframe brightness analysis result.\n * Keys match TEXT_REGIONS in brightness.ts (e.g. \"NatureWild.caption\").\n * 'light' means the video background is bright → overlay needed.\n * Set by HeuristicAgent via keyframe pixel analysis; LLM agents can\n * also populate this from vision analysis.\n */\nexport interface ClipAnnotation {\n tone?: 'warm' | 'cool';\n dramatic?: boolean;\n transition?: 'fade' | 'slide';\n kenBurns?: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right';\n caption?: string;\n /** Per-region brightness result from keyframe analysis */\n textBg?: Record<string, 'light' | 'dark'>;\n}\n\n// ─── LLM Backend interface ────────────────────────────────────────────────────\n\n/**\n * Minimal interface any LLM provider must implement.\n * All prompt logic lives in BaseAgent; backends only handle HTTP transport.\n */\nexport interface LLMBackend {\n callLLM(system: string, user: string): Promise<string>;\n}\n\n// ─── Anthropic backend ────────────────────────────────────────────────────────\n\nexport class AnthropicBackend implements LLMBackend {\n constructor(\n private apiKey: string,\n private model = 'claude-sonnet-4-6',\n ) {}\n\n async callLLM(system: string, user: string): Promise<string> {\n const res = await fetch('https://api.anthropic.com/v1/messages', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'x-api-key': this.apiKey,\n 'anthropic-version': '2023-06-01',\n },\n body: JSON.stringify({\n model: this.model,\n max_tokens: 1024,\n system,\n messages: [{ role: 'user', content: user }],\n }),\n });\n if (!res.ok) throw new Error(`Anthropic API error ${res.status}: ${await res.text()}`);\n const data = await res.json() as any;\n return data.content[0].text as string;\n }\n}\n\n// ─── OpenAI-compatible backend (OpenAI, Ollama, vLLM, Groq, DeepSeek, …) ─────\n\nexport class OpenAICompatBackend implements LLMBackend {\n private baseUrl: string;\n\n constructor(\n private apiKey: string,\n private model: string,\n /** Base URL — defaults to OpenAI. Override for Ollama/vLLM/etc. */\n baseUrl?: string,\n ) {\n const base = (baseUrl ?? 'https://api.openai.com').replace(/\\/$/, '');\n // If caller already includes /v1 (e.g. OpenRouter), don't double-append it\n this.baseUrl = base.endsWith('/v1') ? base : `${base}/v1`;\n }\n\n async callLLM(system: string, user: string): Promise<string> {\n const res = await fetch(`${this.baseUrl}/chat/completions`, {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'Authorization': `Bearer ${this.apiKey}`,\n },\n body: JSON.stringify({\n model: this.model,\n messages: [\n { role: 'system', content: system },\n { role: 'user', content: user },\n ],\n max_tokens: 1024,\n }),\n });\n if (!res.ok) throw new Error(`LLM API error ${res.status}: ${await res.text()}`);\n const data = await res.json() as any;\n return data.choices[0].message.content as string;\n }\n}\n\n// ─── AgentProvider interface ──────────────────────────────────────────────────\n\n/** Compact summary of a user's media library, passed to LLM for dynamic planning. */\nexport interface LibrarySummary {\n videos: Array<{\n name: string;\n duration: number;\n shotCount: number;\n /** Up to 20 shot summaries sampled from the video */\n sampleSummaries: string[];\n /** Unique mood tags found across sampled shots */\n moods: string[];\n /** Unique scene tags found across sampled shots */\n scenes: string[];\n }>;\n}\n\n/** A dynamically generated slot list produced by LLM after seeing the library. */\nexport interface DynamicPlan {\n /** Theme/title for the video */\n theme: string;\n /** Which registered composition to use as the Remotion template */\n compositionId: string;\n /** Music style description */\n musicStyle: string;\n /** Slot list tailored to what actually exists in the library */\n slots: Array<{\n query: string;\n mood?: string;\n /** Any extra metadata the LLM wants to attach (caption, location, etc.) */\n extra?: Record<string, unknown>;\n }>;\n}\n\nexport interface AgentProvider {\n parseIntent(userRequest: string): Promise<ParsedIntent>;\n refineQueries(slots: Array<{ defaultQuery: string; mood?: string }>, theme: string): Promise<string[]>;\n /**\n * Given the actual shot summaries returned by ShotAI, produce per-clip\n * visual annotations that drive Remotion special-effects decisions.\n *\n * @param clips The resolved clip list (summary, optional keyframePath, slot extra, optional visual analysis)\n * @param compositionId Which composition is being rendered (determines which fields matter)\n */\n annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown>; visualAnalysis?: string }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]>;\n /**\n * After seeing what's actually in the user's library, generate a tailored plan:\n * which composition to use and what slots to search for.\n */\n planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan>;\n}\n\n// ─── BaseAgent — all prompt logic lives here ──────────────────────────────────\n\nclass BaseAgent implements AgentProvider {\n constructor(private backend: LLMBackend, private lang: 'zh' | 'en' = 'zh') {}\n\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const langInstr = this.lang === 'en'\n ? 'Output the theme in English.'\n : '主题用中文输出。';\n\n const system = `你是一个视频混剪规划助手。根据用户的混剪需求，选择最合适的合成风格并提取主题。${langInstr}\n只输出原始 JSON（不要 markdown 代码块，不要任何说明），格式如下：\n{\n \"theme\": \"视频主题名称（简短，用做片头片尾标题）\",\n \"compositionId\": \"CyberpunkCity | TravelVlog | MoodDriven\",\n \"musicStyleOverride\": \"（可选）英文音乐风格描述，留空则用默认\"\n}`;\n\n const user = `可选合成风格：\n${compositionList}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`Agent parseIntent: invalid JSON response: ${raw}`);\n }\n\n const meta = REGISTRY.find(m => m.id === parsed.compositionId) ?? REGISTRY[0];\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n composition: meta,\n musicStyleOverride: parsed.musicStyleOverride || undefined,\n };\n }\n\n async refineQueries(\n slots: Array<{ defaultQuery: string; mood?: string }>,\n theme: string,\n ): Promise<string[]> {\n const slotList = slots.map((s, i) =>\n ` ${i + 1}. default: \"${s.defaultQuery}\" mood: ${s.mood ?? 'any'}`\n ).join('\\n');\n\n const system = `You are a video shot search query writer. Rewrite each slot into an effective semantic search query.\n\nRules:\n- Write a descriptive English sentence or rich phrase (10–20 words) describing exactly what the camera captures\n- Include: subject, action, environment, lighting/time of day — the more specific the visual, the better\n- Use real-world vocabulary (e.g. \"Eiffel Tower\", \"snow-capped Alps\", \"soccer player sprinting\") — never genre labels like \"cyberpunk\", \"cinematic\", or \"bokeh\"\n- If the theme names a real place or event, anchor your queries to that location/event\n- Avoid negations (\"not indoors\") — describe the positive target instead\n- Output ONLY a raw JSON array — no explanation, no markdown fences, no preamble`;\n\n const user = `Theme: ${theme}\nSlots:\n${slotList}\n\n[\"query1\",\"query2\",...]`;\n\n const raw = await this.backend.callLLM(system, user);\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== slots.length) {\n throw new Error(`Expected array of length ${slots.length}`);\n }\n return arr as string[];\n } catch (e) {\n console.warn(` ⚠ Agent refineQueries failed (${(e as Error).message}), using defaults`);\n return slots.map(s => s.defaultQuery);\n }\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown>; visualAnalysis?: string }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n const fieldHints: Record<string, string> = {\n SwitzerlandScenic: '\"tone\": \"warm\"|\"cool\" — warm for sunshine/meadow/sunset, cool for snow/glacier/water',\n SportsHighlight: '\"dramatic\": true|false — true for goals/key moments/emotional close-ups, false for regular play',\n NatureWild: '\"kenBurns\": \"zoom-in\"|\"zoom-out\"|\"pan-left\"|\"pan-right\" — choose based on subject (animal approaching=zoom-in, wide landscape=zoom-out, moving subject=pan)',\n };\n const relevantFields = fieldHints[compositionId] ?? '\"tone\": \"warm\"|\"cool\"';\n\n const clipList = clips.map((c, i) => {\n const visualPart = c.visualAnalysis ? `\\n visual_analysis: \"${c.visualAnalysis.slice(0, 200)}\"` : '';\n return ` ${i + 1}. summary: \"${c.summary}\"${visualPart}`;\n }).join('\\n');\n\n const system = `你是一个视频剪辑视效助手。根据每个镜头的语义描述和视觉分析，为每个镜头生成精准的视觉特效注解。\n只输出原始 JSON 数组（不要 markdown 代码块，不要任何说明），每个元素只包含相关字段（不相关字段省略）。\n当前合成需要的字段：${relevantFields}\n\ncaption 字段写作规则（重要）：\n- 必须基于 visual_analysis（如果有），描述这个镜头实际发生的内容\n- 文字必须服务整个视频的叙事弧，而非单纯描述这一个镜头的内容\n- 中文：最多16个字（超过会被截断，务必控制）\n- 英文：最多30个字符\n- 风格：简洁有力，像纪录片旁白或诗句，传递情绪而非描述细节\n- 反例（太长/太平）：\"一只黑豹在非洲热带雨林的茂密植被中悄悄地穿行觅食\" → 正例：\"暗夜行者，无声的猎手\"\n${this.lang === 'en' ? 'Generate caption field in English.' : 'caption 字段用中文。'}\n输出格式示例：[{\"tone\":\"cool\"},{\"tone\":\"warm\",\"dramatic\":true},...]\n元素数量必须与输入完全相同。`;\n\n const user = `合成ID：${compositionId}\n镜头列表（共 ${clips.length} 个）：\n${clipList}\n\n请为每个镜头生成视效注解，输出 JSON 数组。`;\n\n const raw = await this.backend.callLLM(system, user);\n let annotations: ClipAnnotation[];\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== clips.length) {\n throw new Error(`Expected array of length ${clips.length}`);\n }\n annotations = arr as ClipAnnotation[];\n } catch (e) {\n console.warn(` ⚠ annotateClips failed (${(e as Error).message}), skipping annotations`);\n annotations = clips.map(() => ({}));\n }\n\n // Augment with keyframe brightness analysis (fixed algorithm, no LLM)\n for (let i = 0; i < clips.length; i++) {\n const kf = clips[i].keyframePath;\n if (kf) annotations[i].textBg = analyzeKeyframe(kf, compositionId);\n }\n return annotations;\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const libraryDesc = library.videos.map(v =>\n ` 【${v.name}】时长${Math.round(v.duration / 60)}分钟，${v.shotCount}个镜头\\n` +\n ` 情绪: ${v.moods.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 场景: ${v.scenes.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 镜头样本: ${v.sampleSummaries.slice(0, 8).map(s => `\"${s.slice(0, 60)}\"`).join(' / ')}`\n ).join('\\n\\n');\n\n const system = `你是一个视频混剪规划师。用户有一个视频素材库，你需要：\n1. 仔细阅读库里每个视频的镜头样本描述（sampleSummaries）\n2. 根据用户需求，选择最合适的合成模板\n3. 生成专门针对这个库内容的搜索槽位 — 每条 query 必须用英文写成10-20词的具体视觉描述句\n\nQuery 写作规则（非常重要）：\n- 使用真实的视觉描述，例如 \"aerial view of Hong Kong skyscrapers at night with illuminated windows\"\n- 直接借用库里样本描述中出现的词汇和场景，命中率更高\n- 禁止使用风格标签（cyberpunk, cinematic, bokeh）或消极描述（not indoors）\n- 如遇地名或运动项目，直接写进查询（\"Paris Eiffel Tower aerial\", \"soccer player kicking ball on green field\"）\n\n${this.lang === 'en' ? 'Output the theme field in English.' : '主题 theme 字段用中文输出。'}\n只输出原始 JSON（不要 markdown 代码块，不要任何说明）：\n{\n \"theme\": \"视频主题（简短标题）\",\n \"compositionId\": \"选择的合成ID\",\n \"musicStyle\": \"英文音乐风格描述\",\n \"slots\": [\n { \"query\": \"10-20 word English visual description\", \"mood\": \"calm|fast|slow|urban\", \"extra\": {} },\n ...\n ]\n}\nslots数量建议6-10个，与选择的合成模板匹配。`;\n\n const user = `可用合成模板：\n${compositionList}\n\n用户素材库内容：\n${libraryDesc}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`planFromLibrary: invalid JSON response: ${raw}`);\n }\n\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n compositionId: parsed.compositionId ?? REGISTRY[0].id,\n musicStyle: parsed.musicStyle ?? 'cinematic ambient',\n slots: Array.isArray(parsed.slots) ? parsed.slots : [],\n };\n }\n}\n\n// ─── Heuristic fallback (no API key) ──────────────────────────────────────────\n\nclass HeuristicAgent implements AgentProvider {\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const req = userRequest.toLowerCase();\n let meta: CompositionMeta;\n if (req.includes('赛博') || req.includes('cyberpunk') || req.includes('霓虹') || req.includes('夜景')) {\n meta = REGISTRY.find(m => m.id === 'CyberpunkCity')!;\n } else if (req.includes('旅行') || req.includes('vlog') || req.includes('城市名') || req.includes('地标')) {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n } else {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n }\n const theme = userRequest.split(/[,，。\\n]/)[0].slice(0, 20).trim();\n console.log(` ℹ HeuristicAgent: no API key — selected \"${meta.label}\" for \"${theme}\"`);\n return { theme, composition: meta };\n }\n\n async refineQueries(slots: Array<{ defaultQuery: string }>, _theme: string): Promise<string[]> {\n return slots.map(s => s.defaultQuery);\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; visualAnalysis?: string }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n return clips.map(c => {\n const ann: ClipAnnotation = {};\n if (c.keyframePath) ann.textBg = analyzeKeyframe(c.keyframePath, compositionId);\n return ann;\n });\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n // Heuristic: reuse parseIntent to pick a composition, then use registry defaults as slots\n const intent = await this.parseIntent(userRequest);\n return {\n theme: intent.theme,\n compositionId: intent.composition.id,\n musicStyle: intent.musicStyleOverride ?? intent.composition.musicStyle,\n slots: intent.composition.shotSlots.map(s => ({\n query: s.query,\n mood: s.mood,\n extra: s.extra,\n })),\n };\n }\n}\n\n// ─── Factory ──────────────────────────────────────────────────────────────────\n\nexport type AgentProviderName = 'claude' | 'openai' | 'openai-compat' | 'none';\n\nexport interface CreateAgentOptions {\n provider: AgentProviderName;\n apiKey?: string;\n model?: string;\n /** Only for 'openai-compat': base URL of the compatible endpoint */\n baseUrl?: string;\n /** Output language for theme titles and LLM-generated captions ('zh' | 'en', default 'zh') */\n lang?: 'zh' | 'en';\n}\n\n/**\n * createAgent — instantiate the right backend and wrap it in BaseAgent.\n *\n * Examples:\n * createAgent({ provider: 'claude', apiKey: '...' })\n * createAgent({ provider: 'openai', apiKey: '...', model: 'gpt-4o' })\n * createAgent({ provider: 'openai-compat', apiKey: '...', model: 'deepseek-chat', baseUrl: 'https://api.deepseek.com' })\n * createAgent({ provider: 'openai-compat', apiKey: 'ollama', model: 'llama3', baseUrl: 'http://localhost:11434' })\n * createAgent({ provider: 'none' }) // keyword heuristics only\n */\nexport function createAgent(opts: CreateAgentOptions): AgentProvider;\n/** @deprecated Pass an options object instead */\nexport function createAgent(provider: AgentProviderName, apiKey?: string, model?: string): AgentProvider;\nexport function createAgent(\n optsOrProvider: CreateAgentOptions | AgentProviderName,\n apiKey?: string,\n model?: string,\n): AgentProvider {\n const opts: CreateAgentOptions =\n typeof optsOrProvider === 'string'\n ? { provider: optsOrProvider as AgentProviderName, apiKey, model }\n : optsOrProvider;\n\n if (opts.provider === 'claude') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=claude but ANTHROPIC_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new AnthropicBackend(opts.apiKey, opts.model ?? 'claude-sonnet-4-6'), opts.lang);\n }\n\n if (opts.provider === 'openai') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=openai but OPENAI_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'gpt-4o'), opts.lang);\n }\n\n if (opts.provider === 'openai-compat') {\n if (!opts.apiKey || !opts.baseUrl) {\n console.warn(' ⚠ openai-compat requires apiKey + baseUrl — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'default', opts.baseUrl), opts.lang);\n }\n\n return new HeuristicAgent();\n}\n"}, "ae27998efd1cb1e069ccbe5f83037469e7b8be04": {"t": 1785763895.4377685, "n": 17917, "h": "20c27a317a03b73e", "c": "/**\n * Agent Layer — LLM-driven intent parsing and shot selection\n *\n * Architecture:\n * - LLMBackend: thin interface for any LLM provider (one method: callLLM)\n * - BaseAgent: all prompt logic lives here, calls this.backend.callLLM()\n * - Provider backends: AnthropicBackend, OpenAIBackend, OpenAICompatBackend\n * - HeuristicAgent: keyword fallback, no API key required\n *\n * Provider selection (set AGENT_PROVIDER in .env):\n * 'claude' — Anthropic API (ANTHROPIC_API_KEY)\n * 'openai' — OpenAI API (OPENAI_API_KEY)\n * 'openai-compat' — Any OpenAI-compatible endpoint (OPENAI_COMPAT_BASE_URL + OPENAI_COMPAT_API_KEY)\n * Works with: Ollama, vLLM, Together, Groq, DeepSeek, Azure OpenAI, etc.\n * 'none' — Heuristic fallback (no API key required)\n */\nimport { REGISTRY, CompositionMeta } from './registry';\nimport { analyzeKeyframe } from './brightness';\n\nexport interface ParsedIntent {\n /** Main theme / title for the video */\n theme: string;\n /** Which composition to use */\n composition: CompositionMeta;\n /** Per-slot search query overrides (optional — falls back to registry defaults) */\n queryOverrides?: string[];\n /** Music style description override (optional) */\n musicStyleOverride?: string;\n}\n\n/**\n * Per-clip annotation generated by the LLM based on the actual shot summary.\n * These override or supplement the static `extra` fields from the registry.\n *\n * Fields are composition-aware: only relevant fields are used per composition.\n * - tone: 'warm' | 'cool' — color grade direction (SwitzerlandScenic)\n * - dramatic: boolean — slow-motion / emphasis clip (SportsHighlight)\n * - transition: 'fade' | 'slide' — per-clip transition override (future use)\n * - kenBurns: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right' (NatureWild)\n * - caption: string — override narrative caption if LLM improves it\n * - textBg: Record of region→'light'|'dark' — keyframe brightness analysis result.\n * Keys match TEXT_REGIONS in brightness.ts (e.g. \"NatureWild.caption\").\n * 'light' means the video background is bright → overlay needed.\n * Set by HeuristicAgent via keyframe pixel analysis; LLM agents can\n * also populate this from vision analysis.\n */\nexport interface ClipAnnotation {\n tone?: 'warm' | 'cool';\n dramatic?: boolean;\n transition?: 'fade' | 'slide';\n kenBurns?: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right';\n caption?: string;\n /** Per-region brightness result from keyframe analysis */\n textBg?: Record<string, 'light' | 'dark'>;\n}\n\n// ─── LLM Backend interface ────────────────────────────────────────────────────\n\n/**\n * Minimal interface any LLM provider must implement.\n * All prompt logic lives in BaseAgent; backends only handle HTTP transport.\n */\nexport interface LLMBackend {\n callLLM(system: string, user: string): Promise<string>;\n}\n\n// ─── Anthropic backend ────────────────────────────────────────────────────────\n\nexport class AnthropicBackend implements LLMBackend {\n constructor(\n private apiKey: string,\n private model = 'claude-sonnet-4-6',\n ) {}\n\n async callLLM(system: string, user: string): Promise<string> {\n const res = await fetch('https://api.anthropic.com/v1/messages', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'x-api-key': this.apiKey,\n 'anthropic-version': '2023-06-01',\n },\n body: JSON.stringify({\n model: this.model,\n max_tokens: 1024,\n system,\n messages: [{ role: 'user', content: user }],\n }),\n });\n if (!res.ok) throw new Error(`Anthropic API error ${res.status}: ${await res.text()}`);\n const data = await res.json() as any;\n return data.content[0].text as string;\n }\n}\n\n// ─── OpenAI-compatible backend (OpenAI, Ollama, vLLM, Groq, DeepSeek, …) ─────\n\nexport class OpenAICompatBackend implements LLMBackend {\n private baseUrl: string;\n\n constructor(\n private apiKey: string,\n private model: string,\n /** Base URL — defaults to OpenAI. Override for Ollama/vLLM/etc. */\n baseUrl?: string,\n ) {\n const base = (baseUrl ?? 'https://api.openai.com').replace(/\\/$/, '');\n // If caller already includes /v1 (e.g. OpenRouter), don't double-append it\n this.baseUrl = base.endsWith('/v1') ? base : `${base}/v1`;\n }\n\n async callLLM(system: string, user: string): Promise<string> {\n const res = await fetch(`${this.baseUrl}/chat/completions`, {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'Authorization': `Bearer ${this.apiKey}`,\n },\n body: JSON.stringify({\n model: this.model,\n messages: [\n { role: 'system', content: system },\n { role: 'user', content: user },\n ],\n max_tokens: 1024,\n }),\n });\n if (!res.ok) throw new Error(`LLM API error ${res.status}: ${await res.text()}`);\n const data = await res.json() as any;\n return data.choices[0].message.content as string;\n }\n}\n\n// ─── AgentProvider interface ──────────────────────────────────────────────────\n\n/** Compact summary of a user's media library, passed to LLM for dynamic planning. */\nexport interface LibrarySummary {\n videos: Array<{\n name: string;\n duration: number;\n shotCount: number;\n /** Up to 20 shot summaries sampled from the video */\n sampleSummaries: string[];\n /** Unique mood tags found across sampled shots */\n moods: string[];\n /** Unique scene tags found across sampled shots */\n scenes: string[];\n }>;\n}\n\n/** A dynamically generated slot list produced by LLM after seeing the library. */\nexport interface DynamicPlan {\n /** Theme/title for the video */\n theme: string;\n /** Which registered composition to use as the Remotion template */\n compositionId: string;\n /** Music style description */\n musicStyle: string;\n /** Slot list tailored to what actually exists in the library */\n slots: Array<{\n query: string;\n mood?: string;\n /** Any extra metadata the LLM wants to attach (caption, location, etc.) */\n extra?: Record<string, unknown>;\n }>;\n}\n\nexport interface AgentProvider {\n parseIntent(userRequest: string): Promise<ParsedIntent>;\n refineQueries(slots: Array<{ defaultQuery: string; mood?: string }>, theme: string): Promise<string[]>;\n /**\n * Given the actual shot summaries returned by ShotAI, produce per-clip\n * visual annotations that drive Remotion special-effects decisions.\n *\n * @param clips The resolved clip list (summary, optional keyframePath, slot extra)\n * @param compositionId Which composition is being rendered (determines which fields matter)\n */\n annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown> }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]>;\n /**\n * After seeing what's actually in the user's library, generate a tailored plan:\n * which composition to use and what slots to search for.\n */\n planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan>;\n}\n\n// ─── BaseAgent — all prompt logic lives here ──────────────────────────────────\n\nclass BaseAgent implements AgentProvider {\n constructor(private backend: LLMBackend, private lang: 'zh' | 'en' = 'zh') {}\n\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const langInstr = this.lang === 'en'\n ? 'Output the theme in English.'\n : '主题用中文输出。';\n\n const system = `你是一个视频混剪规划助手。根据用户的混剪需求，选择最合适的合成风格并提取主题。${langInstr}\n只输出 JSON，格式如下：\n{\n \"theme\": \"视频主题名称（简短，用做片头片尾标题）\",\n \"compositionId\": \"CyberpunkCity | TravelVlog | MoodDriven\",\n \"musicStyleOverride\": \"（可选）英文音乐风格描述，留空则用默认\"\n}`;\n\n const user = `可选合成风格：\n${compositionList}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`Agent parseIntent: invalid JSON response: ${raw}`);\n }\n\n const meta = REGISTRY.find(m => m.id === parsed.compositionId) ?? REGISTRY[0];\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n composition: meta,\n musicStyleOverride: parsed.musicStyleOverride || undefined,\n };\n }\n\n async refineQueries(\n slots: Array<{ defaultQuery: string; mood?: string }>,\n theme: string,\n ): Promise<string[]> {\n const slotList = slots.map((s, i) =>\n ` ${i + 1}. default: \"${s.defaultQuery}\" mood: ${s.mood ?? 'any'}`\n ).join('\\n');\n\n const system = `Shot search query optimizer. For each slot, write a short English phrase (5–8 words) describing what the camera sees. Used for semantic vector search — concise beats verbose.\nOutput ONLY a JSON array of the same length. English only.`;\n\n const user = `Theme: ${theme}\nSlots:\n${slotList}\n\nOutput JSON array:`;\n\n const raw = await this.backend.callLLM(system, user);\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== slots.length) {\n throw new Error(`Expected array of length ${slots.length}`);\n }\n return arr as string[];\n } catch (e) {\n console.warn(` ⚠ Agent refineQueries failed (${(e as Error).message}), using defaults`);\n return slots.map(s => s.defaultQuery);\n }\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown> }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n const fieldHints: Record<string, string> = {\n SwitzerlandScenic: '\"tone\": \"warm\"|\"cool\" — warm for sunshine/meadow/sunset, cool for snow/glacier/water',\n SportsHighlight: '\"dramatic\": true|false — true for goals/key moments/emotional close-ups, false for regular play',\n NatureWild: '\"kenBurns\": \"zoom-in\"|\"zoom-out\"|\"pan-left\"|\"pan-right\" — choose based on subject (animal approaching=zoom-in, wide landscape=zoom-out, moving subject=pan)',\n };\n const relevantFields = fieldHints[compositionId] ?? '\"tone\": \"warm\"|\"cool\"';\n\n const clipList = clips.map((c, i) =>\n ` ${i + 1}. summary: \"${c.summary}\"${c.slotExtra?.caption ? ` caption_hint: \"${c.slotExtra.caption}\"` : ''}`\n ).join('\\n');\n\n const system = `你是一个视频剪辑视效助手。根据每个镜头的语义描述，为每个镜头生成精准的视觉特效注解。\n只输出 JSON 数组，每个元素只包含相关字段（不相关字段省略）。\n当前合成需要的字段：${relevantFields}\n${this.lang === 'en' ? '如需生成 caption 字段，请用英文。' : '如需生成 caption 字段，请用中文。'}\n输出格式示例：[{\"tone\":\"cool\"},{\"tone\":\"warm\",\"dramatic\":true},...]\n元素数量必须与输入完全相同。`;\n\n const user = `合成ID：${compositionId}\n镜头列表（共 ${clips.length} 个）：\n${clipList}\n\n请为每个镜头生成视效注解，输出 JSON 数组。`;\n\n const raw = await this.backend.callLLM(system, user);\n let annotations: ClipAnnotation[];\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== clips.length) {\n throw new Error(`Expected array of length ${clips.length}`);\n }\n annotations = arr as ClipAnnotation[];\n } catch (e) {\n console.warn(` ⚠ annotateClips failed (${(e as Error).message}), skipping annotations`);\n annotations = clips.map(() => ({}));\n }\n\n // Augment with keyframe brightness analysis (fixed algorithm, no LLM)\n for (let i = 0; i < clips.length; i++) {\n const kf = clips[i].keyframePath;\n if (kf) annotations[i].textBg = analyzeKeyframe(kf, compositionId);\n }\n return annotations;\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const libraryDesc = library.videos.map(v =>\n ` 【${v.name}】时长${Math.round(v.duration / 60)}分钟，${v.shotCount}个镜头\\n` +\n ` 情绪: ${v.moods.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 场景: ${v.scenes.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 镜头样本: ${v.sampleSummaries.slice(0, 8).map(s => `\"${s.slice(0, 60)}\"`).join(' / ')}`\n ).join('\\n\\n');\n\n const system = `你是一个视频混剪规划师。用户有一个视频素材库，你需要：\n1. 分析库里实际存在的素材内容\n2. 根据用户需求和库里有的内容，选择最合适的合成模板\n3. 生成专门针对这个库的镜头搜索槽位列表（不要用通用词，要用能在这个库里搜到东西的词）\n\n${this.lang === 'en' ? 'Output the theme field in English.' : '主题 theme 字段用中文输出。'}\n只输出 JSON，格式如下：\n{\n \"theme\": \"视频主题（简短标题）\",\n \"compositionId\": \"选择的合成ID\",\n \"musicStyle\": \"英文音乐风格描述\",\n \"slots\": [\n { \"query\": \"搜索词（针对这个库定制）\", \"mood\": \"calm|fast|slow|urban\", \"extra\": {} },\n ...\n ]\n}\nslots数量建议6-10个，与选择的合成模板匹配。`;\n\n const user = `可用合成模板：\n${compositionList}\n\n用户素材库内容：\n${libraryDesc}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`planFromLibrary: invalid JSON response: ${raw}`);\n }\n\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n compositionId: parsed.compositionId ?? REGISTRY[0].id,\n musicStyle: parsed.musicStyle ?? 'cinematic ambient',\n slots: Array.isArray(parsed.slots) ? parsed.slots : [],\n };\n }\n}\n\n// ─── Heuristic fallback (no API key) ──────────────────────────────────────────\n\nclass HeuristicAgent implements AgentProvider {\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const req = userRequest.toLowerCase();\n let meta: CompositionMeta;\n if (req.includes('赛博') || req.includes('cyberpunk') || req.includes('霓虹') || req.includes('夜景')) {\n meta = REGISTRY.find(m => m.id === 'CyberpunkCity')!;\n } else if (req.includes('旅行') || req.includes('vlog') || req.includes('城市名') || req.includes('地标')) {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n } else {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n }\n const theme = userRequest.split(/[,，。\\n]/)[0].slice(0, 20).trim();\n console.log(` ℹ HeuristicAgent: no API key — selected \"${meta.label}\" for \"${theme}\"`);\n return { theme, composition: meta };\n }\n\n async refineQueries(slots: Array<{ defaultQuery: string }>, _theme: string): Promise<string[]> {\n return slots.map(s => s.defaultQuery);\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n return clips.map(c => {\n const ann: ClipAnnotation = {};\n if (c.keyframePath) ann.textBg = analyzeKeyframe(c.keyframePath, compositionId);\n return ann;\n });\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n // Heuristic: reuse parseIntent to pick a composition, then use registry defaults as slots\n const intent = await this.parseIntent(userRequest);\n return {\n theme: intent.theme,\n compositionId: intent.composition.id,\n musicStyle: intent.musicStyleOverride ?? intent.composition.musicStyle,\n slots: intent.composition.shotSlots.map(s => ({\n query: s.query,\n mood: s.mood,\n extra: s.extra,\n })),\n };\n }\n}\n\n// ─── Factory ──────────────────────────────────────────────────────────────────\n\nexport type AgentProviderName = 'claude' | 'openai' | 'openai-compat' | 'none';\n\nexport interface CreateAgentOptions {\n provider: AgentProviderName;\n apiKey?: string;\n model?: string;\n /** Only for 'openai-compat': base URL of the compatible endpoint */\n baseUrl?: string;\n /** Output language for theme titles and LLM-generated captions ('zh' | 'en', default 'zh') */\n lang?: 'zh' | 'en';\n}\n\n/**\n * createAgent — instantiate the right backend and wrap it in BaseAgent.\n *\n * Examples:\n * createAgent({ provider: 'claude', apiKey: '...' })\n * createAgent({ provider: 'openai', apiKey: '...', model: 'gpt-4o' })\n * createAgent({ provider: 'openai-compat', apiKey: '...', model: 'deepseek-chat', baseUrl: 'https://api.deepseek.com' })\n * createAgent({ provider: 'openai-compat', apiKey: 'ollama', model: 'llama3', baseUrl: 'http://localhost:11434' })\n * createAgent({ provider: 'none' }) // keyword heuristics only\n */\nexport function createAgent(opts: CreateAgentOptions): AgentProvider;\n/** @deprecated Pass an options object instead */\nexport function createAgent(provider: AgentProviderName, apiKey?: string, model?: string): AgentProvider;\nexport function createAgent(\n optsOrProvider: CreateAgentOptions | AgentProviderName,\n apiKey?: string,\n model?: string,\n): AgentProvider {\n const opts: CreateAgentOptions =\n typeof optsOrProvider === 'string'\n ? { provider: optsOrProvider as AgentProviderName, apiKey, model }\n : optsOrProvider;\n\n if (opts.provider === 'claude') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=claude but ANTHROPIC_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new AnthropicBackend(opts.apiKey, opts.model ?? 'claude-sonnet-4-6'), opts.lang);\n }\n\n if (opts.provider === 'openai') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=openai but OPENAI_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'gpt-4o'), opts.lang);\n }\n\n if (opts.provider === 'openai-compat') {\n if (!opts.apiKey || !opts.baseUrl) {\n console.warn(' ⚠ openai-compat requires apiKey + baseUrl — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'default', opts.baseUrl), opts.lang);\n }\n\n return new HeuristicAgent();\n}\n"}, "d1ee486d0134feb0b9e562f9f774127990cad088": {"t": 1785763898.2207181, "n": 17461, "h": "81d7452acbc4d4ad", "c": "/**\n * Agent Layer — LLM-driven intent parsing and shot selection\n *\n * Architecture:\n * - LLMBackend: thin interface for any LLM provider (one method: callLLM)\n * - BaseAgent: all prompt logic lives here, calls this.backend.callLLM()\n * - Provider backends: AnthropicBackend, OpenAIBackend, OpenAICompatBackend\n * - HeuristicAgent: keyword fallback, no API key required\n *\n * Provider selection (set AGENT_PROVIDER in .env):\n * 'claude' — Anthropic API (ANTHROPIC_API_KEY)\n * 'openai' — OpenAI API (OPENAI_API_KEY)\n * 'openai-compat' — Any OpenAI-compatible endpoint (OPENAI_COMPAT_BASE_URL + OPENAI_COMPAT_API_KEY)\n * Works with: Ollama, vLLM, Together, Groq, DeepSeek, Azure OpenAI, etc.\n * 'none' — Heuristic fallback (no API key required)\n */\nimport { REGISTRY, CompositionMeta } from './registry';\nimport { analyzeKeyframe } from './brightness';\n\nexport interface ParsedIntent {\n /** Main theme / title for the video */\n theme: string;\n /** Which composition to use */\n composition: CompositionMeta;\n /** Per-slot search query overrides (optional — falls back to registry defaults) */\n queryOverrides?: string[];\n /** Music style description override (optional) */\n musicStyleOverride?: string;\n}\n\n/**\n * Per-clip annotation generated by the LLM based on the actual shot summary.\n * These override or supplement the static `extra` fields from the registry.\n *\n * Fields are composition-aware: only relevant fields are used per composition.\n * - tone: 'warm' | 'cool' — color grade direction (SwitzerlandScenic)\n * - dramatic: boolean — slow-motion / emphasis clip (SportsHighlight)\n * - transition: 'fade' | 'slide' — per-clip transition override (future use)\n * - kenBurns: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right' (NatureWild)\n * - caption: string — override narrative caption if LLM improves it\n * - textBg: Record of region→'light'|'dark' — keyframe brightness analysis result.\n * Keys match TEXT_REGIONS in brightness.ts (e.g. \"NatureWild.caption\").\n * 'light' means the video background is bright → overlay needed.\n * Set by HeuristicAgent via keyframe pixel analysis; LLM agents can\n * also populate this from vision analysis.\n */\nexport interface ClipAnnotation {\n tone?: 'warm' | 'cool';\n dramatic?: boolean;\n transition?: 'fade' | 'slide';\n kenBurns?: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right';\n caption?: string;\n /** Per-region brightness result from keyframe analysis */\n textBg?: Record<string, 'light' | 'dark'>;\n}\n\n// ─── LLM Backend interface ────────────────────────────────────────────────────\n\n/**\n * Minimal interface any LLM provider must implement.\n * All prompt logic lives in BaseAgent; backends only handle HTTP transport.\n */\nexport interface LLMBackend {\n callLLM(system: string, user: string): Promise<string>;\n}\n\n// ─── Anthropic backend ────────────────────────────────────────────────────────\n\nexport class AnthropicBackend implements LLMBackend {\n constructor(\n private apiKey: string,\n private model = 'claude-sonnet-4-6',\n ) {}\n\n async callLLM(system: string, user: string): Promise<string> {\n const res = await fetch('https://api.anthropic.com/v1/messages', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'x-api-key': this.apiKey,\n 'anthropic-version': '2023-06-01',\n },\n body: JSON.stringify({\n model: this.model,\n max_tokens: 1024,\n system,\n messages: [{ role: 'user', content: user }],\n }),\n });\n if (!res.ok) throw new Error(`Anthropic API error ${res.status}: ${await res.text()}`);\n const data = await res.json() as any;\n return data.content[0].text as string;\n }\n}\n\n// ─── OpenAI-compatible backend (OpenAI, Ollama, vLLM, Groq, DeepSeek, …) ─────\n\nexport class OpenAICompatBackend implements LLMBackend {\n private baseUrl: string;\n\n constructor(\n private apiKey: string,\n private model: string,\n /** Base URL — defaults to OpenAI. Override for Ollama/vLLM/etc. */\n baseUrl?: string,\n ) {\n const base = (baseUrl ?? 'https://api.openai.com').replace(/\\/$/, '');\n // If caller already includes /v1 (e.g. OpenRouter), don't double-append it\n this.baseUrl = base.endsWith('/v1') ? base : `${base}/v1`;\n }\n\n async callLLM(system: string, user: string): Promise<string> {\n const res = await fetch(`${this.baseUrl}/chat/completions`, {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'Authorization': `Bearer ${this.apiKey}`,\n },\n body: JSON.stringify({\n model: this.model,\n messages: [\n { role: 'system', content: system },\n { role: 'user', content: user },\n ],\n max_tokens: 1024,\n }),\n });\n if (!res.ok) throw new Error(`LLM API error ${res.status}: ${await res.text()}`);\n const data = await res.json() as any;\n return data.choices[0].message.content as string;\n }\n}\n\n// ─── AgentProvider interface ──────────────────────────────────────────────────\n\n/** Compact summary of a user's media library, passed to LLM for dynamic planning. */\nexport interface LibrarySummary {\n videos: Array<{\n name: string;\n duration: number;\n shotCount: number;\n /** Up to 20 shot summaries sampled from the video */\n sampleSummaries: string[];\n /** Unique mood tags found across sampled shots */\n moods: string[];\n /** Unique scene tags found across sampled shots */\n scenes: string[];\n }>;\n}\n\n/** A dynamically generated slot list produced by LLM after seeing the library. */\nexport interface DynamicPlan {\n /** Theme/title for the video */\n theme: string;\n /** Which registered composition to use as the Remotion template */\n compositionId: string;\n /** Music style description */\n musicStyle: string;\n /** Slot list tailored to what actually exists in the library */\n slots: Array<{\n query: string;\n mood?: string;\n /** Any extra metadata the LLM wants to attach (caption, location, etc.) */\n extra?: Record<string, unknown>;\n }>;\n}\n\nexport interface AgentProvider {\n parseIntent(userRequest: string): Promise<ParsedIntent>;\n refineQueries(slots: Array<{ defaultQuery: string; mood?: string }>, theme: string): Promise<string[]>;\n /**\n * Given the actual shot summaries returned by ShotAI, produce per-clip\n * visual annotations that drive Remotion special-effects decisions.\n *\n * @param clips The resolved clip list (summary, optional keyframePath, slot extra)\n * @param compositionId Which composition is being rendered (determines which fields matter)\n */\n annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown> }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]>;\n /**\n * After seeing what's actually in the user's library, generate a tailored plan:\n * which composition to use and what slots to search for.\n */\n planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan>;\n}\n\n// ─── BaseAgent — all prompt logic lives here ──────────────────────────────────\n\nclass BaseAgent implements AgentProvider {\n constructor(private backend: LLMBackend) {}\n\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const system = `你是一个视频混剪规划助手。根据用户的混剪需求，选择最合适的合成风格并提取主题。\n只输出 JSON，格式如下：\n{\n \"theme\": \"视频主题名称（简短，用做片头片尾标题）\",\n \"compositionId\": \"CyberpunkCity | TravelVlog | MoodDriven\",\n \"musicStyleOverride\": \"（可选）英文音乐风格描述，留空则用默认\"\n}`;\n\n const user = `可选合成风格：\n${compositionList}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`Agent parseIntent: invalid JSON response: ${raw}`);\n }\n\n const meta = REGISTRY.find(m => m.id === parsed.compositionId) ?? REGISTRY[0];\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n composition: meta,\n musicStyleOverride: parsed.musicStyleOverride || undefined,\n };\n }\n\n async refineQueries(\n slots: Array<{ defaultQuery: string; mood?: string }>,\n theme: string,\n ): Promise<string[]> {\n const slotList = slots.map((s, i) =>\n ` ${i + 1}. default: \"${s.defaultQuery}\" mood: ${s.mood ?? 'any'}`\n ).join('\\n');\n\n const system = `Shot search query optimizer. For each slot, write a short English phrase (5–8 words) describing what the camera sees. Used for semantic vector search — concise beats verbose.\nOutput ONLY a JSON array of the same length. English only.`;\n\n const user = `Theme: ${theme}\nSlots:\n${slotList}\n\nOutput JSON array:`;\n\n const raw = await this.backend.callLLM(system, user);\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== slots.length) {\n throw new Error(`Expected array of length ${slots.length}`);\n }\n return arr as string[];\n } catch (e) {\n console.warn(` ⚠ Agent refineQueries failed (${(e as Error).message}), using defaults`);\n return slots.map(s => s.defaultQuery);\n }\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown> }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n const fieldHints: Record<string, string> = {\n SwitzerlandScenic: '\"tone\": \"warm\"|\"cool\" — warm for sunshine/meadow/sunset, cool for snow/glacier/water',\n SportsHighlight: '\"dramatic\": true|false — true for goals/key moments/emotional close-ups, false for regular play',\n NatureWild: '\"kenBurns\": \"zoom-in\"|\"zoom-out\"|\"pan-left\"|\"pan-right\" — choose based on subject (animal approaching=zoom-in, wide landscape=zoom-out, moving subject=pan)',\n };\n const relevantFields = fieldHints[compositionId] ?? '\"tone\": \"warm\"|\"cool\"';\n\n const clipList = clips.map((c, i) =>\n ` ${i + 1}. summary: \"${c.summary}\"${c.slotExtra?.caption ? ` caption_hint: \"${c.slotExtra.caption}\"` : ''}`\n ).join('\\n');\n\n const system = `你是一个视频剪辑视效助手。根据每个镜头的语义描述，为每个镜头生成精准的视觉特效注解。\n只输出 JSON 数组，每个元素只包含相关字段（不相关字段省略）。\n当前合成需要的字段：${relevantFields}\n输出格式示例：[{\"tone\":\"cool\"},{\"tone\":\"warm\",\"dramatic\":true},...]\n元素数量必须与输入完全相同。`;\n\n const user = `合成ID：${compositionId}\n镜头列表（共 ${clips.length} 个）：\n${clipList}\n\n请为每个镜头生成视效注解，输出 JSON 数组。`;\n\n const raw = await this.backend.callLLM(system, user);\n let annotations: ClipAnnotation[];\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== clips.length) {\n throw new Error(`Expected array of length ${clips.length}`);\n }\n annotations = arr as ClipAnnotation[];\n } catch (e) {\n console.warn(` ⚠ annotateClips failed (${(e as Error).message}), skipping annotations`);\n annotations = clips.map(() => ({}));\n }\n\n // Augment with keyframe brightness analysis (fixed algorithm, no LLM)\n for (let i = 0; i < clips.length; i++) {\n const kf = clips[i].keyframePath;\n if (kf) annotations[i].textBg = analyzeKeyframe(kf, compositionId);\n }\n return annotations;\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const libraryDesc = library.videos.map(v =>\n ` 【${v.name}】时长${Math.round(v.duration / 60)}分钟，${v.shotCount}个镜头\\n` +\n ` 情绪: ${v.moods.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 场景: ${v.scenes.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 镜头样本: ${v.sampleSummaries.slice(0, 8).map(s => `\"${s.slice(0, 60)}\"`).join(' / ')}`\n ).join('\\n\\n');\n\n const system = `你是一个视频混剪规划师。用户有一个视频素材库，你需要：\n1. 分析库里实际存在的素材内容\n2. 根据用户需求和库里有的内容，选择最合适的合成模板\n3. 生成专门针对这个库的镜头搜索槽位列表（不要用通用词，要用能在这个库里搜到东西的词）\n\n只输出 JSON，格式如下：\n{\n \"theme\": \"视频主题（简短标题）\",\n \"compositionId\": \"选择的合成ID\",\n \"musicStyle\": \"英文音乐风格描述\",\n \"slots\": [\n { \"query\": \"搜索词（针对这个库定制）\", \"mood\": \"calm|fast|slow|urban\", \"extra\": {} },\n ...\n ]\n}\nslots数量建议6-10个，与选择的合成模板匹配。`;\n\n const user = `可用合成模板：\n${compositionList}\n\n用户素材库内容：\n${libraryDesc}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`planFromLibrary: invalid JSON response: ${raw}`);\n }\n\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n compositionId: parsed.compositionId ?? REGISTRY[0].id,\n musicStyle: parsed.musicStyle ?? 'cinematic ambient',\n slots: Array.isArray(parsed.slots) ? parsed.slots : [],\n };\n }\n}\n\n// ─── Heuristic fallback (no API key) ──────────────────────────────────────────\n\nclass HeuristicAgent implements AgentProvider {\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const req = userRequest.toLowerCase();\n let meta: CompositionMeta;\n if (req.includes('赛博') || req.includes('cyberpunk') || req.includes('霓虹') || req.includes('夜景')) {\n meta = REGISTRY.find(m => m.id === 'CyberpunkCity')!;\n } else if (req.includes('旅行') || req.includes('vlog') || req.includes('城市名') || req.includes('地标')) {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n } else {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n }\n const theme = userRequest.split(/[,，。\\n]/)[0].slice(0, 20).trim();\n console.log(` ℹ HeuristicAgent: no API key — selected \"${meta.label}\" for \"${theme}\"`);\n return { theme, composition: meta };\n }\n\n async refineQueries(slots: Array<{ defaultQuery: string }>, _theme: string): Promise<string[]> {\n return slots.map(s => s.defaultQuery);\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n return clips.map(c => {\n const ann: ClipAnnotation = {};\n if (c.keyframePath) ann.textBg = analyzeKeyframe(c.keyframePath, compositionId);\n return ann;\n });\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n // Heuristic: reuse parseIntent to pick a composition, then use registry defaults as slots\n const intent = await this.parseIntent(userRequest);\n return {\n theme: intent.theme,\n compositionId: intent.composition.id,\n musicStyle: intent.musicStyleOverride ?? intent.composition.musicStyle,\n slots: intent.composition.shotSlots.map(s => ({\n query: s.query,\n mood: s.mood,\n extra: s.extra,\n })),\n };\n }\n}\n\n// ─── Factory ──────────────────────────────────────────────────────────────────\n\nexport type AgentProviderName = 'claude' | 'openai' | 'openai-compat' | 'none';\n\nexport interface CreateAgentOptions {\n provider: AgentProviderName;\n apiKey?: string;\n model?: string;\n /** Only for 'openai-compat': base URL of the compatible endpoint */\n baseUrl?: string;\n}\n\n/**\n * createAgent — instantiate the right backend and wrap it in BaseAgent.\n *\n * Examples:\n * createAgent({ provider: 'claude', apiKey: '...' })\n * createAgent({ provider: 'openai', apiKey: '...', model: 'gpt-4o' })\n * createAgent({ provider: 'openai-compat', apiKey: '...', model: 'deepseek-chat', baseUrl: 'https://api.deepseek.com' })\n * createAgent({ provider: 'openai-compat', apiKey: 'ollama', model: 'llama3', baseUrl: 'http://localhost:11434' })\n * createAgent({ provider: 'none' }) // keyword heuristics only\n */\nexport function createAgent(opts: CreateAgentOptions): AgentProvider;\n/** @deprecated Pass an options object instead */\nexport function createAgent(provider: AgentProviderName, apiKey?: string, model?: string): AgentProvider;\nexport function createAgent(\n optsOrProvider: CreateAgentOptions | AgentProviderName,\n apiKey?: string,\n model?: string,\n): AgentProvider {\n const opts: CreateAgentOptions =\n typeof optsOrProvider === 'string'\n ? { provider: optsOrProvider as AgentProviderName, apiKey, model }\n : optsOrProvider;\n\n if (opts.provider === 'claude') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=claude but ANTHROPIC_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new AnthropicBackend(opts.apiKey, opts.model ?? 'claude-sonnet-4-6'));\n }\n\n if (opts.provider === 'openai') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=openai but OPENAI_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'gpt-4o'));\n }\n\n if (opts.provider === 'openai-compat') {\n if (!opts.apiKey || !opts.baseUrl) {\n console.warn(' ⚠ openai-compat requires apiKey + baseUrl — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'default', opts.baseUrl));\n }\n\n return new HeuristicAgent();\n}\n"}, "2ea456ba5c2ccd9263f017f0334513d5fde62446": {"t": 1785763900.760606, "n": 17880, "h": "12fb67c0ae438ec2", "c": "/**\n * Agent Layer — LLM-driven intent parsing and shot selection\n *\n * Architecture:\n * - LLMBackend: thin interface for any LLM provider (one method: callLLM)\n * - BaseAgent: all prompt logic lives here, calls this.backend.callLLM()\n * - Provider backends: AnthropicBackend, OpenAIBackend, OpenAICompatBackend\n * - HeuristicAgent: keyword fallback, no API key required\n *\n * Provider selection (set AGENT_PROVIDER in .env):\n * 'claude' — Anthropic API (ANTHROPIC_API_KEY)\n * 'openai' — OpenAI API (OPENAI_API_KEY)\n * 'openai-compat' — Any OpenAI-compatible endpoint (OPENAI_COMPAT_BASE_URL + OPENAI_COMPAT_API_KEY)\n * Works with: Ollama, vLLM, Together, Groq, DeepSeek, Azure OpenAI, etc.\n * 'none' — Heuristic fallback (no API key required)\n */\nimport { REGISTRY, CompositionMeta } from './registry';\nimport { analyzeKeyframe } from './brightness';\n\nexport interface ParsedIntent {\n /** Main theme / title for the video */\n theme: string;\n /** Which composition to use */\n composition: CompositionMeta;\n /** Per-slot search query overrides (optional — falls back to registry defaults) */\n queryOverrides?: string[];\n /** Music style description override (optional) */\n musicStyleOverride?: string;\n}\n\n/**\n * Per-clip annotation generated by the LLM based on the actual shot summary.\n * These override or supplement the static `extra` fields from the registry.\n *\n * Fields are composition-aware: only relevant fields are used per composition.\n * - tone: 'warm' | 'cool' — color grade direction (SwitzerlandScenic)\n * - dramatic: boolean — slow-motion / emphasis clip (SportsHighlight)\n * - transition: 'fade' | 'slide' — per-clip transition override (future use)\n * - kenBurns: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right' (NatureWild)\n * - caption: string — override narrative caption if LLM improves it\n * - textBg: Record of region→'light'|'dark' — keyframe brightness analysis result.\n * Keys match TEXT_REGIONS in brightness.ts (e.g. \"NatureWild.caption\").\n * 'light' means the video background is bright → overlay needed.\n * Set by HeuristicAgent via keyframe pixel analysis; LLM agents can\n * also populate this from vision analysis.\n */\nexport interface ClipAnnotation {\n tone?: 'warm' | 'cool';\n dramatic?: boolean;\n transition?: 'fade' | 'slide';\n kenBurns?: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right';\n caption?: string;\n /** Per-region brightness result from keyframe analysis */\n textBg?: Record<string, 'light' | 'dark'>;\n}\n\n// ─── LLM Backend interface ────────────────────────────────────────────────────\n\n/**\n * Minimal interface any LLM provider must implement.\n * All prompt logic lives in BaseAgent; backends only handle HTTP transport.\n */\nexport interface LLMBackend {\n callLLM(system: string, user: string): Promise<string>;\n}\n\n// ─── Anthropic backend ────────────────────────────────────────────────────────\n\nexport class AnthropicBackend implements LLMBackend {\n constructor(\n private apiKey: string,\n private model = 'claude-sonnet-4-6',\n ) {}\n\n async callLLM(system: string, user: string): Promise<string> {\n const res = await fetch('https://api.anthropic.com/v1/messages', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'x-api-key': this.apiKey,\n 'anthropic-version': '2023-06-01',\n },\n body: JSON.stringify({\n model: this.model,\n max_tokens: 1024,\n system,\n messages: [{ role: 'user', content: user }],\n }),\n });\n if (!res.ok) throw new Error(`Anthropic API error ${res.status}: ${await res.text()}`);\n const data = await res.json() as any;\n return data.content[0].text as string;\n }\n}\n\n// ─── OpenAI-compatible backend (OpenAI, Ollama, vLLM, Groq, DeepSeek, …) ─────\n\nexport class OpenAICompatBackend implements LLMBackend {\n private baseUrl: string;\n\n constructor(\n private apiKey: string,\n private model: string,\n /** Base URL — defaults to OpenAI. Override for Ollama/vLLM/etc. */\n baseUrl?: string,\n ) {\n const base = (baseUrl ?? 'https://api.openai.com').replace(/\\/$/, '');\n // If caller already includes /v1 (e.g. OpenRouter), don't double-append it\n this.baseUrl = base.endsWith('/v1') ? base : `${base}/v1`;\n }\n\n async callLLM(system: string, user: string): Promise<string> {\n const res = await fetch(`${this.baseUrl}/chat/completions`, {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'Authorization': `Bearer ${this.apiKey}`,\n },\n body: JSON.stringify({\n model: this.model,\n messages: [\n { role: 'system', content: system },\n { role: 'user', content: user },\n ],\n max_tokens: 1024,\n }),\n });\n if (!res.ok) throw new Error(`LLM API error ${res.status}: ${await res.text()}`);\n const data = await res.json() as any;\n return data.choices[0].message.content as string;\n }\n}\n\n// ─── AgentProvider interface ──────────────────────────────────────────────────\n\n/** Compact summary of a user's media library, passed to LLM for dynamic planning. */\nexport interface LibrarySummary {\n videos: Array<{\n name: string;\n duration: number;\n shotCount: number;\n /** Up to 20 shot summaries sampled from the video */\n sampleSummaries: string[];\n /** Unique mood tags found across sampled shots */\n moods: string[];\n /** Unique scene tags found across sampled shots */\n scenes: string[];\n }>;\n}\n\n/** A dynamically generated slot list produced by LLM after seeing the library. */\nexport interface DynamicPlan {\n /** Theme/title for the video */\n theme: string;\n /** Which registered composition to use as the Remotion template */\n compositionId: string;\n /** Music style description */\n musicStyle: string;\n /** Slot list tailored to what actually exists in the library */\n slots: Array<{\n query: string;\n mood?: string;\n /** Any extra metadata the LLM wants to attach (caption, location, etc.) */\n extra?: Record<string, unknown>;\n }>;\n}\n\nexport interface AgentProvider {\n parseIntent(userRequest: string): Promise<ParsedIntent>;\n refineQueries(slots: Array<{ defaultQuery: string; mood?: string }>, theme: string): Promise<string[]>;\n /**\n * Given the actual shot summaries returned by ShotAI, produce per-clip\n * visual annotations that drive Remotion special-effects decisions.\n *\n * @param clips The resolved clip list (summary, optional keyframePath, slot extra)\n * @param compositionId Which composition is being rendered (determines which fields matter)\n */\n annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown> }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]>;\n /**\n * After seeing what's actually in the user's library, generate a tailored plan:\n * which composition to use and what slots to search for.\n */\n planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan>;\n}\n\n// ─── BaseAgent — all prompt logic lives here ──────────────────────────────────\n\nclass BaseAgent implements AgentProvider {\n constructor(private backend: LLMBackend) {}\n\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const system = `你是一个视频混剪规划助手。根据用户的混剪需求，选择最合适的合成风格并提取主题。\n只输出 JSON，格式如下：\n{\n \"theme\": \"视频主题名称（简短，用做片头片尾标题）\",\n \"compositionId\": \"CyberpunkCity | TravelVlog | MoodDriven\",\n \"musicStyleOverride\": \"（可选）英文音乐风格描述，留空则用默认\"\n}`;\n\n const user = `可选合成风格：\n${compositionList}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`Agent parseIntent: invalid JSON response: ${raw}`);\n }\n\n const meta = REGISTRY.find(m => m.id === parsed.compositionId) ?? REGISTRY[0];\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n composition: meta,\n musicStyleOverride: parsed.musicStyleOverride || undefined,\n };\n }\n\n async refineQueries(\n slots: Array<{ defaultQuery: string; mood?: string }>,\n theme: string,\n ): Promise<string[]> {\n const slotList = slots.map((s, i) =>\n ` ${i + 1}. 默认搜索词：「${s.defaultQuery}」 情绪：${s.mood ?? '不限'}`\n ).join('\\n');\n\n const system = `You are a shot search query optimizer. Given the user's video theme, generate natural language English descriptions for each shot slot — written as a single coherent sentence describing what the camera sees, as if narrating a film shot (e.g. \"a rain-slicked street at night with neon signs reflecting in puddles\"). These are used for semantic vector search.\nOutput ONLY a JSON array with the same number of elements as input, e.g.: [\"description 1\", \"description 2\", ...]\nIMPORTANT: Always output in English. Use complete sentences, not keyword phrases.`;\n\n const user = `Video theme: ${theme}\n\nShot slots (${slots.length} total):\n${slotList}\n\nOutput one natural language shot description per slot as a JSON array, preserving order.`;\n\n const raw = await this.backend.callLLM(system, user);\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== slots.length) {\n throw new Error(`Expected array of length ${slots.length}`);\n }\n return arr as string[];\n } catch (e) {\n console.warn(` ⚠ Agent refineQueries failed (${(e as Error).message}), using defaults`);\n return slots.map(s => s.defaultQuery);\n }\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown> }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n const fieldHints: Record<string, string> = {\n SwitzerlandScenic: '\"tone\": \"warm\"|\"cool\" — warm for sunshine/meadow/sunset, cool for snow/glacier/water',\n SportsHighlight: '\"dramatic\": true|false — true for goals/key moments/emotional close-ups, false for regular play',\n NatureWild: '\"kenBurns\": \"zoom-in\"|\"zoom-out\"|\"pan-left\"|\"pan-right\" — choose based on subject (animal approaching=zoom-in, wide landscape=zoom-out, moving subject=pan)',\n };\n const relevantFields = fieldHints[compositionId] ?? '\"tone\": \"warm\"|\"cool\"';\n\n const clipList = clips.map((c, i) =>\n ` ${i + 1}. summary: \"${c.summary}\"${c.slotExtra?.caption ? ` caption_hint: \"${c.slotExtra.caption}\"` : ''}`\n ).join('\\n');\n\n const system = `你是一个视频剪辑视效助手。根据每个镜头的语义描述，为每个镜头生成精准的视觉特效注解。\n只输出 JSON 数组，每个元素只包含相关字段（不相关字段省略）。\n当前合成需要的字段：${relevantFields}\n输出格式示例：[{\"tone\":\"cool\"},{\"tone\":\"warm\",\"dramatic\":true},...]\n元素数量必须与输入完全相同。`;\n\n const user = `合成ID：${compositionId}\n镜头列表（共 ${clips.length} 个）：\n${clipList}\n\n请为每个镜头生成视效注解，输出 JSON 数组。`;\n\n const raw = await this.backend.callLLM(system, user);\n let annotations: ClipAnnotation[];\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== clips.length) {\n throw new Error(`Expected array of length ${clips.length}`);\n }\n annotations = arr as ClipAnnotation[];\n } catch (e) {\n console.warn(` ⚠ annotateClips failed (${(e as Error).message}), skipping annotations`);\n annotations = clips.map(() => ({}));\n }\n\n // Augment with keyframe brightness analysis (fixed algorithm, no LLM)\n for (let i = 0; i < clips.length; i++) {\n const kf = clips[i].keyframePath;\n if (kf) annotations[i].textBg = analyzeKeyframe(kf, compositionId);\n }\n return annotations;\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const libraryDesc = library.videos.map(v =>\n ` 【${v.name}】时长${Math.round(v.duration / 60)}分钟，${v.shotCount}个镜头\\n` +\n ` 情绪: ${v.moods.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 场景: ${v.scenes.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 镜头样本: ${v.sampleSummaries.slice(0, 8).map(s => `\"${s.slice(0, 60)}\"`).join(' / ')}`\n ).join('\\n\\n');\n\n const system = `你是一个视频混剪规划师。用户有一个视频素材库，你需要：\n1. 分析库里实际存在的素材内容\n2. 根据用户需求和库里有的内容，选择最合适的合成模板\n3. 生成专门针对这个库的镜头搜索槽位列表（不要用通用词，要用能在这个库里搜到东西的词）\n\n只输出 JSON，格式如下：\n{\n \"theme\": \"视频主题（简短标题）\",\n \"compositionId\": \"选择的合成ID\",\n \"musicStyle\": \"英文音乐风格描述\",\n \"slots\": [\n { \"query\": \"搜索词（针对这个库定制）\", \"mood\": \"calm|fast|slow|urban\", \"extra\": {} },\n ...\n ]\n}\nslots数量建议6-10个，与选择的合成模板匹配。`;\n\n const user = `可用合成模板：\n${compositionList}\n\n用户素材库内容：\n${libraryDesc}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`planFromLibrary: invalid JSON response: ${raw}`);\n }\n\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n compositionId: parsed.compositionId ?? REGISTRY[0].id,\n musicStyle: parsed.musicStyle ?? 'cinematic ambient',\n slots: Array.isArray(parsed.slots) ? parsed.slots : [],\n };\n }\n}\n\n// ─── Heuristic fallback (no API key) ──────────────────────────────────────────\n\nclass HeuristicAgent implements AgentProvider {\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const req = userRequest.toLowerCase();\n let meta: CompositionMeta;\n if (req.includes('赛博') || req.includes('cyberpunk') || req.includes('霓虹') || req.includes('夜景')) {\n meta = REGISTRY.find(m => m.id === 'CyberpunkCity')!;\n } else if (req.includes('旅行') || req.includes('vlog') || req.includes('城市名') || req.includes('地标')) {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n } else {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n }\n const theme = userRequest.split(/[,，。\\n]/)[0].slice(0, 20).trim();\n console.log(` ℹ HeuristicAgent: no API key — selected \"${meta.label}\" for \"${theme}\"`);\n return { theme, composition: meta };\n }\n\n async refineQueries(slots: Array<{ defaultQuery: string }>, _theme: string): Promise<string[]> {\n return slots.map(s => s.defaultQuery);\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n return clips.map(c => {\n const ann: ClipAnnotation = {};\n if (c.keyframePath) ann.textBg = analyzeKeyframe(c.keyframePath, compositionId);\n return ann;\n });\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n // Heuristic: reuse parseIntent to pick a composition, then use registry defaults as slots\n const intent = await this.parseIntent(userRequest);\n return {\n theme: intent.theme,\n compositionId: intent.composition.id,\n musicStyle: intent.musicStyleOverride ?? intent.composition.musicStyle,\n slots: intent.composition.shotSlots.map(s => ({\n query: s.query,\n mood: s.mood,\n extra: s.extra,\n })),\n };\n }\n}\n\n// ─── Factory ──────────────────────────────────────────────────────────────────\n\nexport type AgentProviderName = 'claude' | 'openai' | 'openai-compat' | 'none';\n\nexport interface CreateAgentOptions {\n provider: AgentProviderName;\n apiKey?: string;\n model?: string;\n /** Only for 'openai-compat': base URL of the compatible endpoint */\n baseUrl?: string;\n}\n\n/**\n * createAgent — instantiate the right backend and wrap it in BaseAgent.\n *\n * Examples:\n * createAgent({ provider: 'claude', apiKey: '...' })\n * createAgent({ provider: 'openai', apiKey: '...', model: 'gpt-4o' })\n * createAgent({ provider: 'openai-compat', apiKey: '...', model: 'deepseek-chat', baseUrl: 'https://api.deepseek.com' })\n * createAgent({ provider: 'openai-compat', apiKey: 'ollama', model: 'llama3', baseUrl: 'http://localhost:11434' })\n * createAgent({ provider: 'none' }) // keyword heuristics only\n */\nexport function createAgent(opts: CreateAgentOptions): AgentProvider;\n/** @deprecated Pass an options object instead */\nexport function createAgent(provider: AgentProviderName, apiKey?: string, model?: string): AgentProvider;\nexport function createAgent(\n optsOrProvider: CreateAgentOptions | AgentProviderName,\n apiKey?: string,\n model?: string,\n): AgentProvider {\n const opts: CreateAgentOptions =\n typeof optsOrProvider === 'string'\n ? { provider: optsOrProvider as AgentProviderName, apiKey, model }\n : optsOrProvider;\n\n if (opts.provider === 'claude') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=claude but ANTHROPIC_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new AnthropicBackend(opts.apiKey, opts.model ?? 'claude-sonnet-4-6'));\n }\n\n if (opts.provider === 'openai') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=openai but OPENAI_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'gpt-4o'));\n }\n\n if (opts.provider === 'openai-compat') {\n if (!opts.apiKey || !opts.baseUrl) {\n console.warn(' ⚠ openai-compat requires apiKey + baseUrl — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'default', opts.baseUrl));\n }\n\n return new HeuristicAgent();\n}\n"}, "61a3b14c6e0e4286fad5a005c1bd097caa6a9903": {"t": 1785763902.115394, "n": 17374, "h": "075826bad9d688ef", "c": "/**\n * Agent Layer — LLM-driven intent parsing and shot selection\n *\n * Architecture:\n * - LLMBackend: thin interface for any LLM provider (one method: callLLM)\n * - BaseAgent: all prompt logic lives here, calls this.backend.callLLM()\n * - Provider backends: AnthropicBackend, OpenAIBackend, OpenAICompatBackend\n * - HeuristicAgent: keyword fallback, no API key required\n *\n * Provider selection (set AGENT_PROVIDER in .env):\n * 'claude' — Anthropic API (ANTHROPIC_API_KEY)\n * 'openai' — OpenAI API (OPENAI_API_KEY)\n * 'openai-compat' — Any OpenAI-compatible endpoint (OPENAI_COMPAT_BASE_URL + OPENAI_COMPAT_API_KEY)\n * Works with: Ollama, vLLM, Together, Groq, DeepSeek, Azure OpenAI, etc.\n * 'none' — Heuristic fallback (no API key required)\n */\nimport { REGISTRY, CompositionMeta } from './registry';\nimport { analyzeKeyframe } from './brightness';\n\nexport interface ParsedIntent {\n /** Main theme / title for the video */\n theme: string;\n /** Which composition to use */\n composition: CompositionMeta;\n /** Per-slot search query overrides (optional — falls back to registry defaults) */\n queryOverrides?: string[];\n /** Music style description override (optional) */\n musicStyleOverride?: string;\n}\n\n/**\n * Per-clip annotation generated by the LLM based on the actual shot summary.\n * These override or supplement the static `extra` fields from the registry.\n *\n * Fields are composition-aware: only relevant fields are used per composition.\n * - tone: 'warm' | 'cool' — color grade direction (SwitzerlandScenic)\n * - dramatic: boolean — slow-motion / emphasis clip (SportsHighlight)\n * - transition: 'fade' | 'slide' — per-clip transition override (future use)\n * - kenBurns: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right' (NatureWild)\n * - caption: string — override narrative caption if LLM improves it\n * - textBg: Record of region→'light'|'dark' — keyframe brightness analysis result.\n * Keys match TEXT_REGIONS in brightness.ts (e.g. \"NatureWild.caption\").\n * 'light' means the video background is bright → overlay needed.\n * Set by HeuristicAgent via keyframe pixel analysis; LLM agents can\n * also populate this from vision analysis.\n */\nexport interface ClipAnnotation {\n tone?: 'warm' | 'cool';\n dramatic?: boolean;\n transition?: 'fade' | 'slide';\n kenBurns?: 'zoom-in' | 'zoom-out' | 'pan-left' | 'pan-right';\n caption?: string;\n /** Per-region brightness result from keyframe analysis */\n textBg?: Record<string, 'light' | 'dark'>;\n}\n\n// ─── LLM Backend interface ────────────────────────────────────────────────────\n\n/**\n * Minimal interface any LLM provider must implement.\n * All prompt logic lives in BaseAgent; backends only handle HTTP transport.\n */\nexport interface LLMBackend {\n callLLM(system: string, user: string): Promise<string>;\n}\n\n// ─── Anthropic backend ────────────────────────────────────────────────────────\n\nexport class AnthropicBackend implements LLMBackend {\n constructor(\n private apiKey: string,\n private model = 'claude-sonnet-4-6',\n ) {}\n\n async callLLM(system: string, user: string): Promise<string> {\n const res = await fetch('https://api.anthropic.com/v1/messages', {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'x-api-key': this.apiKey,\n 'anthropic-version': '2023-06-01',\n },\n body: JSON.stringify({\n model: this.model,\n max_tokens: 1024,\n system,\n messages: [{ role: 'user', content: user }],\n }),\n });\n if (!res.ok) throw new Error(`Anthropic API error ${res.status}: ${await res.text()}`);\n const data = await res.json() as any;\n return data.content[0].text as string;\n }\n}\n\n// ─── OpenAI-compatible backend (OpenAI, Ollama, vLLM, Groq, DeepSeek, …) ─────\n\nexport class OpenAICompatBackend implements LLMBackend {\n private baseUrl: string;\n\n constructor(\n private apiKey: string,\n private model: string,\n /** Base URL — defaults to OpenAI. Override for Ollama/vLLM/etc. */\n baseUrl?: string,\n ) {\n const base = (baseUrl ?? 'https://api.openai.com').replace(/\\/$/, '');\n // If caller already includes /v1 (e.g. OpenRouter), don't double-append it\n this.baseUrl = base.endsWith('/v1') ? base : `${base}/v1`;\n }\n\n async callLLM(system: string, user: string): Promise<string> {\n const res = await fetch(`${this.baseUrl}/chat/completions`, {\n method: 'POST',\n headers: {\n 'Content-Type': 'application/json',\n 'Authorization': `Bearer ${this.apiKey}`,\n },\n body: JSON.stringify({\n model: this.model,\n messages: [\n { role: 'system', content: system },\n { role: 'user', content: user },\n ],\n max_tokens: 1024,\n }),\n });\n if (!res.ok) throw new Error(`LLM API error ${res.status}: ${await res.text()}`);\n const data = await res.json() as any;\n return data.choices[0].message.content as string;\n }\n}\n\n// ─── AgentProvider interface ──────────────────────────────────────────────────\n\n/** Compact summary of a user's media library, passed to LLM for dynamic planning. */\nexport interface LibrarySummary {\n videos: Array<{\n name: string;\n duration: number;\n shotCount: number;\n /** Up to 20 shot summaries sampled from the video */\n sampleSummaries: string[];\n /** Unique mood tags found across sampled shots */\n moods: string[];\n /** Unique scene tags found across sampled shots */\n scenes: string[];\n }>;\n}\n\n/** A dynamically generated slot list produced by LLM after seeing the library. */\nexport interface DynamicPlan {\n /** Theme/title for the video */\n theme: string;\n /** Which registered composition to use as the Remotion template */\n compositionId: string;\n /** Music style description */\n musicStyle: string;\n /** Slot list tailored to what actually exists in the library */\n slots: Array<{\n query: string;\n mood?: string;\n /** Any extra metadata the LLM wants to attach (caption, location, etc.) */\n extra?: Record<string, unknown>;\n }>;\n}\n\nexport interface AgentProvider {\n parseIntent(userRequest: string): Promise<ParsedIntent>;\n refineQueries(slots: Array<{ defaultQuery: string; mood?: string }>, theme: string): Promise<string[]>;\n /**\n * Given the actual shot summaries returned by ShotAI, produce per-clip\n * visual annotations that drive Remotion special-effects decisions.\n *\n * @param clips The resolved clip list (summary, optional keyframePath, slot extra)\n * @param compositionId Which composition is being rendered (determines which fields matter)\n */\n annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown> }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]>;\n /**\n * After seeing what's actually in the user's library, generate a tailored plan:\n * which composition to use and what slots to search for.\n */\n planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan>;\n}\n\n// ─── BaseAgent — all prompt logic lives here ──────────────────────────────────\n\nclass BaseAgent implements AgentProvider {\n constructor(private backend: LLMBackend) {}\n\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const system = `你是一个视频混剪规划助手。根据用户的混剪需求，选择最合适的合成风格并提取主题。\n只输出 JSON，格式如下：\n{\n \"theme\": \"视频主题名称（简短，用做片头片尾标题）\",\n \"compositionId\": \"CyberpunkCity | TravelVlog | MoodDriven\",\n \"musicStyleOverride\": \"（可选）英文音乐风格描述，留空则用默认\"\n}`;\n\n const user = `可选合成风格：\n${compositionList}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`Agent parseIntent: invalid JSON response: ${raw}`);\n }\n\n const meta = REGISTRY.find(m => m.id === parsed.compositionId) ?? REGISTRY[0];\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n composition: meta,\n musicStyleOverride: parsed.musicStyleOverride || undefined,\n };\n }\n\n async refineQueries(\n slots: Array<{ defaultQuery: string; mood?: string }>,\n theme: string,\n ): Promise<string[]> {\n const slotList = slots.map((s, i) =>\n ` ${i + 1}. 默认搜索词：「${s.defaultQuery}」 情绪：${s.mood ?? '不限'}`\n ).join('\\n');\n\n const system = `你是一个镜头搜索词优化助手。根据用户的视频主题，为每个镜头槽位生成更精准的中文语义搜索词（用于向量数据库搜索，简洁有画面感，不超过15字）。\n只输出 JSON 数组，元素数量必须与输入完全相同，例如：[\"搜索词1\", \"搜索词2\", ...]`;\n\n const user = `视频主题：${theme}\n\n镜头槽位（共 ${slots.length} 个）：\n${slotList}\n\n请为每个槽位生成优化后的搜索词，保持顺序输出 JSON 数组。`;\n\n const raw = await this.backend.callLLM(system, user);\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== slots.length) {\n throw new Error(`Expected array of length ${slots.length}`);\n }\n return arr as string[];\n } catch (e) {\n console.warn(` ⚠ Agent refineQueries failed (${(e as Error).message}), using defaults`);\n return slots.map(s => s.defaultQuery);\n }\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string; slotExtra?: Record<string, unknown> }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n const fieldHints: Record<string, string> = {\n SwitzerlandScenic: '\"tone\": \"warm\"|\"cool\" — warm for sunshine/meadow/sunset, cool for snow/glacier/water',\n SportsHighlight: '\"dramatic\": true|false — true for goals/key moments/emotional close-ups, false for regular play',\n NatureWild: '\"kenBurns\": \"zoom-in\"|\"zoom-out\"|\"pan-left\"|\"pan-right\" — choose based on subject (animal approaching=zoom-in, wide landscape=zoom-out, moving subject=pan)',\n };\n const relevantFields = fieldHints[compositionId] ?? '\"tone\": \"warm\"|\"cool\"';\n\n const clipList = clips.map((c, i) =>\n ` ${i + 1}. summary: \"${c.summary}\"${c.slotExtra?.caption ? ` caption_hint: \"${c.slotExtra.caption}\"` : ''}`\n ).join('\\n');\n\n const system = `你是一个视频剪辑视效助手。根据每个镜头的语义描述，为每个镜头生成精准的视觉特效注解。\n只输出 JSON 数组，每个元素只包含相关字段（不相关字段省略）。\n当前合成需要的字段：${relevantFields}\n输出格式示例：[{\"tone\":\"cool\"},{\"tone\":\"warm\",\"dramatic\":true},...]\n元素数量必须与输入完全相同。`;\n\n const user = `合成ID：${compositionId}\n镜头列表（共 ${clips.length} 个）：\n${clipList}\n\n请为每个镜头生成视效注解，输出 JSON 数组。`;\n\n const raw = await this.backend.callLLM(system, user);\n let annotations: ClipAnnotation[];\n try {\n const arr = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n if (!Array.isArray(arr) || arr.length !== clips.length) {\n throw new Error(`Expected array of length ${clips.length}`);\n }\n annotations = arr as ClipAnnotation[];\n } catch (e) {\n console.warn(` ⚠ annotateClips failed (${(e as Error).message}), skipping annotations`);\n annotations = clips.map(() => ({}));\n }\n\n // Augment with keyframe brightness analysis (fixed algorithm, no LLM)\n for (let i = 0; i < clips.length; i++) {\n const kf = clips[i].keyframePath;\n if (kf) annotations[i].textBg = analyzeKeyframe(kf, compositionId);\n }\n return annotations;\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n const compositionList = REGISTRY.map(m =>\n ` - ${m.id}: ${m.label} — ${m.description}`\n ).join('\\n');\n\n const libraryDesc = library.videos.map(v =>\n ` 【${v.name}】时长${Math.round(v.duration / 60)}分钟，${v.shotCount}个镜头\\n` +\n ` 情绪: ${v.moods.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 场景: ${v.scenes.slice(0, 6).join(', ') || '未知'}\\n` +\n ` 镜头样本: ${v.sampleSummaries.slice(0, 8).map(s => `\"${s.slice(0, 60)}\"`).join(' / ')}`\n ).join('\\n\\n');\n\n const system = `你是一个视频混剪规划师。用户有一个视频素材库，你需要：\n1. 分析库里实际存在的素材内容\n2. 根据用户需求和库里有的内容，选择最合适的合成模板\n3. 生成专门针对这个库的镜头搜索槽位列表（不要用通用词，要用能在这个库里搜到东西的词）\n\n只输出 JSON，格式如下：\n{\n \"theme\": \"视频主题（简短标题）\",\n \"compositionId\": \"选择的合成ID\",\n \"musicStyle\": \"英文音乐风格描述\",\n \"slots\": [\n { \"query\": \"搜索词（针对这个库定制）\", \"mood\": \"calm|fast|slow|urban\", \"extra\": {} },\n ...\n ]\n}\nslots数量建议6-10个，与选择的合成模板匹配。`;\n\n const user = `可用合成模板：\n${compositionList}\n\n用户素材库内容：\n${libraryDesc}\n\n用户需求：${userRequest}`;\n\n const raw = await this.backend.callLLM(system, user);\n let parsed: any;\n try {\n parsed = JSON.parse(raw.replace(/```json\\n?|\\n?```/g, '').trim());\n } catch {\n throw new Error(`planFromLibrary: invalid JSON response: ${raw}`);\n }\n\n return {\n theme: parsed.theme ?? userRequest.slice(0, 30),\n compositionId: parsed.compositionId ?? REGISTRY[0].id,\n musicStyle: parsed.musicStyle ?? 'cinematic ambient',\n slots: Array.isArray(parsed.slots) ? parsed.slots : [],\n };\n }\n}\n\n// ─── Heuristic fallback (no API key) ──────────────────────────────────────────\n\nclass HeuristicAgent implements AgentProvider {\n async parseIntent(userRequest: string): Promise<ParsedIntent> {\n const req = userRequest.toLowerCase();\n let meta: CompositionMeta;\n if (req.includes('赛博') || req.includes('cyberpunk') || req.includes('霓虹') || req.includes('夜景')) {\n meta = REGISTRY.find(m => m.id === 'CyberpunkCity')!;\n } else if (req.includes('旅行') || req.includes('vlog') || req.includes('城市名') || req.includes('地标')) {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n } else {\n meta = REGISTRY.find(m => m.id === 'TravelVlog')!;\n }\n const theme = userRequest.split(/[,，。\\n]/)[0].slice(0, 20).trim();\n console.log(` ℹ HeuristicAgent: no API key — selected \"${meta.label}\" for \"${theme}\"`);\n return { theme, composition: meta };\n }\n\n async refineQueries(slots: Array<{ defaultQuery: string }>, _theme: string): Promise<string[]> {\n return slots.map(s => s.defaultQuery);\n }\n\n async annotateClips(\n clips: Array<{ summary: string; keyframePath?: string }>,\n compositionId: string,\n ): Promise<ClipAnnotation[]> {\n return clips.map(c => {\n const ann: ClipAnnotation = {};\n if (c.keyframePath) ann.textBg = analyzeKeyframe(c.keyframePath, compositionId);\n return ann;\n });\n }\n\n async planFromLibrary(userRequest: string, library: LibrarySummary): Promise<DynamicPlan> {\n // Heuristic: reuse parseIntent to pick a composition, then use registry defaults as slots\n const intent = await this.parseIntent(userRequest);\n return {\n theme: intent.theme,\n compositionId: intent.composition.id,\n musicStyle: intent.musicStyleOverride ?? intent.composition.musicStyle,\n slots: intent.composition.shotSlots.map(s => ({\n query: s.query,\n mood: s.mood,\n extra: s.extra,\n })),\n };\n }\n}\n\n// ─── Factory ──────────────────────────────────────────────────────────────────\n\nexport type AgentProviderName = 'claude' | 'openai' | 'openai-compat' | 'none';\n\nexport interface CreateAgentOptions {\n provider: AgentProviderName;\n apiKey?: string;\n model?: string;\n /** Only for 'openai-compat': base URL of the compatible endpoint */\n baseUrl?: string;\n}\n\n/**\n * createAgent — instantiate the right backend and wrap it in BaseAgent.\n *\n * Examples:\n * createAgent({ provider: 'claude', apiKey: '...' })\n * createAgent({ provider: 'openai', apiKey: '...', model: 'gpt-4o' })\n * createAgent({ provider: 'openai-compat', apiKey: '...', model: 'deepseek-chat', baseUrl: 'https://api.deepseek.com' })\n * createAgent({ provider: 'openai-compat', apiKey: 'ollama', model: 'llama3', baseUrl: 'http://localhost:11434' })\n * createAgent({ provider: 'none' }) // keyword heuristics only\n */\nexport function createAgent(opts: CreateAgentOptions): AgentProvider;\n/** @deprecated Pass an options object instead */\nexport function createAgent(provider: AgentProviderName, apiKey?: string, model?: string): AgentProvider;\nexport function createAgent(\n optsOrProvider: CreateAgentOptions | AgentProviderName,\n apiKey?: string,\n model?: string,\n): AgentProvider {\n const opts: CreateAgentOptions =\n typeof optsOrProvider === 'string'\n ? { provider: optsOrProvider as AgentProviderName, apiKey, model }\n : optsOrProvider;\n\n if (opts.provider === 'claude') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=claude but ANTHROPIC_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new AnthropicBackend(opts.apiKey, opts.model ?? 'claude-sonnet-4-6'));\n }\n\n if (opts.provider === 'openai') {\n if (!opts.apiKey) {\n console.warn(' ⚠ AGENT_PROVIDER=openai but OPENAI_API_KEY is not set — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'gpt-4o'));\n }\n\n if (opts.provider === 'openai-compat') {\n if (!opts.apiKey || !opts.baseUrl) {\n console.warn(' ⚠ openai-compat requires apiKey + baseUrl — falling back to heuristic');\n return new HeuristicAgent();\n }\n return new BaseAgent(new OpenAICompatBackend(opts.apiKey, opts.model ?? 'default', opts.baseUrl));\n }\n\n return new HeuristicAgent();\n}\n"}}, ".env.example": {"7ca00a40057e92da07fc40a8e3eae4c7038a8387": {"t": 1785747426.741257, "n": 1808, "h": "057a58683bd76d62", "c": "# ── ShotAI MCP (Required) ──────────────────────────────────────────────────────\nSHOTAI_URL=http://127.0.0.1:23817 # Default ShotAI MCP server address\nSHOTAI_TOKEN= # Token from ShotAI → Settings → MCP Server\n\n# ── LLM Agent (Optional — falls back to heuristic mode if AGENT_PROVIDER=none) ──\nAGENT_PROVIDER=none # 'claude' | 'openai' | 'openai-compat' | 'none'\n\n# Claude (Anthropic)\n# AGENT_PROVIDER=claude\n# ANTHROPIC_API_KEY=sk-ant-...\n# AGENT_MODEL=claude-sonnet-4-6\n\n# OpenAI\n# AGENT_PROVIDER=openai\n# OPENAI_API_KEY=sk-...\n# AGENT_MODEL=gpt-4o\n\n# OpenRouter (recommended for China / multi-provider)\n# AGENT_PROVIDER=openai-compat\n# OPENAI_COMPAT_BASE_URL=https://openrouter.ai/api/v1\n# OPENAI_COMPAT_API_KEY=sk-or-v1-...\n# AGENT_MODEL=deepseek/deepseek-chat-v3-0324\n\n# DeepSeek (direct)\n# AGENT_PROVIDER=openai-compat\n# OPENAI_COMPAT_BASE_URL=https://api.deepseek.com/v1\n# OPENAI_COMPAT_API_KEY=sk-...\n# AGENT_MODEL=deepseek-chat\n\n# Ollama (local, no API key)\n# AGENT_PROVIDER=openai-compat\n# OPENAI_COMPAT_BASE_URL=http://localhost:11434/v1\n# OPENAI_COMPAT_API_KEY=ollama\n# AGENT_MODEL=llama3.1\n\n# ── Music (Optional) ───────────────────────────────────────────────────────────\n# BGM_PATH=/path/to/music.mp3 # Local MP3; if set, skips YouTube search\n\n# ── Output ─────────────────────────────────────────────────────────────────────\nOUTPUT_DIR=./output # Final video output directory\nFILE_PORT=8080 # Localhost-only HTTP port for serving clips to Remotion\n\n# ── Quality ────────────────────────────────────────────────────────────────────\nMIN_SCORE=0.5 # Shot quality threshold 0–1 (recommended: 0.5)\nMAX_RETRIES=1 # ShotAI search retry attempts on low-score result\n"}, "7d4cdc2e305682287cdc6a6e8701e2afb5a6d52d": {"t": 1785763890.497091, "n": 1808, "h": "057a58683bd76d62", "c": "# ── ShotAI MCP (Required) ──────────────────────────────────────────────────────\nSHOTAI_URL=http://127.0.0.1:23817 # Default ShotAI MCP server address\nSHOTAI_TOKEN= # Token from ShotAI → Settings → MCP Server\n\n# ── LLM Agent (Optional — falls back to heuristic mode if AGENT_PROVIDER=none) ──\nAGENT_PROVIDER=none # 'claude' | 'openai' | 'openai-compat' | 'none'\n\n# Claude (Anthropic)\n# AGENT_PROVIDER=claude\n# ANTHROPIC_API_KEY=sk-ant-...\n# AGENT_MODEL=claude-sonnet-4-6\n\n# OpenAI\n# AGENT_PROVIDER=openai\n# OPENAI_API_KEY=sk-...\n# AGENT_MODEL=gpt-4o\n\n# OpenRouter (recommended for China / multi-provider)\n# AGENT_PROVIDER=openai-compat\n# OPENAI_COMPAT_BASE_URL=https://openrouter.ai/api/v1\n# OPENAI_COMPAT_API_KEY=sk-or-v1-...\n# AGENT_MODEL=deepseek/deepseek-chat-v3-0324\n\n# DeepSeek (direct)\n# AGENT_PROVIDER=openai-compat\n# OPENAI_COMPAT_BASE_URL=https://api.deepseek.com/v1\n# OPENAI_COMPAT_API_KEY=sk-...\n# AGENT_MODEL=deepseek-chat\n\n# Ollama (local, no API key)\n# AGENT_PROVIDER=openai-compat\n# OPENAI_COMPAT_BASE_URL=http://localhost:11434/v1\n# OPENAI_COMPAT_API_KEY=ollama\n# AGENT_MODEL=llama3.1\n\n# ── Music (Optional) ───────────────────────────────────────────────────────────\n# BGM_PATH=/path/to/music.mp3 # Local MP3; if set, skips YouTube search\n\n# ── Output ─────────────────────────────────────────────────────────────────────\nOUTPUT_DIR=./output # Final video output directory\nFILE_PORT=8080 # Localhost-only HTTP port for serving clips to Remotion\n\n# ── Quality ────────────────────────────────────────────────────────────────────\nMIN_SCORE=0.5 # Shot quality threshold 0–1 (recommended: 0.5)\nMAX_RETRIES=1 # ShotAI search retry attempts on low-score result\n"}, "61a3b14c6e0e4286fad5a005c1bd097caa6a9903": {"t": 1785763892.7983384, "n": 1793, "h": "1deae551a8ebfdeb", "c": "# ── ShotAI MCP (Required) ──────────────────────────────────────────────────────\nSHOTAI_URL=http://127.0.0.1:23817 # Default ShotAI MCP server address\nSHOTAI_TOKEN= # Token from ShotAI → Settings → MCP Server\n\n# ── LLM Agent (Optional — falls back to heuristic mode if AGENT_PROVIDER=none) ──\nAGENT_PROVIDER=none # 'claude' | 'openai' | 'openai-compat' | 'none'\n\n# Claude (Anthropic)\n# AGENT_PROVIDER=claude\n# ANTHROPIC_API_KEY=sk-ant-...\n# AGENT_MODEL=claude-sonnet-4-6\n\n# OpenAI\n# AGENT_PROVIDER=openai\n# OPENAI_API_KEY=sk-...\n# AGENT_MODEL=gpt-4o\n\n# OpenRouter (recommended for China / multi-provider)\n# AGENT_PROVIDER=openai-compat\n# OPENAI_COMPAT_BASE_URL=https://openrouter.ai/api/v1\n# OPENAI_COMPAT_API_KEY=sk-or-v1-...\n# AGENT_MODEL=deepseek/deepseek-chat-v3-0324\n\n# DeepSeek (direct)\n# AGENT_PROVIDER=openai-compat\n# OPENAI_COMPAT_BASE_URL=https://api.deepseek.com/v1\n# OPENAI_COMPAT_API_KEY=sk-...\n# AGENT_MODEL=deepseek-chat\n\n# Ollama (local, no API key)\n# AGENT_PROVIDER=openai-compat\n# OPENAI_COMPAT_BASE_URL=http://localhost:11434/v1\n# OPENAI_COMPAT_API_KEY=ollama\n# AGENT_MODEL=llama3.1\n\n# ── Music (Optional) ───────────────────────────────────────────────────────────\n# BGM_PATH=/path/to/music.mp3 # Local MP3; if set, skips YouTube search\n\n# ── Output ─────────────────────────────────────────────────────────────────────\nOUTPUT_DIR=./output # Final video output directory\nFILE_PORT=9877 # HTTP port for serving clips to Remotion\n\n# ── Quality ────────────────────────────────────────────────────────────────────\nMIN_SCORE=0.5 # Shot quality threshold 0–1 (recommended: 0.5)\nMAX_RETRIES=1 # ShotAI search retry attempts on low-score result\n"}}}} |