1 line
8.3 KiB
JSON
1 line
8.3 KiB
JSON
{"repo": "alfaoz/opencode-see-image", "files": {"README.md": {"7fbc1c565f86cdbd58aeb09a064177833d3ff9c9": {"t": 1785595229.5547025, "n": 8072, "h": "968fd7cc544a9853", "c": "# opencode-see-image\n\ngive non-vision opencode models the ability to see images and screenshots by routing them to a vision-capable model.\n\nwhen a user attaches a screenshot to a text-only model, opencode rejects it with an error. This plugin intercepts that flow by registering a `see_image` tool that sends the image to a vision model and returns a textual description the primary model can reason about.\n\nwhen the active model is already vision-capable (its `capabilities.input.image` is true), the plugin stays out of the way: the see_image instructions are not injected, and a stray `see_image` call returns a harmless \"you already saw this natively\" note instead of an error. beware that this might lead to issues while switching models mid session, so a clean session might fix most of your problems (if any).\n\n## install\n\n**one command (recommended):**\n```bash\nopencode plugin opencode-see-image --global\n```\nThis installs the package and adds it to your config. Then restart opencode.\n\n**edit config manually:**\n\nAdd the plugin to your opencode config:\n\n```jsonc\n// ~/.config/opencode/opencode.jsonc\n{\n \"$schema\": \"https://opencode.ai/config.json\",\n \"plugin\": [\"opencode-see-image\"]\n}\n```\nThen restart opencode.\n\n## install via your agent (for some reason?)\n\nask your agent:\n```\ninstall the opencode-see-image plugin\n```\nit'll run `opencode plugin opencode-see-image --global` and tell you to restart. \n\n## prerequisites\n\nyou need a connected vision-capable provider. The plugin auto-detects whichever you have connected, **either of these work**:\n\n### free (OpenCode Zen)\n1. run `/connect` in opencode\n2. select **opencode** (OpenCode Zen)\n3. paste your API key from [opencode.ai/auth](https://opencode.ai/auth)\n\nthe plugin uses **mimo-v2.5-free** automatically — if you don't have an OpenCode Go sub, it skips the paid model entirely (no errors) and routes to the free model.\n\n### paid, w/ OpenCode Go\n1. run `/connect` in opencode\n2. select **opencode-go**\n3. paste your API key from [opencode.ai/auth](https://opencode.ai/auth)\n\nthe plugin prefers **minimax-m3** via opencode-go when available.\n\n### paid, w/ another provider\n\nset the `SEE_IMAGE_*` env vars to point at any Anthropic-Messages-compatible endpoint. see [Configuration](#configuration) below.\n\n**the resolve order:** explicit `SEE_IMAGE_API_KEY` env → configured `SEE_IMAGE_PROVIDER` → `opencode-go` (MiniMax M3) *if you have a Go sub connected* → `opencode` (mimo-v2.5-free). If no Go sub is connected, `opencode-go` is skipped so free users never hit a \"model not found\" error.\n\n## how the _eye surgery_ works\n\n```\nuser attaches screenshot\n |\n v\nopencode rejects it: 'this model does not support image input'\n | (the model only sees the filename, or nothing at all)\n v\nplugin's system-prompt instructions tell the model to call see_image\n |\n v\nsee_image tool resolves the image:\n 1. asks the opencode server for the session's attachments (exact name,\n then unicode-normalized name, then most recent image)\n 2. falls back to reading opencode's SQLite DB directly (this session,\n then any session)\n 3. falls back to a filesystem search\n 4. if the name matched nothing but the session has an image, uses the\n most recent one anyway\n |\n v\nsends the image to the vision model and returns the textual description\n |\n v\nprimary model answers using the description\n```\n\nif everything misses, the error message lists the images actually attached to the session so the model can retry with a correct filename.\n\n## the `see_image` tool\n\nthe plugin registers a `see_image` tool with two arguments:\n\n| arg | type | required? | description |\n|---|---|---|---|\n| `filePath` | string | n | path to the image. Absolute path, or a bare filename like `\"Screenshot 2026-06-18 at 17.32.24.png\"` to auto-locate. Omit (or pass `\"latest\"`) to use the most recent image attached to the conversation. |\n| `question` | string | n | a specific question about the image. Defaults to a general detailed description. Use this to focus on a particular detail (e.g. `\"What error is shown in the terminal?\"`). |\n\nyour model calls this tool automatically when you attach a screenshot, you don't need to do anything special. The `question` arg is optional; the model uses it when you ask something specific about the image.\n\n## configuration\n\nall settings are env-var overrides. The plugin uses opencode's SDK client by default (handles auth automatically). Set `SEE_IMAGE_API_KEY` to bypass the SDK and call an HTTP endpoint directly.\n\n| env var | default | description |\n|---|---|---|\n| `SEE_IMAGE_MODEL` | `minimax-m3` | Vision model ID |\n| `SEE_IMAGE_PROVIDER` | `opencode-go` | Provider ID for SDK routing |\n| `SEE_IMAGE_API_KEY` | _(uses SDK)_ | Bypass SDK, call HTTP endpoint directly |\n| `SEE_IMAGE_ENDPOINT` | `https://opencode.ai/zen/go/v1/messages` | HTTP endpoint (only used if `SEE_IMAGE_API_KEY` is set) |\n| `SEE_IMAGE_API_VERSION` | `2023-06-01` | `anthropic-version` header (HTTP mode only) |\n| `SEE_IMAGE_USER_AGENT` | _(Chrome UA)_ | User-Agent header (HTTP mode only) |\n| `SEE_IMAGE_TIMEOUT` | `30000` | Per-candidate timeout in ms. Prevents hanging on slow models. |\n\n### using a different vision model\n\nany Anthropic-Messages-compatible endpoint works. for example, to use a direct MiniMax key:\n\n```bash\nexport SEE_IMAGE_ENDPOINT=\"https://api.minimax.io/v1/messages\"\nexport SEE_IMAGE_MODEL=\"minimax-m3\"\nexport SEE_IMAGE_API_KEY=\"your-minimax-key\"\n```\n\nto use a different opencode-go model (e.g. Kimi K2.7):\n\n```bash\nexport SEE_IMAGE_MODEL=\"kimi-k2.7-code\"\n```\n\n### verified vision-capable models\n\n**Free (OpenCode Zen):**\n\n| model | Notes |\n|---|---|\n| `mimo-v2.5-free` | free. may be a bit slow. default fallback when only Zen is connected (routed via CLI). |\n| `big-pickle` | for some reason, big pickle works as an image capable model when called through the sdk w/ an active opencode go sub. |\n\n**paid (OpenCode Go):**\n\n| model | speed | notes |\n|---|---|---|\n| `minimax-m3` | ~3000ms | default. fast, clean, and accurate. |\n| `kimi-k2.7-code` | ~7000ms | clean and accurate. |\n| `kimi-k2.6` | ~12000ms | accurate but slow. |\n| `qwen3.7-plus` | ~15000ms | slow, spends a bit more tokens because of thinking. |\n\n## updating\n\n**auto-update (built in):** uses the opencode-plugin-update-kit and shows a toast: *\"opencode-see-image updated to X.Y.Z, restart opencode to apply\"*. You just need to restart opencode to load the new version.\n\n**manual update**:\n```bash\nopencode plugin opencode-see-image --force --global\n```\nthen restart opencode.\n\n**pin a version** in your config to opt out of auto-updates:\n```jsonc\n\"plugin\": [\"opencode-see-image@0.4.2\"]\n```\n\n## platform support\n\nworks on **macOS, Windows, and Linux**, in both the **CLI/TUI** (Bun runtime) and the **desktop app** (whose server runs on Node inside Electron). The package ships compiled JS and avoids Bun-only APIs at load time, so it loads on either runtime. Attachment lookup goes through the opencode server API first (works everywhere, including remote workspaces), with a direct SQLite read and a filesystem search as fallbacks.\n\n## file search locations\n\nwhen opencode rejects an image attachment, the model only receives a bare filename. `see_image` first checks the opencode DB (cross-platform), then falls back to searching these filesystem locations, in order:\n\n**macOS**\n1. `$TMPDIR/TemporaryItems/NSIRD_screencaptureui_*/` (where macOS stashes dragged screenshots)\n2. `$TMPDIR/TemporaryItems/`\n3. `~/Desktop`, `~/Downloads`, current working directory\n\n**Windows**\n1. `%TEMP%` / `%TMP%` (dragged/temp images)\n2. `%USERPROFILE%\\Pictures\\Screenshots` and the OneDrive-redirected `%USERPROFILE%\\OneDrive\\Pictures\\Screenshots` (Win+PrtScn / Snipping Tool)\n3. `%USERPROFILE%\\Pictures`\n4. `~\\Desktop`, `~\\Downloads`, current working directory\n\n**Linux**\n1. `$TMPDIR` / `/tmp`\n2. `~/Pictures/Screenshots`, `~/Pictures`\n3. `~/Desktop`, `~/Downloads`, current working directory\n\npass an absolute `filePath` to skip the search.\n\n## License\n\nMIT\n"}}}} |