1 line
2.9 KiB
JSON
1 line
2.9 KiB
JSON
{"repo": "chichou/grapefruit", "files": {"docs/llm.md": {"0ed32feb34510bb3f2ce60feea7ffc044f18be03": {"t": 1785674474.091243, "n": 2744, "h": "0d58b6b63b79df3b", "c": "# LLM Configuration\n\nigf integrates with LLM providers for features like the GUI assistant and the `/mastg` skill. Configuration is via environment variables.\n\n## Quick Start\n\n```bash\n# Anthropic\nLLM_PROVIDER=anthropic LLM_API_KEY=sk-ant-... LLM_MODEL=claude-sonnet-4-20250514 igf\n\n# OpenAI\nLLM_PROVIDER=openai LLM_API_KEY=sk-... LLM_MODEL=gpt-4o igf\n\n# Local (Ollama)\nLLM_BASE_URL=http://localhost:11434/v1 LLM_MODEL=llama3 igf\n```\n\n## Environment Variables\n\n| Variable | Required | Description |\n|----------|----------|-------------|\n| `LLM_PROVIDER` | No* | Built-in provider name (see below) |\n| `LLM_BASE_URL` | No* | Custom API base URL |\n| `LLM_API_KEY` | No | API key (skipped if empty, for local endpoints) |\n| `LLM_MODEL` | Yes | Model identifier |\n\n*At least one of `LLM_PROVIDER` or `LLM_BASE_URL` must be set.\n\n## Built-in Providers\n\n| Provider | Endpoint | Format |\n|----------|----------|--------|\n| `anthropic` | `https://api.anthropic.com` | Anthropic Messages API |\n| `openai` | `https://api.openai.com` | OpenAI Chat Completions |\n| `gemini` | `https://generativelanguage.googleapis.com` | Google Gemini |\n| `openrouter` | `https://openrouter.ai/api` | OpenAI-compatible |\n\nWhen using a built-in provider, `LLM_BASE_URL` is optional — the known endpoint is used automatically.\n\n## Custom / OpenAI-Compatible Endpoints\n\nAny endpoint that implements the OpenAI Chat Completions format (`POST /v1/chat/completions`) works via `LLM_BASE_URL`:\n\n```bash\n# Ollama (local, no API key needed)\nLLM_BASE_URL=http://localhost:11434/v1 LLM_MODEL=llama3\n\n# vLLM\nLLM_BASE_URL=http://localhost:8000/v1 LLM_MODEL=meta-llama/Llama-3-70b\n\n# Together AI\nLLM_BASE_URL=https://api.together.xyz LLM_API_KEY=... LLM_MODEL=meta-llama/Llama-3-70b-chat-hf\n\n# Groq\nLLM_BASE_URL=https://api.groq.com/openai LLM_API_KEY=... LLM_MODEL=llama3-70b-8192\n\n# Fireworks\nLLM_BASE_URL=https://api.fireworks.ai/inference LLM_API_KEY=... LLM_MODEL=accounts/fireworks/models/llama-v3-70b\n\n# LM Studio (local)\nLLM_BASE_URL=http://localhost:1234/v1 LLM_MODEL=local-model\n```\n\n## Format Resolution\n\nThe request/response format is determined automatically:\n\n1. `LLM_PROVIDER=anthropic` → Anthropic Messages API format\n2. `LLM_PROVIDER=gemini` → Google Gemini format\n3. Everything else (including custom `LLM_BASE_URL`) → OpenAI Chat Completions format\n\n## API Endpoints\n\nThe server exposes `POST /api/llm` which accepts a plain text body and returns the LLM response as plain text. This is used by the GUI for inline analysis features.\n\n```bash\ncurl -X POST http://localhost:31337/api/llm \\\n -H \"Content-Type: text/plain\" \\\n -d \"Explain what this class does: NSURLSession\"\n```\n\nFor streaming responses, use `POST /api/llm/stream` with the same plain text request body.\n"}}}} |