1 line
397 KiB
JSON
1 line
397 KiB
JSON
{"repo": "aicatalyst-team/flock", "files": {"MODELS.md": {"21eb39d5e32fd8e7a18c43fff49355b930b2e862": {"t": 1785747172.0981128, "n": 29745, "h": "7f585d5e15821398", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 19 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## 🎯 Picker table — what to install\n\nScan the column that matches your hardware, then pick by use case. ⭐ = recommended starting point in each row. Catalog has **37 models** at last update (run `flock model search` for the live list, or `flock model search --sort=released` for newest-first).\n\n| Model ID | Size | Min RAM | Chat | Code | Reasoning | Vision | Audio | Long ctx | License | Notes |\n| ------------------------------ | ------ | ------- | :--: | :--: | :-------: | :----: | :---: | :------: | ----------- | -------------------------------------- |\n| **Edge — laptop / Raspberry-Pi-ish** | | | | | | | | | | |\n| `llama-3.2-1b` | 1.3 GB | 2 GB | • | | | | | | Llama 3.2 \\* | Smoke test only |\n| `nomic-embed-text` | 0.3 GB | 2 GB | | | | | | | Apache-2.0 | Embeddings for RAG (768-dim, 8K ctx) |\n| `moondream3` | 1.2 GB | 4 GB | ● | | | ● | | | BSL-1.1 \\* | Tiny vision-language, runs on Pi |\n| `llama-3.2-3b` ⭐ | 2.0 GB | 4 GB | ● | | | | | | Llama 3.2 \\* | Edge default |\n| **Small — 8-16 GB box** | | | | | | | | | | |\n| `mimo-7b` | 4.5 GB | 8 GB | ● | | ●● | | | | Apache-2.0 | Xiaomi reasoning-focused 7B |\n| `qwen-coder-7b` | 4.7 GB | 8 GB | ● | ● | | | | | Apache-2.0 | FIM-capable, older |\n| `mimo-vl-7b` | 4.8 GB | 8 GB | ● | | | ● | | | Apache-2.0 | Small vision-language |\n| `mimo-audio` | 4.8 GB | 8 GB | ● | | | | ● | | Apache-2.0 | Speech + audio understanding |\n| `deepseek-r1-8b` | 4.9 GB | 12 GB | ● | | ●● | | | | MIT | Distilled reasoning |\n| `lfm2.5-8b-a1b` ⭐ | 5.0 GB | 8 GB | ● | | ●● | | | ●● | LFM Open | Best on-device MoE (1B active) |\n| `qwen3-8b` | 5.2 GB | 12 GB | ● | | | | | | Apache-2.0 | General chat |\n| `qwen3-vl-8b` | 5.5 GB | 10 GB | ● | ● | | ●● | | | Apache-2.0 | Vision + tools (charts, OCR, UI) |\n| `gemma4-e2b` | 7.2 GB | 8 GB | ● | | | ● | ● | ●● | Gemma \\* | Mobile/edge multimodal (T/I/A) |\n| `mellum2-12b` | 7.0 GB | 12 GB | ● | ●● | ●● | | | | Apache-2.0 | JetBrains MoE coder (2.5B active) |\n| `mistral-nemo-12b` | 7.1 GB | 12 GB | ● | | | | | ●● | Apache-2.0 | 128K context |\n| `gemma4-12b` | 7.6 GB | 12 GB | ● | | | ●● | ● | ●● | Gemma \\* | Encoder-free any-to-any (T/I/A/V) |\n| `pixtral-12b` | 7.8 GB | 16 GB | ● | | | ●● | | | Apache-2.0 | Mistral vision-language |\n| `qwen-coder-14b` | 9.0 GB | 16 GB | ● | ●● | | | | | Apache-2.0 | Dense code+agent |\n| `qwen3-14b` | 9.0 GB | 16 GB | ● | | | | | | Apache-2.0 | More capable Qwen3 chat |\n| `phi-4-14b` | 9.1 GB | 12 GB | ● | | ●● | | | | MIT | Strong reasoning per byte |\n| `gemma4-e4b` | 9.6 GB | 12 GB | ● | | | ● | ● | ●● | Gemma \\* | Mobile/edge multimodal, larger E-tier |\n| **Mid — 24-32 GB box** | | | | | | | | | | |\n| `gpt-oss-20b` ⭐ | 14 GB | 16 GB | ● | ● | ●● | | | ● | Apache-2.0 | OpenAI open-weight, adjustable thinking |\n| `qwen3.6-27b` ⭐ | 17 GB | 24 GB | ●● | ●● | ● | | | ●● | Apache-2.0 | 77 % SWE-bench, top consumer pick |\n| `gemma4-26b` | 18 GB | 24 GB | ● | | | ● | | ●● | Gemma \\* | MoE 4B-active, multimodal |\n| `qwen3-30b` | 19 GB | 24 GB | ●● | | | | | ●● | Apache-2.0 | MoE 3B-active, very fast |\n| `qwen3-coder-30b` | 19 GB | 24 GB | ● | ●● | | | | ●● | Apache-2.0 | MoE 3.3B-active coder |\n| `qwen-coder-32b` | 20 GB | 32 GB | ● | ●● | | | | | Apache-2.0 | Dense, older but proven |\n| `qwen3-vl-32b` | 20 GB | 32 GB | ● | ● | | ●●● | | | Apache-2.0 | Frontier-tier vision-language |\n| `gemma4-31b` | 20 GB | 32 GB | ● | | | ●● | | ●● | Gemma \\* | Multimodal, larger than 26B |\n| **Power user — single 80 GB GPU / sharded** | | | | | | | | | | |\n| `llama-3.3-70b-sharded` | 43 GB | 48 GB | ●● | | | | | ●● | Llama 3.3 \\* | Needs ≥2 nodes |\n| `gpt-oss-120b` | 65 GB | 80 GB | ●● | ● | ●●● | | | ● | Apache-2.0 | ≈ o4-mini reasoning, single H100 |\n| `llama-4-scout` | 67 GB | 80 GB | ●● | | | ●● | | ●●● | Llama 4 \\* | 10M context, multimodal |\n| **Frontier — multi-machine sharded** | | | | | | | | | | |\n| `step-3.7-flash-sharded` ⭐ | 100 GB | 128 GB | ●● | ●● | ●● | ●● | | ●● | Apache-2.0 | 11B active VLM, fastest frontier MoE |\n| `deepseek-v4-flash-sharded` ⭐ | 150 GB | 160 GB | ●● | ●● | ●●● | | | ●● | MIT | 13B active = fast at frontier quality |\n| `nemotron-3-ultra-sharded` | 280 GB | 320 GB | ●● | ●● | ●●● | | | ●●● | NVIDIA Open \\* | Hybrid Mamba-MoE, 1M ctx, MMLU 89.1 |\n| `glm-5.1-sharded` | 400 GB | 416 GB | ●● | ●●● | ●● | | | ●● | MIT | Best agentic coder |\n| `kimi-k2.6-sharded` | 500 GB | 512 GB | ●● | ●●● | ●● | | | ●● | Mod. MIT | #1 open coding benchmarks |\n\n**Legend** — • basic / ● good / ●● strong / ●●● best-in-tier · ⭐ recommended starting point · \\* restricted license — these 11 entries carry the `restricted-license` catalog tag (extra terms beyond Apache/MIT-style permissive; same amber badge the dashboard shows). `flock model search restricted-license` lists them; `flock model info <id>` links the license text.\n\n> Sizes assume Q4_K_M (Ollama's default). Sharded entries assume Q4 baseline; unsloth's 2-bit dynamic GGUFs cut them roughly in half.\n\n**Quick rules of thumb:**\n- **I just want to try Flock** → `llama-3.2-3b` (2 GB, runs anywhere).\n- **Best on-device edge MoE** → `lfm2.5-8b-a1b` — only 1 B active, MATH500 88.8, MLX-ready.\n- **My laptop coding agent** → `mellum2-12b` (Apache-2.0, MoE, 2.5 B active) on 12 GB or `qwen3-coder-30b` on 24 GB.\n- **Multimodal on a laptop (text/image/audio/video)** → `gemma4-12b` — encoder-free unified architecture.\n- **Best consumer general model** → `qwen3.6-27b` (24 GB) or `gpt-oss-20b` (16 GB).\n- **Reasoning-heavy work** → `gpt-oss-20b` (small) or `gpt-oss-120b` (big).\n- **My team has 4 Mac Studios** → install the same model (e.g. `qwen3.6-27b`) on each — Flock load-balances automatically. No sharding needed for throughput.\n- **A model bigger than any one machine** → sharded tier. Start with `step-3.7-flash-sharded` (Apache-2.0) or `deepseek-v4-flash-sharded` (MIT).\n\nFor per-model walkthroughs (with detailed install + client snippets), the curated set below covers the longest-standing entries. For the rest of the catalog (10+ newer additions including Gemma 4 family, MiMo, Qwen3-VL, Pixtral, Moondream), run `flock model info <id>` — same fields as the table above, plus engine compatibility, license, and release date.\n\n---\n\n## How to choose\n\nUse the [picker table above](#-picker-table--what-to-install). It covers every model in the catalog with size, RAM, capability ratings, and license — pick the row that matches your hardware, then go by use case.\n\nThe per-model walkthroughs below cover a curated subset with full install + client snippets. For every other catalog entry, run `flock model info <id>` for the same metadata (size, RAM, capabilities, license, release date, engine compatibility), or open a [catalog issue](https://github.com/hadihonarvar/flock/issues/new?template=catalog_request.yml) if you'd like a full walkthrough added.\n\n### Installing models that aren't in the catalog\n\n`flock model add` gives you three paths beyond the curated 37 entries:\n\n**1. Scheme prefixes** — one-liner; no YAML, no hardware-floor check:\n\n```bash\nflock model add hf:Qwen/Qwen3-72B-AWQ # vLLM / MLX / llama-server\nflock model add hf:bartowski/Phi-3-mini-GGUF:Phi-3-mini-4k-instruct.Q4_K_M.gguf\nflock model add ollama:phi3:mini # ollama engine only\nflock model add file:/abs/path/my-finetune.gguf # llama-server / MLX, pre-downloaded\n```\n\nThe dashboard's **Models** tab has the same input under \"Add custom model\".\n\n**2. `--from <my.yaml>`** — install from your own catalog YAML; copied into `~/.flock/catalog/` so it persists:\n\n```bash\nflock model add --from ./my-model.yaml\n```\n\n**3. Drop-in dir** — write a YAML to `~/.flock/catalog/<id>.yaml` directly, then install by id:\n\n```bash\nmkdir -p ~/.flock/catalog\ncat > ~/.flock/catalog/my-llama.yaml <<'EOF'\nid: my-llama\ndisplay_name: My fine-tuned llama\nsource:\n type: huggingface\n repo: my-org/my-llama-gguf\nsize_bytes: 8000000000\nhardware:\n min_ram_gb: 8\ncapabilities: [chat]\nrecommended_engines: [llamacpp]\nlicense: apache-2.0\nEOF\nflock model add my-llama\n```\n\nPaths 2 and 3 give you the same UX as a catalog entry — `flock model search my-llama`, `flock model info my-llama`, hardware-floor checks, license badges all work. Path 1 is the quick fallback when you don't want to write YAML.\n\n---\n\n## Table of contents (curated walkthroughs)\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n- [`nomic-embed-text`](#nomic-embed-text--embeddings-for-rag) — embeddings for RAG\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## `nomic-embed-text` — embeddings for RAG\n\n**Why pick it.** It's the default embedding model in Flock's catalog — Apache-2.0 licensed, 137 M parameters, 768-dimensional vectors, 8K-token context window. Beats OpenAI `text-embedding-ada-002` on MTEB benchmarks. If you're wiring Flock into a RAG pipeline (LangChain, LlamaIndex, Haystack, raw vector DB), this is the one to start with.\n\n**Picker fit.** Any machine — it needs ~270 MB on disk and 2 GB RAM. Runs alongside your chat model with no extra hardware budget.\n\n**Install.**\n\n```bash\nflock model add nomic-embed-text\n```\n\n**Use via API (OpenAI embeddings shape).**\n\n```bash\nKEY=\"sk-orc-...\"\n\ncurl http://localhost:8080/v1/embeddings \\\n -H \"Authorization: Bearer $KEY\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\":\"nomic-embed-text\",\"input\":\"the quick brown fox\"}'\n```\n\nResponse (truncated):\n\n```json\n{\n \"object\": \"list\",\n \"data\": [{\"object\":\"embedding\",\"index\":0,\"embedding\":[0.0214, -0.0179, ...768 floats...]}],\n \"model\": \"nomic-embed-text\",\n \"usage\": {\"prompt_tokens\": 5, \"total_tokens\": 5}\n}\n```\n\n**Use via SDK.**\n\n```python\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.embeddings.create(model=\"nomic-embed-text\", input=[\"hello\", \"world\"])\nprint(len(resp.data[0].embedding)) # → 768\n```\n\nDrop-in for any code that calls OpenAI `text-embedding-3-small` / `text-embedding-ada-002` — same request and response shape.\n\n**Use via LangChain / LlamaIndex.** Point the `OpenAIEmbeddings` wrapper at `http://localhost:8080/v1` with your `sk-orc-*` key and `model=\"nomic-embed-text\"`. No other code changes.\n\n**What it doesn't do.** This is an embedding-only model. Don't try `/v1/chat/completions` with it — it won't respond. Pair it with a chat model (Qwen, Llama, Gemma) for the generation half of your RAG stack.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "f6bef62f62999ac25d80d0dde8ff707e5e1028b2": {"t": 1785770944.8657768, "n": 29745, "h": "7f585d5e15821398", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 19 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## 🎯 Picker table — what to install\n\nScan the column that matches your hardware, then pick by use case. ⭐ = recommended starting point in each row. Catalog has **37 models** at last update (run `flock model search` for the live list, or `flock model search --sort=released` for newest-first).\n\n| Model ID | Size | Min RAM | Chat | Code | Reasoning | Vision | Audio | Long ctx | License | Notes |\n| ------------------------------ | ------ | ------- | :--: | :--: | :-------: | :----: | :---: | :------: | ----------- | -------------------------------------- |\n| **Edge — laptop / Raspberry-Pi-ish** | | | | | | | | | | |\n| `llama-3.2-1b` | 1.3 GB | 2 GB | • | | | | | | Llama 3.2 \\* | Smoke test only |\n| `nomic-embed-text` | 0.3 GB | 2 GB | | | | | | | Apache-2.0 | Embeddings for RAG (768-dim, 8K ctx) |\n| `moondream3` | 1.2 GB | 4 GB | ● | | | ● | | | BSL-1.1 \\* | Tiny vision-language, runs on Pi |\n| `llama-3.2-3b` ⭐ | 2.0 GB | 4 GB | ● | | | | | | Llama 3.2 \\* | Edge default |\n| **Small — 8-16 GB box** | | | | | | | | | | |\n| `mimo-7b` | 4.5 GB | 8 GB | ● | | ●● | | | | Apache-2.0 | Xiaomi reasoning-focused 7B |\n| `qwen-coder-7b` | 4.7 GB | 8 GB | ● | ● | | | | | Apache-2.0 | FIM-capable, older |\n| `mimo-vl-7b` | 4.8 GB | 8 GB | ● | | | ● | | | Apache-2.0 | Small vision-language |\n| `mimo-audio` | 4.8 GB | 8 GB | ● | | | | ● | | Apache-2.0 | Speech + audio understanding |\n| `deepseek-r1-8b` | 4.9 GB | 12 GB | ● | | ●● | | | | MIT | Distilled reasoning |\n| `lfm2.5-8b-a1b` ⭐ | 5.0 GB | 8 GB | ● | | ●● | | | ●● | LFM Open | Best on-device MoE (1B active) |\n| `qwen3-8b` | 5.2 GB | 12 GB | ● | | | | | | Apache-2.0 | General chat |\n| `qwen3-vl-8b` | 5.5 GB | 10 GB | ● | ● | | ●● | | | Apache-2.0 | Vision + tools (charts, OCR, UI) |\n| `gemma4-e2b` | 7.2 GB | 8 GB | ● | | | ● | ● | ●● | Gemma \\* | Mobile/edge multimodal (T/I/A) |\n| `mellum2-12b` | 7.0 GB | 12 GB | ● | ●● | ●● | | | | Apache-2.0 | JetBrains MoE coder (2.5B active) |\n| `mistral-nemo-12b` | 7.1 GB | 12 GB | ● | | | | | ●● | Apache-2.0 | 128K context |\n| `gemma4-12b` | 7.6 GB | 12 GB | ● | | | ●● | ● | ●● | Gemma \\* | Encoder-free any-to-any (T/I/A/V) |\n| `pixtral-12b` | 7.8 GB | 16 GB | ● | | | ●● | | | Apache-2.0 | Mistral vision-language |\n| `qwen-coder-14b` | 9.0 GB | 16 GB | ● | ●● | | | | | Apache-2.0 | Dense code+agent |\n| `qwen3-14b` | 9.0 GB | 16 GB | ● | | | | | | Apache-2.0 | More capable Qwen3 chat |\n| `phi-4-14b` | 9.1 GB | 12 GB | ● | | ●● | | | | MIT | Strong reasoning per byte |\n| `gemma4-e4b` | 9.6 GB | 12 GB | ● | | | ● | ● | ●● | Gemma \\* | Mobile/edge multimodal, larger E-tier |\n| **Mid — 24-32 GB box** | | | | | | | | | | |\n| `gpt-oss-20b` ⭐ | 14 GB | 16 GB | ● | ● | ●● | | | ● | Apache-2.0 | OpenAI open-weight, adjustable thinking |\n| `qwen3.6-27b` ⭐ | 17 GB | 24 GB | ●● | ●● | ● | | | ●● | Apache-2.0 | 77 % SWE-bench, top consumer pick |\n| `gemma4-26b` | 18 GB | 24 GB | ● | | | ● | | ●● | Gemma \\* | MoE 4B-active, multimodal |\n| `qwen3-30b` | 19 GB | 24 GB | ●● | | | | | ●● | Apache-2.0 | MoE 3B-active, very fast |\n| `qwen3-coder-30b` | 19 GB | 24 GB | ● | ●● | | | | ●● | Apache-2.0 | MoE 3.3B-active coder |\n| `qwen-coder-32b` | 20 GB | 32 GB | ● | ●● | | | | | Apache-2.0 | Dense, older but proven |\n| `qwen3-vl-32b` | 20 GB | 32 GB | ● | ● | | ●●● | | | Apache-2.0 | Frontier-tier vision-language |\n| `gemma4-31b` | 20 GB | 32 GB | ● | | | ●● | | ●● | Gemma \\* | Multimodal, larger than 26B |\n| **Power user — single 80 GB GPU / sharded** | | | | | | | | | | |\n| `llama-3.3-70b-sharded` | 43 GB | 48 GB | ●● | | | | | ●● | Llama 3.3 \\* | Needs ≥2 nodes |\n| `gpt-oss-120b` | 65 GB | 80 GB | ●● | ● | ●●● | | | ● | Apache-2.0 | ≈ o4-mini reasoning, single H100 |\n| `llama-4-scout` | 67 GB | 80 GB | ●● | | | ●● | | ●●● | Llama 4 \\* | 10M context, multimodal |\n| **Frontier — multi-machine sharded** | | | | | | | | | | |\n| `step-3.7-flash-sharded` ⭐ | 100 GB | 128 GB | ●● | ●● | ●● | ●● | | ●● | Apache-2.0 | 11B active VLM, fastest frontier MoE |\n| `deepseek-v4-flash-sharded` ⭐ | 150 GB | 160 GB | ●● | ●● | ●●● | | | ●● | MIT | 13B active = fast at frontier quality |\n| `nemotron-3-ultra-sharded` | 280 GB | 320 GB | ●● | ●● | ●●● | | | ●●● | NVIDIA Open \\* | Hybrid Mamba-MoE, 1M ctx, MMLU 89.1 |\n| `glm-5.1-sharded` | 400 GB | 416 GB | ●● | ●●● | ●● | | | ●● | MIT | Best agentic coder |\n| `kimi-k2.6-sharded` | 500 GB | 512 GB | ●● | ●●● | ●● | | | ●● | Mod. MIT | #1 open coding benchmarks |\n\n**Legend** — • basic / ● good / ●● strong / ●●● best-in-tier · ⭐ recommended starting point · \\* restricted license — these 11 entries carry the `restricted-license` catalog tag (extra terms beyond Apache/MIT-style permissive; same amber badge the dashboard shows). `flock model search restricted-license` lists them; `flock model info <id>` links the license text.\n\n> Sizes assume Q4_K_M (Ollama's default). Sharded entries assume Q4 baseline; unsloth's 2-bit dynamic GGUFs cut them roughly in half.\n\n**Quick rules of thumb:**\n- **I just want to try Flock** → `llama-3.2-3b` (2 GB, runs anywhere).\n- **Best on-device edge MoE** → `lfm2.5-8b-a1b` — only 1 B active, MATH500 88.8, MLX-ready.\n- **My laptop coding agent** → `mellum2-12b` (Apache-2.0, MoE, 2.5 B active) on 12 GB or `qwen3-coder-30b` on 24 GB.\n- **Multimodal on a laptop (text/image/audio/video)** → `gemma4-12b` — encoder-free unified architecture.\n- **Best consumer general model** → `qwen3.6-27b` (24 GB) or `gpt-oss-20b` (16 GB).\n- **Reasoning-heavy work** → `gpt-oss-20b` (small) or `gpt-oss-120b` (big).\n- **My team has 4 Mac Studios** → install the same model (e.g. `qwen3.6-27b`) on each — Flock load-balances automatically. No sharding needed for throughput.\n- **A model bigger than any one machine** → sharded tier. Start with `step-3.7-flash-sharded` (Apache-2.0) or `deepseek-v4-flash-sharded` (MIT).\n\nFor per-model walkthroughs (with detailed install + client snippets), the curated set below covers the longest-standing entries. For the rest of the catalog (10+ newer additions including Gemma 4 family, MiMo, Qwen3-VL, Pixtral, Moondream), run `flock model info <id>` — same fields as the table above, plus engine compatibility, license, and release date.\n\n---\n\n## How to choose\n\nUse the [picker table above](#-picker-table--what-to-install). It covers every model in the catalog with size, RAM, capability ratings, and license — pick the row that matches your hardware, then go by use case.\n\nThe per-model walkthroughs below cover a curated subset with full install + client snippets. For every other catalog entry, run `flock model info <id>` for the same metadata (size, RAM, capabilities, license, release date, engine compatibility), or open a [catalog issue](https://github.com/hadihonarvar/flock/issues/new?template=catalog_request.yml) if you'd like a full walkthrough added.\n\n### Installing models that aren't in the catalog\n\n`flock model add` gives you three paths beyond the curated 37 entries:\n\n**1. Scheme prefixes** — one-liner; no YAML, no hardware-floor check:\n\n```bash\nflock model add hf:Qwen/Qwen3-72B-AWQ # vLLM / MLX / llama-server\nflock model add hf:bartowski/Phi-3-mini-GGUF:Phi-3-mini-4k-instruct.Q4_K_M.gguf\nflock model add ollama:phi3:mini # ollama engine only\nflock model add file:/abs/path/my-finetune.gguf # llama-server / MLX, pre-downloaded\n```\n\nThe dashboard's **Models** tab has the same input under \"Add custom model\".\n\n**2. `--from <my.yaml>`** — install from your own catalog YAML; copied into `~/.flock/catalog/` so it persists:\n\n```bash\nflock model add --from ./my-model.yaml\n```\n\n**3. Drop-in dir** — write a YAML to `~/.flock/catalog/<id>.yaml` directly, then install by id:\n\n```bash\nmkdir -p ~/.flock/catalog\ncat > ~/.flock/catalog/my-llama.yaml <<'EOF'\nid: my-llama\ndisplay_name: My fine-tuned llama\nsource:\n type: huggingface\n repo: my-org/my-llama-gguf\nsize_bytes: 8000000000\nhardware:\n min_ram_gb: 8\ncapabilities: [chat]\nrecommended_engines: [llamacpp]\nlicense: apache-2.0\nEOF\nflock model add my-llama\n```\n\nPaths 2 and 3 give you the same UX as a catalog entry — `flock model search my-llama`, `flock model info my-llama`, hardware-floor checks, license badges all work. Path 1 is the quick fallback when you don't want to write YAML.\n\n---\n\n## Table of contents (curated walkthroughs)\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n- [`nomic-embed-text`](#nomic-embed-text--embeddings-for-rag) — embeddings for RAG\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## `nomic-embed-text` — embeddings for RAG\n\n**Why pick it.** It's the default embedding model in Flock's catalog — Apache-2.0 licensed, 137 M parameters, 768-dimensional vectors, 8K-token context window. Beats OpenAI `text-embedding-ada-002` on MTEB benchmarks. If you're wiring Flock into a RAG pipeline (LangChain, LlamaIndex, Haystack, raw vector DB), this is the one to start with.\n\n**Picker fit.** Any machine — it needs ~270 MB on disk and 2 GB RAM. Runs alongside your chat model with no extra hardware budget.\n\n**Install.**\n\n```bash\nflock model add nomic-embed-text\n```\n\n**Use via API (OpenAI embeddings shape).**\n\n```bash\nKEY=\"sk-orc-...\"\n\ncurl http://localhost:8080/v1/embeddings \\\n -H \"Authorization: Bearer $KEY\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\":\"nomic-embed-text\",\"input\":\"the quick brown fox\"}'\n```\n\nResponse (truncated):\n\n```json\n{\n \"object\": \"list\",\n \"data\": [{\"object\":\"embedding\",\"index\":0,\"embedding\":[0.0214, -0.0179, ...768 floats...]}],\n \"model\": \"nomic-embed-text\",\n \"usage\": {\"prompt_tokens\": 5, \"total_tokens\": 5}\n}\n```\n\n**Use via SDK.**\n\n```python\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.embeddings.create(model=\"nomic-embed-text\", input=[\"hello\", \"world\"])\nprint(len(resp.data[0].embedding)) # → 768\n```\n\nDrop-in for any code that calls OpenAI `text-embedding-3-small` / `text-embedding-ada-002` — same request and response shape.\n\n**Use via LangChain / LlamaIndex.** Point the `OpenAIEmbeddings` wrapper at `http://localhost:8080/v1` with your `sk-orc-*` key and `model=\"nomic-embed-text\"`. No other code changes.\n\n**What it doesn't do.** This is an embedding-only model. Don't try `/v1/chat/completions` with it — it won't respond. Pair it with a chat model (Qwen, Llama, Gemma) for the generation half of your RAG stack.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "f8e597648dbb12a008e36efafd53832b1051bf2a": {"t": 1785770946.2934194, "n": 29302, "h": "b0031f59478ffd53", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 19 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## 🎯 Picker table — what to install\n\nScan the column that matches your hardware, then pick by use case. ⭐ = recommended starting point in each row. Catalog has **37 models** at last update (run `flock model search` for the live list, or `flock model search --sort=released` for newest-first).\n\n| Model ID | Size | Min RAM | Chat | Code | Reasoning | Vision | Audio | Long ctx | License | Notes |\n| ------------------------------ | ------ | ------- | :--: | :--: | :-------: | :----: | :---: | :------: | ----------- | -------------------------------------- |\n| **Edge — laptop / Raspberry-Pi-ish** | | | | | | | | | | |\n| `llama-3.2-1b` | 1.3 GB | 2 GB | • | | | | | | Llama 3.2 | Smoke test only |\n| `moondream3` | 1.2 GB | 4 GB | ● | | | ● | | | Apache-2.0 | Tiny vision-language, runs on Pi |\n| `llama-3.2-3b` ⭐ | 2.0 GB | 4 GB | ● | | | | | | Llama 3.2 | Edge default |\n| **Small — 8-16 GB box** | | | | | | | | | | |\n| `mimo-7b` | 4.5 GB | 8 GB | ● | | ●● | | | | Apache-2.0 | Xiaomi reasoning-focused 7B |\n| `qwen-coder-7b` | 4.7 GB | 8 GB | ● | ● | | | | | Apache-2.0 | FIM-capable, older |\n| `mimo-vl-7b` | 4.8 GB | 8 GB | ● | | | ● | | | Apache-2.0 | Small vision-language |\n| `mimo-audio` | 4.8 GB | 8 GB | ● | | | | ● | | Apache-2.0 | Speech + audio understanding |\n| `deepseek-r1-8b` | 4.9 GB | 12 GB | ● | | ●● | | | | MIT | Distilled reasoning |\n| `lfm2.5-8b-a1b` ⭐ | 5.0 GB | 8 GB | ● | | ●● | | | ●● | LFM Open | Best on-device MoE (1B active) |\n| `qwen3-8b` | 5.2 GB | 12 GB | ● | | | | | | Apache-2.0 | General chat |\n| `qwen3-vl-8b` | 5.5 GB | 10 GB | ● | ● | | ●● | | | Apache-2.0 | Vision + tools (charts, OCR, UI) |\n| `gemma4-e2b` | 7.2 GB | 8 GB | ● | | | ● | ● | ●● | Gemma | Mobile/edge multimodal (T/I/A) |\n| `mellum2-12b` | 7.0 GB | 12 GB | ● | ●● | ●● | | | | Apache-2.0 | JetBrains MoE coder (2.5B active) |\n| `mistral-nemo-12b` | 7.1 GB | 12 GB | ● | | | | | ●● | Apache-2.0 | 128K context |\n| `gemma4-12b` | 7.6 GB | 12 GB | ● | | | ●● | ● | ●● | Gemma | Encoder-free any-to-any (T/I/A/V) |\n| `pixtral-12b` | 7.8 GB | 16 GB | ● | | | ●● | | | Apache-2.0 | Mistral vision-language |\n| `qwen-coder-14b` | 9.0 GB | 16 GB | ● | ●● | | | | | Apache-2.0 | Dense code+agent |\n| `qwen3-14b` | 9.0 GB | 16 GB | ● | | | | | | Apache-2.0 | More capable Qwen3 chat |\n| `phi-4-14b` | 9.1 GB | 12 GB | ● | | ●● | | | | MIT | Strong reasoning per byte |\n| `gemma4-e4b` | 9.6 GB | 12 GB | ● | | | ● | ● | ●● | Gemma | Mobile/edge multimodal, larger E-tier |\n| **Mid — 24-32 GB box** | | | | | | | | | | |\n| `gpt-oss-20b` ⭐ | 14 GB | 16 GB | ● | ● | ●● | | | ● | Apache-2.0 | OpenAI open-weight, adjustable thinking |\n| `qwen3.6-27b` ⭐ | 17 GB | 24 GB | ●● | ●● | ● | | | ●● | Apache-2.0 | 77 % SWE-bench, top consumer pick |\n| `gemma4-26b` | 18 GB | 24 GB | ● | | | ● | | ●● | Gemma | MoE 4B-active, multimodal |\n| `qwen3-30b` | 19 GB | 24 GB | ●● | | | | | ●● | Apache-2.0 | MoE 3B-active, very fast |\n| `qwen3-coder-30b` | 19 GB | 24 GB | ● | ●● | | | | ●● | Apache-2.0 | MoE 3.3B-active coder |\n| `qwen-coder-32b` | 20 GB | 32 GB | ● | ●● | | | | | Apache-2.0 | Dense, older but proven |\n| `qwen3-vl-32b` | 20 GB | 32 GB | ● | ● | | ●●● | | | Apache-2.0 | Frontier-tier vision-language |\n| `gemma4-31b` | 20 GB | 32 GB | ● | | | ●● | | ●● | Gemma | Multimodal, larger than 26B |\n| **Power user — single 80 GB GPU / sharded** | | | | | | | | | | |\n| `llama-3.3-70b-sharded` | 43 GB | 48 GB | ●● | | | | | ●● | Llama 3.3 | Needs ≥2 nodes |\n| `gpt-oss-120b` | 65 GB | 80 GB | ●● | ● | ●●● | | | ● | Apache-2.0 | ≈ o4-mini reasoning, single H100 |\n| `llama-4-scout` | 67 GB | 80 GB | ●● | | | ●● | | ●●● | Llama 4 | 10M context, multimodal |\n| **Frontier — multi-machine sharded** | | | | | | | | | | |\n| `step-3.7-flash-sharded` ⭐ | 100 GB | 128 GB | ●● | ●● | ●● | ●● | | ●● | Apache-2.0 | 11B active VLM, fastest frontier MoE |\n| `deepseek-v4-flash-sharded` ⭐ | 150 GB | 160 GB | ●● | ●● | ●●● | | | ●● | MIT | 13B active = fast at frontier quality |\n| `nemotron-3-ultra-sharded` | 280 GB | 320 GB | ●● | ●● | ●●● | | | ●●● | NVIDIA Open | Hybrid Mamba-MoE, 1M ctx, MMLU 89.1 |\n| `glm-5.1-sharded` | 400 GB | 416 GB | ●● | ●●● | ●● | | | ●● | MIT | Best agentic coder |\n| `kimi-k2.6-sharded` | 500 GB | 512 GB | ●● | ●●● | ●● | | | ●● | Mod. MIT | #1 open coding benchmarks |\n\n**Legend** — • basic / ● good / ●● strong / ●●● best-in-tier · ⭐ recommended starting point\n\n> Sizes assume Q4_K_M (Ollama's default). Sharded entries assume Q4 baseline; unsloth's 2-bit dynamic GGUFs cut them roughly in half.\n\n**Quick rules of thumb:**\n- **I just want to try Flock** → `llama-3.2-3b` (2 GB, runs anywhere).\n- **Best on-device edge MoE** → `lfm2.5-8b-a1b` — only 1 B active, MATH500 88.8, MLX-ready.\n- **My laptop coding agent** → `mellum2-12b` (Apache-2.0, MoE, 2.5 B active) on 12 GB or `qwen3-coder-30b` on 24 GB.\n- **Multimodal on a laptop (text/image/audio/video)** → `gemma4-12b` — encoder-free unified architecture.\n- **Best consumer general model** → `qwen3.6-27b` (24 GB) or `gpt-oss-20b` (16 GB).\n- **Reasoning-heavy work** → `gpt-oss-20b` (small) or `gpt-oss-120b` (big).\n- **My team has 4 Mac Studios** → install the same model (e.g. `qwen3.6-27b`) on each — Flock load-balances automatically. No sharding needed for throughput.\n- **A model bigger than any one machine** → sharded tier. Start with `step-3.7-flash-sharded` (Apache-2.0) or `deepseek-v4-flash-sharded` (MIT).\n\nFor per-model walkthroughs (with detailed install + client snippets), the curated set below covers the longest-standing entries. For the rest of the catalog (10+ newer additions including Gemma 4 family, MiMo, Qwen3-VL, Pixtral, Moondream), run `flock model info <id>` — same fields as the table above, plus engine compatibility, license, and release date.\n\n---\n\n## How to choose\n\nUse the [picker table above](#-picker-table--what-to-install). It covers every model in the catalog with size, RAM, capability ratings, and license — pick the row that matches your hardware, then go by use case.\n\nThe per-model walkthroughs below cover a curated subset with full install + client snippets. For every other catalog entry, run `flock model info <id>` for the same metadata (size, RAM, capabilities, license, release date, engine compatibility), or open a [catalog issue](https://github.com/hadihonarvar/flock/issues/new?template=catalog_request.yml) if you'd like a full walkthrough added.\n\n### Installing models that aren't in the catalog\n\n`flock model add` gives you three paths beyond the curated 37 entries:\n\n**1. Scheme prefixes** — one-liner; no YAML, no hardware-floor check:\n\n```bash\nflock model add hf:Qwen/Qwen3-72B-AWQ # vLLM / MLX / llama-server\nflock model add hf:bartowski/Phi-3-mini-GGUF:Phi-3-mini-4k-instruct.Q4_K_M.gguf\nflock model add ollama:phi3:mini # ollama engine only\nflock model add file:/abs/path/my-finetune.gguf # llama-server / MLX, pre-downloaded\n```\n\nThe dashboard's **Models** tab has the same input under \"Add custom model\".\n\n**2. `--from <my.yaml>`** — install from your own catalog YAML; copied into `~/.flock/catalog/` so it persists:\n\n```bash\nflock model add --from ./my-model.yaml\n```\n\n**3. Drop-in dir** — write a YAML to `~/.flock/catalog/<id>.yaml` directly, then install by id:\n\n```bash\nmkdir -p ~/.flock/catalog\ncat > ~/.flock/catalog/my-llama.yaml <<'EOF'\nid: my-llama\ndisplay_name: My fine-tuned llama\nsource:\n type: huggingface\n repo: my-org/my-llama-gguf\nsize_bytes: 8000000000\nhardware:\n min_ram_gb: 8\ncapabilities: [chat]\nrecommended_engines: [llamacpp]\nlicense: apache-2.0\nEOF\nflock model add my-llama\n```\n\nPaths 2 and 3 give you the same UX as a catalog entry — `flock model search my-llama`, `flock model info my-llama`, hardware-floor checks, license badges all work. Path 1 is the quick fallback when you don't want to write YAML.\n\n---\n\n## Table of contents (curated walkthroughs)\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n- [`nomic-embed-text`](#nomic-embed-text--embeddings-for-rag) — embeddings for RAG\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## `nomic-embed-text` — embeddings for RAG\n\n**Why pick it.** It's the default embedding model in Flock's catalog — Apache-2.0 licensed, 137 M parameters, 768-dimensional vectors, 8K-token context window. Beats OpenAI `text-embedding-ada-002` on MTEB benchmarks. If you're wiring Flock into a RAG pipeline (LangChain, LlamaIndex, Haystack, raw vector DB), this is the one to start with.\n\n**Picker fit.** Any machine — it needs ~270 MB on disk and 2 GB RAM. Runs alongside your chat model with no extra hardware budget.\n\n**Install.**\n\n```bash\nflock model add nomic-embed-text\n```\n\n**Use via API (OpenAI embeddings shape).**\n\n```bash\nKEY=\"sk-orc-...\"\n\ncurl http://localhost:8080/v1/embeddings \\\n -H \"Authorization: Bearer $KEY\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\":\"nomic-embed-text\",\"input\":\"the quick brown fox\"}'\n```\n\nResponse (truncated):\n\n```json\n{\n \"object\": \"list\",\n \"data\": [{\"object\":\"embedding\",\"index\":0,\"embedding\":[0.0214, -0.0179, ...768 floats...]}],\n \"model\": \"nomic-embed-text\",\n \"usage\": {\"prompt_tokens\": 5, \"total_tokens\": 5}\n}\n```\n\n**Use via SDK.**\n\n```python\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.embeddings.create(model=\"nomic-embed-text\", input=[\"hello\", \"world\"])\nprint(len(resp.data[0].embedding)) # → 768\n```\n\nDrop-in for any code that calls OpenAI `text-embedding-3-small` / `text-embedding-ada-002` — same request and response shape.\n\n**Use via LangChain / LlamaIndex.** Point the `OpenAIEmbeddings` wrapper at `http://localhost:8080/v1` with your `sk-orc-*` key and `model=\"nomic-embed-text\"`. No other code changes.\n\n**What it doesn't do.** This is an embedding-only model. Don't try `/v1/chat/completions` with it — it won't respond. Pair it with a chat model (Qwen, Llama, Gemma) for the generation half of your RAG stack.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "20ce480f9972b2abc6ea832635c0933c7d37a9af": {"t": 1785770947.7404745, "n": 28592, "h": "c4dae4d9f144803d", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 19 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## 🎯 Picker table — what to install\n\nScan the column that matches your hardware, then pick by use case. ⭐ = recommended starting point in each row. Catalog has **37 models** at last update (run `flock model search` for the live list, or `flock model search --sort=released` for newest-first).\n\n| Model ID | Size | Min RAM | Chat | Code | Reasoning | Vision | Audio | Long ctx | License | Notes |\n| ------------------------------ | ------ | ------- | :--: | :--: | :-------: | :----: | :---: | :------: | ----------- | -------------------------------------- |\n| **Edge — laptop / Raspberry-Pi-ish** | | | | | | | | | | |\n| `llama-3.2-1b` | 1.3 GB | 2 GB | • | | | | | | Llama 3.2 | Smoke test only |\n| `moondream3` | 1.2 GB | 4 GB | ● | | | ● | | | Apache-2.0 | Tiny vision-language, runs on Pi |\n| `llama-3.2-3b` ⭐ | 2.0 GB | 4 GB | ● | | | | | | Llama 3.2 | Edge default |\n| **Small — 8-16 GB box** | | | | | | | | | | |\n| `mimo-7b` | 4.5 GB | 8 GB | ● | | ●● | | | | Apache-2.0 | Xiaomi reasoning-focused 7B |\n| `qwen-coder-7b` | 4.7 GB | 8 GB | ● | ● | | | | | Apache-2.0 | FIM-capable, older |\n| `mimo-vl-7b` | 4.8 GB | 8 GB | ● | | | ● | | | Apache-2.0 | Small vision-language |\n| `mimo-audio` | 4.8 GB | 8 GB | ● | | | | ● | | Apache-2.0 | Speech + audio understanding |\n| `deepseek-r1-8b` | 4.9 GB | 12 GB | ● | | ●● | | | | MIT | Distilled reasoning |\n| `lfm2.5-8b-a1b` ⭐ | 5.0 GB | 8 GB | ● | | ●● | | | ●● | LFM Open | Best on-device MoE (1B active) |\n| `qwen3-8b` | 5.2 GB | 12 GB | ● | | | | | | Apache-2.0 | General chat |\n| `qwen3-vl-8b` | 5.5 GB | 10 GB | ● | ● | | ●● | | | Apache-2.0 | Vision + tools (charts, OCR, UI) |\n| `gemma4-e2b` | 7.2 GB | 8 GB | ● | | | ● | ● | ●● | Gemma | Mobile/edge multimodal (T/I/A) |\n| `mellum2-12b` | 7.0 GB | 12 GB | ● | ●● | ●● | | | | Apache-2.0 | JetBrains MoE coder (2.5B active) |\n| `mistral-nemo-12b` | 7.1 GB | 12 GB | ● | | | | | ●● | Apache-2.0 | 128K context |\n| `gemma4-12b` | 7.6 GB | 12 GB | ● | | | ●● | ● | ●● | Gemma | Encoder-free any-to-any (T/I/A/V) |\n| `pixtral-12b` | 7.8 GB | 16 GB | ● | | | ●● | | | Apache-2.0 | Mistral vision-language |\n| `qwen-coder-14b` | 9.0 GB | 16 GB | ● | ●● | | | | | Apache-2.0 | Dense code+agent |\n| `qwen3-14b` | 9.0 GB | 16 GB | ● | | | | | | Apache-2.0 | More capable Qwen3 chat |\n| `phi-4-14b` | 9.1 GB | 12 GB | ● | | ●● | | | | MIT | Strong reasoning per byte |\n| `gemma4-e4b` | 9.6 GB | 12 GB | ● | | | ● | ● | ●● | Gemma | Mobile/edge multimodal, larger E-tier |\n| **Mid — 24-32 GB box** | | | | | | | | | | |\n| `gpt-oss-20b` ⭐ | 14 GB | 16 GB | ● | ● | ●● | | | ● | Apache-2.0 | OpenAI open-weight, adjustable thinking |\n| `qwen3.6-27b` ⭐ | 17 GB | 24 GB | ●● | ●● | ● | | | ●● | Apache-2.0 | 77 % SWE-bench, top consumer pick |\n| `gemma4-26b` | 18 GB | 24 GB | ● | | | ● | | ●● | Gemma | MoE 4B-active, multimodal |\n| `qwen3-30b` | 19 GB | 24 GB | ●● | | | | | ●● | Apache-2.0 | MoE 3B-active, very fast |\n| `qwen3-coder-30b` | 19 GB | 24 GB | ● | ●● | | | | ●● | Apache-2.0 | MoE 3.3B-active coder |\n| `qwen-coder-32b` | 20 GB | 32 GB | ● | ●● | | | | | Apache-2.0 | Dense, older but proven |\n| `qwen3-vl-32b` | 20 GB | 32 GB | ● | ● | | ●●● | | | Apache-2.0 | Frontier-tier vision-language |\n| `gemma4-31b` | 20 GB | 32 GB | ● | | | ●● | | ●● | Gemma | Multimodal, larger than 26B |\n| **Power user — single 80 GB GPU / sharded** | | | | | | | | | | |\n| `llama-3.3-70b-sharded` | 43 GB | 48 GB | ●● | | | | | ●● | Llama 3.3 | Needs ≥2 nodes |\n| `gpt-oss-120b` | 65 GB | 80 GB | ●● | ● | ●●● | | | ● | Apache-2.0 | ≈ o4-mini reasoning, single H100 |\n| `llama-4-scout` | 67 GB | 80 GB | ●● | | | ●● | | ●●● | Llama 4 | 10M context, multimodal |\n| **Frontier — multi-machine sharded** | | | | | | | | | | |\n| `step-3.7-flash-sharded` ⭐ | 100 GB | 128 GB | ●● | ●● | ●● | ●● | | ●● | Apache-2.0 | 11B active VLM, fastest frontier MoE |\n| `deepseek-v4-flash-sharded` ⭐ | 150 GB | 160 GB | ●● | ●● | ●●● | | | ●● | MIT | 13B active = fast at frontier quality |\n| `nemotron-3-ultra-sharded` | 280 GB | 320 GB | ●● | ●● | ●●● | | | ●●● | NVIDIA Open | Hybrid Mamba-MoE, 1M ctx, MMLU 89.1 |\n| `glm-5.1-sharded` | 400 GB | 416 GB | ●● | ●●● | ●● | | | ●● | MIT | Best agentic coder |\n| `kimi-k2.6-sharded` | 500 GB | 512 GB | ●● | ●●● | ●● | | | ●● | Mod. MIT | #1 open coding benchmarks |\n\n**Legend** — • basic / ● good / ●● strong / ●●● best-in-tier · ⭐ recommended starting point\n\n> Sizes assume Q4_K_M (Ollama's default). Sharded entries assume Q4 baseline; unsloth's 2-bit dynamic GGUFs cut them roughly in half.\n\n**Quick rules of thumb:**\n- **I just want to try Flock** → `llama-3.2-3b` (2 GB, runs anywhere).\n- **Best on-device edge MoE** → `lfm2.5-8b-a1b` — only 1 B active, MATH500 88.8, MLX-ready.\n- **My laptop coding agent** → `mellum2-12b` (Apache-2.0, MoE, 2.5 B active) on 12 GB or `qwen3-coder-30b` on 24 GB.\n- **Multimodal on a laptop (text/image/audio/video)** → `gemma4-12b` — encoder-free unified architecture.\n- **Best consumer general model** → `qwen3.6-27b` (24 GB) or `gpt-oss-20b` (16 GB).\n- **Reasoning-heavy work** → `gpt-oss-20b` (small) or `gpt-oss-120b` (big).\n- **My team has 4 Mac Studios** → install the same model (e.g. `qwen3.6-27b`) on each — Flock load-balances automatically. No sharding needed for throughput.\n- **A model bigger than any one machine** → sharded tier. Start with `step-3.7-flash-sharded` (Apache-2.0) or `deepseek-v4-flash-sharded` (MIT).\n\nFor per-model walkthroughs (with detailed install + client snippets), the curated set below covers the longest-standing entries. For the rest of the catalog (10+ newer additions including Gemma 4 family, MiMo, Qwen3-VL, Pixtral, Moondream), run `flock model info <id>` — same fields as the table above, plus engine compatibility, license, and release date.\n\n---\n\n## How to choose\n\nUse the [picker table above](#-picker-table--what-to-install). It covers every model in the catalog with size, RAM, capability ratings, and license — pick the row that matches your hardware, then go by use case.\n\nThe per-model walkthroughs below cover a curated subset with full install + client snippets. For every other catalog entry, run `flock model info <id>` for the same metadata (size, RAM, capabilities, license, release date, engine compatibility), or open a [catalog issue](https://github.com/hadihonarvar/flock/issues/new?template=catalog_request.yml) if you'd like a full walkthrough added.\n\n### Installing models that aren't in the catalog\n\n`flock model add` accepts three scheme prefixes that bypass the catalog and pull anything the configured engine can serve. The hardware-floor check is skipped (no curated `min_ram_gb` to compare against), so the user is responsible for picking weights that fit.\n\n```bash\nflock model add hf:Qwen/Qwen3-72B-AWQ # vLLM / MLX / llama-server\nflock model add hf:bartowski/Phi-3-mini-GGUF:Phi-3-mini-4k-instruct.Q4_K_M.gguf\nflock model add ollama:phi3:mini # ollama engine only\nflock model add file:/abs/path/my-finetune.gguf # llama-server / MLX, pre-downloaded\n```\n\nThe dashboard's **Models** tab has the same input under \"Add custom model\".\n\n---\n\n## Table of contents (curated walkthroughs)\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n- [`nomic-embed-text`](#nomic-embed-text--embeddings-for-rag) — embeddings for RAG\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## `nomic-embed-text` — embeddings for RAG\n\n**Why pick it.** It's the default embedding model in Flock's catalog — Apache-2.0 licensed, 137 M parameters, 768-dimensional vectors, 8K-token context window. Beats OpenAI `text-embedding-ada-002` on MTEB benchmarks. If you're wiring Flock into a RAG pipeline (LangChain, LlamaIndex, Haystack, raw vector DB), this is the one to start with.\n\n**Picker fit.** Any machine — it needs ~270 MB on disk and 2 GB RAM. Runs alongside your chat model with no extra hardware budget.\n\n**Install.**\n\n```bash\nflock model add nomic-embed-text\n```\n\n**Use via API (OpenAI embeddings shape).**\n\n```bash\nKEY=\"sk-orc-...\"\n\ncurl http://localhost:8080/v1/embeddings \\\n -H \"Authorization: Bearer $KEY\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\":\"nomic-embed-text\",\"input\":\"the quick brown fox\"}'\n```\n\nResponse (truncated):\n\n```json\n{\n \"object\": \"list\",\n \"data\": [{\"object\":\"embedding\",\"index\":0,\"embedding\":[0.0214, -0.0179, ...768 floats...]}],\n \"model\": \"nomic-embed-text\",\n \"usage\": {\"prompt_tokens\": 5, \"total_tokens\": 5}\n}\n```\n\n**Use via SDK.**\n\n```python\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.embeddings.create(model=\"nomic-embed-text\", input=[\"hello\", \"world\"])\nprint(len(resp.data[0].embedding)) # → 768\n```\n\nDrop-in for any code that calls OpenAI `text-embedding-3-small` / `text-embedding-ada-002` — same request and response shape.\n\n**Use via LangChain / LlamaIndex.** Point the `OpenAIEmbeddings` wrapper at `http://localhost:8080/v1` with your `sk-orc-*` key and `model=\"nomic-embed-text\"`. No other code changes.\n\n**What it doesn't do.** This is an embedding-only model. Don't try `/v1/chat/completions` with it — it won't respond. Pair it with a chat model (Qwen, Llama, Gemma) for the generation half of your RAG stack.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "9c2cc772636a3911eae26a783faf6e67143ac062": {"t": 1785770949.2278998, "n": 27879, "h": "01ab510d00d3f816", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 19 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## 🎯 Picker table — what to install\n\nScan the column that matches your hardware, then pick by use case. ⭐ = recommended starting point in each row. Catalog has **37 models** at last update (run `flock model search` for the live list, or `flock model search --sort=released` for newest-first).\n\n| Model ID | Size | Min RAM | Chat | Code | Reasoning | Vision | Audio | Long ctx | License | Notes |\n| ------------------------------ | ------ | ------- | :--: | :--: | :-------: | :----: | :---: | :------: | ----------- | -------------------------------------- |\n| **Edge — laptop / Raspberry-Pi-ish** | | | | | | | | | | |\n| `llama-3.2-1b` | 1.3 GB | 2 GB | • | | | | | | Llama 3.2 | Smoke test only |\n| `moondream3` | 1.2 GB | 4 GB | ● | | | ● | | | Apache-2.0 | Tiny vision-language, runs on Pi |\n| `llama-3.2-3b` ⭐ | 2.0 GB | 4 GB | ● | | | | | | Llama 3.2 | Edge default |\n| **Small — 8-16 GB box** | | | | | | | | | | |\n| `mimo-7b` | 4.5 GB | 8 GB | ● | | ●● | | | | Apache-2.0 | Xiaomi reasoning-focused 7B |\n| `qwen-coder-7b` | 4.7 GB | 8 GB | ● | ● | | | | | Apache-2.0 | FIM-capable, older |\n| `mimo-vl-7b` | 4.8 GB | 8 GB | ● | | | ● | | | Apache-2.0 | Small vision-language |\n| `mimo-audio` | 4.8 GB | 8 GB | ● | | | | ● | | Apache-2.0 | Speech + audio understanding |\n| `deepseek-r1-8b` | 4.9 GB | 12 GB | ● | | ●● | | | | MIT | Distilled reasoning |\n| `lfm2.5-8b-a1b` ⭐ | 5.0 GB | 8 GB | ● | | ●● | | | ●● | LFM Open | Best on-device MoE (1B active) |\n| `qwen3-8b` | 5.2 GB | 12 GB | ● | | | | | | Apache-2.0 | General chat |\n| `qwen3-vl-8b` | 5.5 GB | 10 GB | ● | ● | | ●● | | | Apache-2.0 | Vision + tools (charts, OCR, UI) |\n| `gemma4-e2b` | 7.2 GB | 8 GB | ● | | | ● | ● | ●● | Gemma | Mobile/edge multimodal (T/I/A) |\n| `mellum2-12b` | 7.0 GB | 12 GB | ● | ●● | ●● | | | | Apache-2.0 | JetBrains MoE coder (2.5B active) |\n| `mistral-nemo-12b` | 7.1 GB | 12 GB | ● | | | | | ●● | Apache-2.0 | 128K context |\n| `gemma4-12b` | 7.6 GB | 12 GB | ● | | | ●● | ● | ●● | Gemma | Encoder-free any-to-any (T/I/A/V) |\n| `pixtral-12b` | 7.8 GB | 16 GB | ● | | | ●● | | | Apache-2.0 | Mistral vision-language |\n| `qwen-coder-14b` | 9.0 GB | 16 GB | ● | ●● | | | | | Apache-2.0 | Dense code+agent |\n| `qwen3-14b` | 9.0 GB | 16 GB | ● | | | | | | Apache-2.0 | More capable Qwen3 chat |\n| `phi-4-14b` | 9.1 GB | 12 GB | ● | | ●● | | | | MIT | Strong reasoning per byte |\n| `gemma4-e4b` | 9.6 GB | 12 GB | ● | | | ● | ● | ●● | Gemma | Mobile/edge multimodal, larger E-tier |\n| **Mid — 24-32 GB box** | | | | | | | | | | |\n| `gpt-oss-20b` ⭐ | 14 GB | 16 GB | ● | ● | ●● | | | ● | Apache-2.0 | OpenAI open-weight, adjustable thinking |\n| `qwen3.6-27b` ⭐ | 17 GB | 24 GB | ●● | ●● | ● | | | ●● | Apache-2.0 | 77 % SWE-bench, top consumer pick |\n| `gemma4-26b` | 18 GB | 24 GB | ● | | | ● | | ●● | Gemma | MoE 4B-active, multimodal |\n| `qwen3-30b` | 19 GB | 24 GB | ●● | | | | | ●● | Apache-2.0 | MoE 3B-active, very fast |\n| `qwen3-coder-30b` | 19 GB | 24 GB | ● | ●● | | | | ●● | Apache-2.0 | MoE 3.3B-active coder |\n| `qwen-coder-32b` | 20 GB | 32 GB | ● | ●● | | | | | Apache-2.0 | Dense, older but proven |\n| `qwen3-vl-32b` | 20 GB | 32 GB | ● | ● | | ●●● | | | Apache-2.0 | Frontier-tier vision-language |\n| `gemma4-31b` | 20 GB | 32 GB | ● | | | ●● | | ●● | Gemma | Multimodal, larger than 26B |\n| **Power user — single 80 GB GPU / sharded** | | | | | | | | | | |\n| `llama-3.3-70b-sharded` | 43 GB | 48 GB | ●● | | | | | ●● | Llama 3.3 | Needs ≥2 nodes |\n| `gpt-oss-120b` | 65 GB | 80 GB | ●● | ● | ●●● | | | ● | Apache-2.0 | ≈ o4-mini reasoning, single H100 |\n| `llama-4-scout` | 67 GB | 80 GB | ●● | | | ●● | | ●●● | Llama 4 | 10M context, multimodal |\n| **Frontier — multi-machine sharded** | | | | | | | | | | |\n| `step-3.7-flash-sharded` ⭐ | 100 GB | 128 GB | ●● | ●● | ●● | ●● | | ●● | Apache-2.0 | 11B active VLM, fastest frontier MoE |\n| `deepseek-v4-flash-sharded` ⭐ | 150 GB | 160 GB | ●● | ●● | ●●● | | | ●● | MIT | 13B active = fast at frontier quality |\n| `nemotron-3-ultra-sharded` | 280 GB | 320 GB | ●● | ●● | ●●● | | | ●●● | NVIDIA Open | Hybrid Mamba-MoE, 1M ctx, MMLU 89.1 |\n| `glm-5.1-sharded` | 400 GB | 416 GB | ●● | ●●● | ●● | | | ●● | MIT | Best agentic coder |\n| `kimi-k2.6-sharded` | 500 GB | 512 GB | ●● | ●●● | ●● | | | ●● | Mod. MIT | #1 open coding benchmarks |\n\n**Legend** — • basic / ● good / ●● strong / ●●● best-in-tier · ⭐ recommended starting point\n\n> Sizes assume Q4_K_M (Ollama's default). Sharded entries assume Q4 baseline; unsloth's 2-bit dynamic GGUFs cut them roughly in half.\n\n**Quick rules of thumb:**\n- **I just want to try Flock** → `llama-3.2-3b` (2 GB, runs anywhere).\n- **Best on-device edge MoE** → `lfm2.5-8b-a1b` — only 1 B active, MATH500 88.8, MLX-ready.\n- **My laptop coding agent** → `mellum2-12b` (Apache-2.0, MoE, 2.5 B active) on 12 GB or `qwen3-coder-30b` on 24 GB.\n- **Multimodal on a laptop (text/image/audio/video)** → `gemma4-12b` — encoder-free unified architecture.\n- **Best consumer general model** → `qwen3.6-27b` (24 GB) or `gpt-oss-20b` (16 GB).\n- **Reasoning-heavy work** → `gpt-oss-20b` (small) or `gpt-oss-120b` (big).\n- **My team has 4 Mac Studios** → install the same model (e.g. `qwen3.6-27b`) on each — Flock load-balances automatically. No sharding needed for throughput.\n- **A model bigger than any one machine** → sharded tier. Start with `step-3.7-flash-sharded` (Apache-2.0) or `deepseek-v4-flash-sharded` (MIT).\n\nFor per-model walkthroughs (with detailed install + client snippets), the curated set below covers the longest-standing entries. For the rest of the catalog (10+ newer additions including Gemma 4 family, MiMo, Qwen3-VL, Pixtral, Moondream), run `flock model info <id>` — same fields as the table above, plus engine compatibility, license, and release date.\n\n---\n\n## How to choose\n\nUse the [picker table above](#-picker-table--what-to-install). It covers every model in the catalog with size, RAM, capability ratings, and license — pick the row that matches your hardware, then go by use case.\n\nThe per-model walkthroughs below cover a curated subset with full install + client snippets. For every other catalog entry, run `flock model info <id>` for the same metadata (size, RAM, capabilities, license, release date, engine compatibility), or open a [catalog issue](https://github.com/hadihonarvar/flock/issues/new?template=catalog_request.yml) if you'd like a full walkthrough added.\n\n---\n\n## Table of contents (curated walkthroughs)\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n- [`nomic-embed-text`](#nomic-embed-text--embeddings-for-rag) — embeddings for RAG\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## `nomic-embed-text` — embeddings for RAG\n\n**Why pick it.** It's the default embedding model in Flock's catalog — Apache-2.0 licensed, 137 M parameters, 768-dimensional vectors, 8K-token context window. Beats OpenAI `text-embedding-ada-002` on MTEB benchmarks. If you're wiring Flock into a RAG pipeline (LangChain, LlamaIndex, Haystack, raw vector DB), this is the one to start with.\n\n**Picker fit.** Any machine — it needs ~270 MB on disk and 2 GB RAM. Runs alongside your chat model with no extra hardware budget.\n\n**Install.**\n\n```bash\nflock model add nomic-embed-text\n```\n\n**Use via API (OpenAI embeddings shape).**\n\n```bash\nKEY=\"sk-orc-...\"\n\ncurl http://localhost:8080/v1/embeddings \\\n -H \"Authorization: Bearer $KEY\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\":\"nomic-embed-text\",\"input\":\"the quick brown fox\"}'\n```\n\nResponse (truncated):\n\n```json\n{\n \"object\": \"list\",\n \"data\": [{\"object\":\"embedding\",\"index\":0,\"embedding\":[0.0214, -0.0179, ...768 floats...]}],\n \"model\": \"nomic-embed-text\",\n \"usage\": {\"prompt_tokens\": 5, \"total_tokens\": 5}\n}\n```\n\n**Use via SDK.**\n\n```python\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.embeddings.create(model=\"nomic-embed-text\", input=[\"hello\", \"world\"])\nprint(len(resp.data[0].embedding)) # → 768\n```\n\nDrop-in for any code that calls OpenAI `text-embedding-3-small` / `text-embedding-ada-002` — same request and response shape.\n\n**Use via LangChain / LlamaIndex.** Point the `OpenAIEmbeddings` wrapper at `http://localhost:8080/v1` with your `sk-orc-*` key and `model=\"nomic-embed-text\"`. No other code changes.\n\n**What it doesn't do.** This is an embedding-only model. Don't try `/v1/chat/completions` with it — it won't respond. Pair it with a chat model (Qwen, Llama, Gemma) for the generation half of your RAG stack.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "e85d36916cb5e0278f94d46bf1e622c5c1a60fc6": {"t": 1785770950.7374163, "n": 26047, "h": "2686b6d6fe83ed3a", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 10 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## 🎯 Picker table — what to install\n\nScan the column that matches your hardware, then pick by use case. ⭐ = recommended starting point in each row. Catalog has **37 models** as of v0.21.0 (run `flock model search` for the live list).\n\n| Model ID | Size | Min RAM | Chat | Code | Reasoning | Vision | Audio | Long ctx | License | Notes |\n| ------------------------------ | ------ | ------- | :--: | :--: | :-------: | :----: | :---: | :------: | ----------- | -------------------------------------- |\n| **Edge — laptop / Raspberry-Pi-ish** | | | | | | | | | | |\n| `llama-3.2-1b` | 1.3 GB | 2 GB | • | | | | | | Llama 3.2 | Smoke test only |\n| `llama-3.2-3b` ⭐ | 2.0 GB | 4 GB | ● | | | | | | Llama 3.2 | Edge default |\n| **Small — 8-16 GB box** | | | | | | | | | | |\n| `qwen-coder-7b` | 4.7 GB | 8 GB | ● | ● | | | | | Apache-2.0 | FIM-capable, older |\n| `deepseek-r1-8b` | 4.9 GB | 12 GB | ● | | ●● | | | | MIT | Distilled reasoning |\n| `lfm2.5-8b-a1b` ⭐ | 5.0 GB | 8 GB | ● | | ●● | | | ●● | LFM Open | Best on-device MoE (1B active) |\n| `qwen3-8b` | 5.2 GB | 12 GB | ● | | | | | | Apache-2.0 | General chat |\n| `mellum2-12b` | 7.0 GB | 12 GB | ● | ●● | ●● | | | | Apache-2.0 | JetBrains MoE coder (2.5B active) |\n| `mistral-nemo-12b` | 7.1 GB | 12 GB | ● | | | | | ●● | Apache-2.0 | 128K context |\n| `gemma4-12b` | 7.6 GB | 12 GB | ● | | | ●● | ● | ●● | Gemma | Encoder-free any-to-any (T/I/A/V) |\n| `qwen-coder-14b` | 9.0 GB | 16 GB | ● | ●● | | | | | Apache-2.0 | Dense code+agent |\n| `qwen3-14b` | 9.0 GB | 16 GB | ● | | | | | | Apache-2.0 | More capable Qwen3 chat |\n| `phi-4-14b` | 9.1 GB | 12 GB | ● | | ●● | | | | MIT | Strong reasoning per byte |\n| **Mid — 24-32 GB box** | | | | | | | | | | |\n| `gpt-oss-20b` ⭐ | 14 GB | 16 GB | ● | ● | ●● | | | ● | Apache-2.0 | OpenAI open-weight, adjustable thinking |\n| `qwen3.6-27b` ⭐ | 17 GB | 24 GB | ●● | ●● | ● | | | ●● | Apache-2.0 | 77 % SWE-bench, top consumer pick |\n| `gemma4-26b` | 18 GB | 24 GB | ● | | | ● | | ●● | Gemma | MoE 4B-active, multimodal |\n| `qwen3-30b` | 19 GB | 24 GB | ●● | | | | | ●● | Apache-2.0 | MoE 3B-active, very fast |\n| `qwen3-coder-30b` | 19 GB | 24 GB | ● | ●● | | | | ●● | Apache-2.0 | MoE 3.3B-active coder |\n| `qwen-coder-32b` | 20 GB | 32 GB | ● | ●● | | | | | Apache-2.0 | Dense, older but proven |\n| **Power user — single 80 GB GPU / sharded** | | | | | | | | | | |\n| `llama-3.3-70b-sharded` | 43 GB | 48 GB | ●● | | | | | ●● | Llama 3.3 | Needs ≥2 nodes |\n| `gpt-oss-120b` | 65 GB | 80 GB | ●● | ● | ●●● | | | ● | Apache-2.0 | ≈ o4-mini reasoning, single H100 |\n| `llama-4-scout` | 67 GB | 80 GB | ●● | | | ●● | | ●●● | Llama 4 | 10M context, multimodal |\n| **Frontier — multi-machine sharded** | | | | | | | | | | |\n| `step-3.7-flash-sharded` ⭐ | 100 GB | 128 GB | ●● | ●● | ●● | ●● | | ●● | Apache-2.0 | 11B active VLM, fastest frontier MoE |\n| `deepseek-v4-flash-sharded` ⭐ | 150 GB | 160 GB | ●● | ●● | ●●● | | | ●● | MIT | 13B active = fast at frontier quality |\n| `nemotron-3-ultra-sharded` | 280 GB | 320 GB | ●● | ●● | ●●● | | | ●●● | NVIDIA Open | Hybrid Mamba-MoE, 1M ctx, MMLU 89.1 |\n| `glm-5.1-sharded` | 400 GB | 416 GB | ●● | ●●● | ●● | | | ●● | MIT | Best agentic coder |\n| `kimi-k2.6-sharded` | 500 GB | 512 GB | ●● | ●●● | ●● | | | ●● | Mod. MIT | #1 open coding benchmarks |\n\n**Legend** — • basic / ● good / ●● strong / ●●● best-in-tier · ⭐ recommended starting point\n\n> Sizes assume Q4_K_M (Ollama's default). Sharded entries assume Q4 baseline; unsloth's 2-bit dynamic GGUFs cut them roughly in half.\n\n**Quick rules of thumb:**\n- **I just want to try Flock** → `llama-3.2-3b` (2 GB, runs anywhere).\n- **Best on-device edge MoE** → `lfm2.5-8b-a1b` — only 1 B active, MATH500 88.8, MLX-ready.\n- **My laptop coding agent** → `mellum2-12b` (Apache-2.0, MoE, 2.5 B active) on 12 GB or `qwen3-coder-30b` on 24 GB.\n- **Multimodal on a laptop (text/image/audio/video)** → `gemma4-12b` — encoder-free unified architecture.\n- **Best consumer general model** → `qwen3.6-27b` (24 GB) or `gpt-oss-20b` (16 GB).\n- **Reasoning-heavy work** → `gpt-oss-20b` (small) or `gpt-oss-120b` (big).\n- **My team has 4 Mac Studios** → install the same model (e.g. `qwen3.6-27b`) on each — Flock load-balances automatically. No sharding needed for throughput.\n- **A model bigger than any one machine** → sharded tier. Start with `step-3.7-flash-sharded` (Apache-2.0) or `deepseek-v4-flash-sharded` (MIT).\n\nFor the full per-model walkthrough (the 9 original entries below have detailed install + client snippets), keep scrolling. For the 13 newer entries, run `flock model info <id>` — same fields as the table above, plus engine compatibility.\n\n---\n\n## How to choose\n\nUse the [picker table above](#-picker-table--what-to-install). It covers every model in the catalog with size, RAM, capability ratings, and license — pick the row that matches your hardware, then go by use case.\n\nThe per-model walkthroughs below (with full install + client snippets) cover the **9 original v0.1.0 entries**. For the **17 newer entries** added in v0.2 / v0.2.1, run `flock model info <id>` for the same metadata, or open a [catalog issue](https://github.com/hadihonarvar/flock/issues/new?template=catalog_request.yml) if you want a full walkthrough added.\n\n---\n\n## Table of contents (original v0.1.0 walkthroughs)\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n- [`nomic-embed-text`](#nomic-embed-text--embeddings-for-rag) — embeddings for RAG\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## `nomic-embed-text` — embeddings for RAG\n\n**Why pick it.** It's the default embedding model in Flock's catalog — Apache-2.0 licensed, 137 M parameters, 768-dimensional vectors, 8K-token context window. Beats OpenAI `text-embedding-ada-002` on MTEB benchmarks. If you're wiring Flock into a RAG pipeline (LangChain, LlamaIndex, Haystack, raw vector DB), this is the one to start with.\n\n**Picker fit.** Any machine — it needs ~270 MB on disk and 2 GB RAM. Runs alongside your chat model with no extra hardware budget.\n\n**Install.**\n\n```bash\nflock model add nomic-embed-text\n```\n\n**Use via API (OpenAI embeddings shape).**\n\n```bash\nKEY=\"sk-orc-...\"\n\ncurl http://localhost:8080/v1/embeddings \\\n -H \"Authorization: Bearer $KEY\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\":\"nomic-embed-text\",\"input\":\"the quick brown fox\"}'\n```\n\nResponse (truncated):\n\n```json\n{\n \"object\": \"list\",\n \"data\": [{\"object\":\"embedding\",\"index\":0,\"embedding\":[0.0214, -0.0179, ...768 floats...]}],\n \"model\": \"nomic-embed-text\",\n \"usage\": {\"prompt_tokens\": 5, \"total_tokens\": 5}\n}\n```\n\n**Use via SDK.**\n\n```python\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.embeddings.create(model=\"nomic-embed-text\", input=[\"hello\", \"world\"])\nprint(len(resp.data[0].embedding)) # → 768\n```\n\nDrop-in for any code that calls OpenAI `text-embedding-3-small` / `text-embedding-ada-002` — same request and response shape.\n\n**Use via LangChain / LlamaIndex.** Point the `OpenAIEmbeddings` wrapper at `http://localhost:8080/v1` with your `sk-orc-*` key and `model=\"nomic-embed-text\"`. No other code changes.\n\n**What it doesn't do.** This is an embedding-only model. Don't try `/v1/chat/completions` with it — it won't respond. Pair it with a chat model (Qwen, Llama, Gemma) for the generation half of your RAG stack.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "3b6caccad93d5ed6fb5e8b6361ecfefc4a350e49": {"t": 1785770953.1824276, "n": 26047, "h": "fb6abc803a589535", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 10 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## 🎯 Picker table — what to install\n\nScan the column that matches your hardware, then pick by use case. ⭐ = recommended starting point in each row. Catalog has **27 models** as of v0.21.0 (run `flock model search` for the live list).\n\n| Model ID | Size | Min RAM | Chat | Code | Reasoning | Vision | Audio | Long ctx | License | Notes |\n| ------------------------------ | ------ | ------- | :--: | :--: | :-------: | :----: | :---: | :------: | ----------- | -------------------------------------- |\n| **Edge — laptop / Raspberry-Pi-ish** | | | | | | | | | | |\n| `llama-3.2-1b` | 1.3 GB | 2 GB | • | | | | | | Llama 3.2 | Smoke test only |\n| `llama-3.2-3b` ⭐ | 2.0 GB | 4 GB | ● | | | | | | Llama 3.2 | Edge default |\n| **Small — 8-16 GB box** | | | | | | | | | | |\n| `qwen-coder-7b` | 4.7 GB | 8 GB | ● | ● | | | | | Apache-2.0 | FIM-capable, older |\n| `deepseek-r1-8b` | 4.9 GB | 12 GB | ● | | ●● | | | | MIT | Distilled reasoning |\n| `lfm2.5-8b-a1b` ⭐ | 5.0 GB | 8 GB | ● | | ●● | | | ●● | LFM Open | Best on-device MoE (1B active) |\n| `qwen3-8b` | 5.2 GB | 12 GB | ● | | | | | | Apache-2.0 | General chat |\n| `mellum2-12b` | 7.0 GB | 12 GB | ● | ●● | ●● | | | | Apache-2.0 | JetBrains MoE coder (2.5B active) |\n| `mistral-nemo-12b` | 7.1 GB | 12 GB | ● | | | | | ●● | Apache-2.0 | 128K context |\n| `gemma4-12b` | 7.6 GB | 12 GB | ● | | | ●● | ● | ●● | Gemma | Encoder-free any-to-any (T/I/A/V) |\n| `qwen-coder-14b` | 9.0 GB | 16 GB | ● | ●● | | | | | Apache-2.0 | Dense code+agent |\n| `qwen3-14b` | 9.0 GB | 16 GB | ● | | | | | | Apache-2.0 | More capable Qwen3 chat |\n| `phi-4-14b` | 9.1 GB | 12 GB | ● | | ●● | | | | MIT | Strong reasoning per byte |\n| **Mid — 24-32 GB box** | | | | | | | | | | |\n| `gpt-oss-20b` ⭐ | 14 GB | 16 GB | ● | ● | ●● | | | ● | Apache-2.0 | OpenAI open-weight, adjustable thinking |\n| `qwen3.6-27b` ⭐ | 17 GB | 24 GB | ●● | ●● | ● | | | ●● | Apache-2.0 | 77 % SWE-bench, top consumer pick |\n| `gemma4-26b` | 18 GB | 24 GB | ● | | | ● | | ●● | Gemma | MoE 4B-active, multimodal |\n| `qwen3-30b` | 19 GB | 24 GB | ●● | | | | | ●● | Apache-2.0 | MoE 3B-active, very fast |\n| `qwen3-coder-30b` | 19 GB | 24 GB | ● | ●● | | | | ●● | Apache-2.0 | MoE 3.3B-active coder |\n| `qwen-coder-32b` | 20 GB | 32 GB | ● | ●● | | | | | Apache-2.0 | Dense, older but proven |\n| **Power user — single 80 GB GPU / sharded** | | | | | | | | | | |\n| `llama-3.3-70b-sharded` | 43 GB | 48 GB | ●● | | | | | ●● | Llama 3.3 | Needs ≥2 nodes |\n| `gpt-oss-120b` | 65 GB | 80 GB | ●● | ● | ●●● | | | ● | Apache-2.0 | ≈ o4-mini reasoning, single H100 |\n| `llama-4-scout` | 67 GB | 80 GB | ●● | | | ●● | | ●●● | Llama 4 | 10M context, multimodal |\n| **Frontier — multi-machine sharded** | | | | | | | | | | |\n| `step-3.7-flash-sharded` ⭐ | 100 GB | 128 GB | ●● | ●● | ●● | ●● | | ●● | Apache-2.0 | 11B active VLM, fastest frontier MoE |\n| `deepseek-v4-flash-sharded` ⭐ | 150 GB | 160 GB | ●● | ●● | ●●● | | | ●● | MIT | 13B active = fast at frontier quality |\n| `nemotron-3-ultra-sharded` | 280 GB | 320 GB | ●● | ●● | ●●● | | | ●●● | NVIDIA Open | Hybrid Mamba-MoE, 1M ctx, MMLU 89.1 |\n| `glm-5.1-sharded` | 400 GB | 416 GB | ●● | ●●● | ●● | | | ●● | MIT | Best agentic coder |\n| `kimi-k2.6-sharded` | 500 GB | 512 GB | ●● | ●●● | ●● | | | ●● | Mod. MIT | #1 open coding benchmarks |\n\n**Legend** — • basic / ● good / ●● strong / ●●● best-in-tier · ⭐ recommended starting point\n\n> Sizes assume Q4_K_M (Ollama's default). Sharded entries assume Q4 baseline; unsloth's 2-bit dynamic GGUFs cut them roughly in half.\n\n**Quick rules of thumb:**\n- **I just want to try Flock** → `llama-3.2-3b` (2 GB, runs anywhere).\n- **Best on-device edge MoE** → `lfm2.5-8b-a1b` — only 1 B active, MATH500 88.8, MLX-ready.\n- **My laptop coding agent** → `mellum2-12b` (Apache-2.0, MoE, 2.5 B active) on 12 GB or `qwen3-coder-30b` on 24 GB.\n- **Multimodal on a laptop (text/image/audio/video)** → `gemma4-12b` — encoder-free unified architecture.\n- **Best consumer general model** → `qwen3.6-27b` (24 GB) or `gpt-oss-20b` (16 GB).\n- **Reasoning-heavy work** → `gpt-oss-20b` (small) or `gpt-oss-120b` (big).\n- **My team has 4 Mac Studios** → install the same model (e.g. `qwen3.6-27b`) on each — Flock load-balances automatically. No sharding needed for throughput.\n- **A model bigger than any one machine** → sharded tier. Start with `step-3.7-flash-sharded` (Apache-2.0) or `deepseek-v4-flash-sharded` (MIT).\n\nFor the full per-model walkthrough (the 9 original entries below have detailed install + client snippets), keep scrolling. For the 13 newer entries, run `flock model info <id>` — same fields as the table above, plus engine compatibility.\n\n---\n\n## How to choose\n\nUse the [picker table above](#-picker-table--what-to-install). It covers every model in the catalog with size, RAM, capability ratings, and license — pick the row that matches your hardware, then go by use case.\n\nThe per-model walkthroughs below (with full install + client snippets) cover the **9 original v0.1.0 entries**. For the **17 newer entries** added in v0.2 / v0.2.1, run `flock model info <id>` for the same metadata, or open a [catalog issue](https://github.com/hadihonarvar/flock/issues/new?template=catalog_request.yml) if you want a full walkthrough added.\n\n---\n\n## Table of contents (original v0.1.0 walkthroughs)\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n- [`nomic-embed-text`](#nomic-embed-text--embeddings-for-rag) — embeddings for RAG\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## `nomic-embed-text` — embeddings for RAG\n\n**Why pick it.** It's the default embedding model in Flock's catalog — Apache-2.0 licensed, 137 M parameters, 768-dimensional vectors, 8K-token context window. Beats OpenAI `text-embedding-ada-002` on MTEB benchmarks. If you're wiring Flock into a RAG pipeline (LangChain, LlamaIndex, Haystack, raw vector DB), this is the one to start with.\n\n**Picker fit.** Any machine — it needs ~270 MB on disk and 2 GB RAM. Runs alongside your chat model with no extra hardware budget.\n\n**Install.**\n\n```bash\nflock model add nomic-embed-text\n```\n\n**Use via API (OpenAI embeddings shape).**\n\n```bash\nKEY=\"sk-orc-...\"\n\ncurl http://localhost:8080/v1/embeddings \\\n -H \"Authorization: Bearer $KEY\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\":\"nomic-embed-text\",\"input\":\"the quick brown fox\"}'\n```\n\nResponse (truncated):\n\n```json\n{\n \"object\": \"list\",\n \"data\": [{\"object\":\"embedding\",\"index\":0,\"embedding\":[0.0214, -0.0179, ...768 floats...]}],\n \"model\": \"nomic-embed-text\",\n \"usage\": {\"prompt_tokens\": 5, \"total_tokens\": 5}\n}\n```\n\n**Use via SDK.**\n\n```python\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.embeddings.create(model=\"nomic-embed-text\", input=[\"hello\", \"world\"])\nprint(len(resp.data[0].embedding)) # → 768\n```\n\nDrop-in for any code that calls OpenAI `text-embedding-3-small` / `text-embedding-ada-002` — same request and response shape.\n\n**Use via LangChain / LlamaIndex.** Point the `OpenAIEmbeddings` wrapper at `http://localhost:8080/v1` with your `sk-orc-*` key and `model=\"nomic-embed-text\"`. No other code changes.\n\n**What it doesn't do.** This is an embedding-only model. Don't try `/v1/chat/completions` with it — it won't respond. Pair it with a chat model (Qwen, Llama, Gemma) for the generation half of your RAG stack.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "9c712b3de5f8c9432e958e4b7884ea31ef3b750d": {"t": 1785770955.2930646, "n": 26046, "h": "c690720b0267d40c", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 10 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## 🎯 Picker table — what to install\n\nScan the column that matches your hardware, then pick by use case. ⭐ = recommended starting point in each row. Catalog has **27 models** as of v0.5.0 (run `flock model search` for the live list).\n\n| Model ID | Size | Min RAM | Chat | Code | Reasoning | Vision | Audio | Long ctx | License | Notes |\n| ------------------------------ | ------ | ------- | :--: | :--: | :-------: | :----: | :---: | :------: | ----------- | -------------------------------------- |\n| **Edge — laptop / Raspberry-Pi-ish** | | | | | | | | | | |\n| `llama-3.2-1b` | 1.3 GB | 2 GB | • | | | | | | Llama 3.2 | Smoke test only |\n| `llama-3.2-3b` ⭐ | 2.0 GB | 4 GB | ● | | | | | | Llama 3.2 | Edge default |\n| **Small — 8-16 GB box** | | | | | | | | | | |\n| `qwen-coder-7b` | 4.7 GB | 8 GB | ● | ● | | | | | Apache-2.0 | FIM-capable, older |\n| `deepseek-r1-8b` | 4.9 GB | 12 GB | ● | | ●● | | | | MIT | Distilled reasoning |\n| `lfm2.5-8b-a1b` ⭐ | 5.0 GB | 8 GB | ● | | ●● | | | ●● | LFM Open | Best on-device MoE (1B active) |\n| `qwen3-8b` | 5.2 GB | 12 GB | ● | | | | | | Apache-2.0 | General chat |\n| `mellum2-12b` | 7.0 GB | 12 GB | ● | ●● | ●● | | | | Apache-2.0 | JetBrains MoE coder (2.5B active) |\n| `mistral-nemo-12b` | 7.1 GB | 12 GB | ● | | | | | ●● | Apache-2.0 | 128K context |\n| `gemma4-12b` | 7.6 GB | 12 GB | ● | | | ●● | ● | ●● | Gemma | Encoder-free any-to-any (T/I/A/V) |\n| `qwen-coder-14b` | 9.0 GB | 16 GB | ● | ●● | | | | | Apache-2.0 | Dense code+agent |\n| `qwen3-14b` | 9.0 GB | 16 GB | ● | | | | | | Apache-2.0 | More capable Qwen3 chat |\n| `phi-4-14b` | 9.1 GB | 12 GB | ● | | ●● | | | | MIT | Strong reasoning per byte |\n| **Mid — 24-32 GB box** | | | | | | | | | | |\n| `gpt-oss-20b` ⭐ | 14 GB | 16 GB | ● | ● | ●● | | | ● | Apache-2.0 | OpenAI open-weight, adjustable thinking |\n| `qwen3.6-27b` ⭐ | 17 GB | 24 GB | ●● | ●● | ● | | | ●● | Apache-2.0 | 77 % SWE-bench, top consumer pick |\n| `gemma4-26b` | 18 GB | 24 GB | ● | | | ● | | ●● | Gemma | MoE 4B-active, multimodal |\n| `qwen3-30b` | 19 GB | 24 GB | ●● | | | | | ●● | Apache-2.0 | MoE 3B-active, very fast |\n| `qwen3-coder-30b` | 19 GB | 24 GB | ● | ●● | | | | ●● | Apache-2.0 | MoE 3.3B-active coder |\n| `qwen-coder-32b` | 20 GB | 32 GB | ● | ●● | | | | | Apache-2.0 | Dense, older but proven |\n| **Power user — single 80 GB GPU / sharded** | | | | | | | | | | |\n| `llama-3.3-70b-sharded` | 43 GB | 48 GB | ●● | | | | | ●● | Llama 3.3 | Needs ≥2 nodes |\n| `gpt-oss-120b` | 65 GB | 80 GB | ●● | ● | ●●● | | | ● | Apache-2.0 | ≈ o4-mini reasoning, single H100 |\n| `llama-4-scout` | 67 GB | 80 GB | ●● | | | ●● | | ●●● | Llama 4 | 10M context, multimodal |\n| **Frontier — multi-machine sharded** | | | | | | | | | | |\n| `step-3.7-flash-sharded` ⭐ | 100 GB | 128 GB | ●● | ●● | ●● | ●● | | ●● | Apache-2.0 | 11B active VLM, fastest frontier MoE |\n| `deepseek-v4-flash-sharded` ⭐ | 150 GB | 160 GB | ●● | ●● | ●●● | | | ●● | MIT | 13B active = fast at frontier quality |\n| `nemotron-3-ultra-sharded` | 280 GB | 320 GB | ●● | ●● | ●●● | | | ●●● | NVIDIA Open | Hybrid Mamba-MoE, 1M ctx, MMLU 89.1 |\n| `glm-5.1-sharded` | 400 GB | 416 GB | ●● | ●●● | ●● | | | ●● | MIT | Best agentic coder |\n| `kimi-k2.6-sharded` | 500 GB | 512 GB | ●● | ●●● | ●● | | | ●● | Mod. MIT | #1 open coding benchmarks |\n\n**Legend** — • basic / ● good / ●● strong / ●●● best-in-tier · ⭐ recommended starting point\n\n> Sizes assume Q4_K_M (Ollama's default). Sharded entries assume Q4 baseline; unsloth's 2-bit dynamic GGUFs cut them roughly in half.\n\n**Quick rules of thumb:**\n- **I just want to try Flock** → `llama-3.2-3b` (2 GB, runs anywhere).\n- **Best on-device edge MoE** → `lfm2.5-8b-a1b` — only 1 B active, MATH500 88.8, MLX-ready.\n- **My laptop coding agent** → `mellum2-12b` (Apache-2.0, MoE, 2.5 B active) on 12 GB or `qwen3-coder-30b` on 24 GB.\n- **Multimodal on a laptop (text/image/audio/video)** → `gemma4-12b` — encoder-free unified architecture.\n- **Best consumer general model** → `qwen3.6-27b` (24 GB) or `gpt-oss-20b` (16 GB).\n- **Reasoning-heavy work** → `gpt-oss-20b` (small) or `gpt-oss-120b` (big).\n- **My team has 4 Mac Studios** → install the same model (e.g. `qwen3.6-27b`) on each — Flock load-balances automatically. No sharding needed for throughput.\n- **A model bigger than any one machine** → sharded tier. Start with `step-3.7-flash-sharded` (Apache-2.0) or `deepseek-v4-flash-sharded` (MIT).\n\nFor the full per-model walkthrough (the 9 original entries below have detailed install + client snippets), keep scrolling. For the 13 newer entries, run `flock model info <id>` — same fields as the table above, plus engine compatibility.\n\n---\n\n## How to choose\n\nUse the [picker table above](#-picker-table--what-to-install). It covers every model in the catalog with size, RAM, capability ratings, and license — pick the row that matches your hardware, then go by use case.\n\nThe per-model walkthroughs below (with full install + client snippets) cover the **9 original v0.1.0 entries**. For the **17 newer entries** added in v0.2 / v0.2.1, run `flock model info <id>` for the same metadata, or open a [catalog issue](https://github.com/hadihonarvar/flock/issues/new?template=catalog_request.yml) if you want a full walkthrough added.\n\n---\n\n## Table of contents (original v0.1.0 walkthroughs)\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n- [`nomic-embed-text`](#nomic-embed-text--embeddings-for-rag) — embeddings for RAG\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## `nomic-embed-text` — embeddings for RAG\n\n**Why pick it.** It's the default embedding model in Flock's catalog — Apache-2.0 licensed, 137 M parameters, 768-dimensional vectors, 8K-token context window. Beats OpenAI `text-embedding-ada-002` on MTEB benchmarks. If you're wiring Flock into a RAG pipeline (LangChain, LlamaIndex, Haystack, raw vector DB), this is the one to start with.\n\n**Picker fit.** Any machine — it needs ~270 MB on disk and 2 GB RAM. Runs alongside your chat model with no extra hardware budget.\n\n**Install.**\n\n```bash\nflock model add nomic-embed-text\n```\n\n**Use via API (OpenAI embeddings shape).**\n\n```bash\nKEY=\"sk-orc-...\"\n\ncurl http://localhost:8080/v1/embeddings \\\n -H \"Authorization: Bearer $KEY\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\":\"nomic-embed-text\",\"input\":\"the quick brown fox\"}'\n```\n\nResponse (truncated):\n\n```json\n{\n \"object\": \"list\",\n \"data\": [{\"object\":\"embedding\",\"index\":0,\"embedding\":[0.0214, -0.0179, ...768 floats...]}],\n \"model\": \"nomic-embed-text\",\n \"usage\": {\"prompt_tokens\": 5, \"total_tokens\": 5}\n}\n```\n\n**Use via SDK.**\n\n```python\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.embeddings.create(model=\"nomic-embed-text\", input=[\"hello\", \"world\"])\nprint(len(resp.data[0].embedding)) # → 768\n```\n\nDrop-in for any code that calls OpenAI `text-embedding-3-small` / `text-embedding-ada-002` — same request and response shape.\n\n**Use via LangChain / LlamaIndex.** Point the `OpenAIEmbeddings` wrapper at `http://localhost:8080/v1` with your `sk-orc-*` key and `model=\"nomic-embed-text\"`. No other code changes.\n\n**What it doesn't do.** This is an embedding-only model. Don't try `/v1/chat/completions` with it — it won't respond. Pair it with a chat model (Qwen, Llama, Gemma) for the generation half of your RAG stack.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "82c8aa111e94cdd784e100c6c723e23c92625096": {"t": 1785770956.7777743, "n": 26046, "h": "9d1f798e091e597f", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 10 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## 🎯 Picker table — what to install\n\nScan the column that matches your hardware, then pick by use case. ⭐ = recommended starting point in each row. Catalog has **27 models** as of v0.4.0 (run `flock model search` for the live list).\n\n| Model ID | Size | Min RAM | Chat | Code | Reasoning | Vision | Audio | Long ctx | License | Notes |\n| ------------------------------ | ------ | ------- | :--: | :--: | :-------: | :----: | :---: | :------: | ----------- | -------------------------------------- |\n| **Edge — laptop / Raspberry-Pi-ish** | | | | | | | | | | |\n| `llama-3.2-1b` | 1.3 GB | 2 GB | • | | | | | | Llama 3.2 | Smoke test only |\n| `llama-3.2-3b` ⭐ | 2.0 GB | 4 GB | ● | | | | | | Llama 3.2 | Edge default |\n| **Small — 8-16 GB box** | | | | | | | | | | |\n| `qwen-coder-7b` | 4.7 GB | 8 GB | ● | ● | | | | | Apache-2.0 | FIM-capable, older |\n| `deepseek-r1-8b` | 4.9 GB | 12 GB | ● | | ●● | | | | MIT | Distilled reasoning |\n| `lfm2.5-8b-a1b` ⭐ | 5.0 GB | 8 GB | ● | | ●● | | | ●● | LFM Open | Best on-device MoE (1B active) |\n| `qwen3-8b` | 5.2 GB | 12 GB | ● | | | | | | Apache-2.0 | General chat |\n| `mellum2-12b` | 7.0 GB | 12 GB | ● | ●● | ●● | | | | Apache-2.0 | JetBrains MoE coder (2.5B active) |\n| `mistral-nemo-12b` | 7.1 GB | 12 GB | ● | | | | | ●● | Apache-2.0 | 128K context |\n| `gemma4-12b` | 7.6 GB | 12 GB | ● | | | ●● | ● | ●● | Gemma | Encoder-free any-to-any (T/I/A/V) |\n| `qwen-coder-14b` | 9.0 GB | 16 GB | ● | ●● | | | | | Apache-2.0 | Dense code+agent |\n| `qwen3-14b` | 9.0 GB | 16 GB | ● | | | | | | Apache-2.0 | More capable Qwen3 chat |\n| `phi-4-14b` | 9.1 GB | 12 GB | ● | | ●● | | | | MIT | Strong reasoning per byte |\n| **Mid — 24-32 GB box** | | | | | | | | | | |\n| `gpt-oss-20b` ⭐ | 14 GB | 16 GB | ● | ● | ●● | | | ● | Apache-2.0 | OpenAI open-weight, adjustable thinking |\n| `qwen3.6-27b` ⭐ | 17 GB | 24 GB | ●● | ●● | ● | | | ●● | Apache-2.0 | 77 % SWE-bench, top consumer pick |\n| `gemma4-26b` | 18 GB | 24 GB | ● | | | ● | | ●● | Gemma | MoE 4B-active, multimodal |\n| `qwen3-30b` | 19 GB | 24 GB | ●● | | | | | ●● | Apache-2.0 | MoE 3B-active, very fast |\n| `qwen3-coder-30b` | 19 GB | 24 GB | ● | ●● | | | | ●● | Apache-2.0 | MoE 3.3B-active coder |\n| `qwen-coder-32b` | 20 GB | 32 GB | ● | ●● | | | | | Apache-2.0 | Dense, older but proven |\n| **Power user — single 80 GB GPU / sharded** | | | | | | | | | | |\n| `llama-3.3-70b-sharded` | 43 GB | 48 GB | ●● | | | | | ●● | Llama 3.3 | Needs ≥2 nodes |\n| `gpt-oss-120b` | 65 GB | 80 GB | ●● | ● | ●●● | | | ● | Apache-2.0 | ≈ o4-mini reasoning, single H100 |\n| `llama-4-scout` | 67 GB | 80 GB | ●● | | | ●● | | ●●● | Llama 4 | 10M context, multimodal |\n| **Frontier — multi-machine sharded** | | | | | | | | | | |\n| `step-3.7-flash-sharded` ⭐ | 100 GB | 128 GB | ●● | ●● | ●● | ●● | | ●● | Apache-2.0 | 11B active VLM, fastest frontier MoE |\n| `deepseek-v4-flash-sharded` ⭐ | 150 GB | 160 GB | ●● | ●● | ●●● | | | ●● | MIT | 13B active = fast at frontier quality |\n| `nemotron-3-ultra-sharded` | 280 GB | 320 GB | ●● | ●● | ●●● | | | ●●● | NVIDIA Open | Hybrid Mamba-MoE, 1M ctx, MMLU 89.1 |\n| `glm-5.1-sharded` | 400 GB | 416 GB | ●● | ●●● | ●● | | | ●● | MIT | Best agentic coder |\n| `kimi-k2.6-sharded` | 500 GB | 512 GB | ●● | ●●● | ●● | | | ●● | Mod. MIT | #1 open coding benchmarks |\n\n**Legend** — • basic / ● good / ●● strong / ●●● best-in-tier · ⭐ recommended starting point\n\n> Sizes assume Q4_K_M (Ollama's default). Sharded entries assume Q4 baseline; unsloth's 2-bit dynamic GGUFs cut them roughly in half.\n\n**Quick rules of thumb:**\n- **I just want to try Flock** → `llama-3.2-3b` (2 GB, runs anywhere).\n- **Best on-device edge MoE** → `lfm2.5-8b-a1b` — only 1 B active, MATH500 88.8, MLX-ready.\n- **My laptop coding agent** → `mellum2-12b` (Apache-2.0, MoE, 2.5 B active) on 12 GB or `qwen3-coder-30b` on 24 GB.\n- **Multimodal on a laptop (text/image/audio/video)** → `gemma4-12b` — encoder-free unified architecture.\n- **Best consumer general model** → `qwen3.6-27b` (24 GB) or `gpt-oss-20b` (16 GB).\n- **Reasoning-heavy work** → `gpt-oss-20b` (small) or `gpt-oss-120b` (big).\n- **My team has 4 Mac Studios** → install the same model (e.g. `qwen3.6-27b`) on each — Flock load-balances automatically. No sharding needed for throughput.\n- **A model bigger than any one machine** → sharded tier. Start with `step-3.7-flash-sharded` (Apache-2.0) or `deepseek-v4-flash-sharded` (MIT).\n\nFor the full per-model walkthrough (the 9 original entries below have detailed install + client snippets), keep scrolling. For the 13 newer entries, run `flock model info <id>` — same fields as the table above, plus engine compatibility.\n\n---\n\n## How to choose\n\nUse the [picker table above](#-picker-table--what-to-install). It covers every model in the catalog with size, RAM, capability ratings, and license — pick the row that matches your hardware, then go by use case.\n\nThe per-model walkthroughs below (with full install + client snippets) cover the **9 original v0.1.0 entries**. For the **17 newer entries** added in v0.2 / v0.2.1, run `flock model info <id>` for the same metadata, or open a [catalog issue](https://github.com/hadihonarvar/flock/issues/new?template=catalog_request.yml) if you want a full walkthrough added.\n\n---\n\n## Table of contents (original v0.1.0 walkthroughs)\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n- [`nomic-embed-text`](#nomic-embed-text--embeddings-for-rag) — embeddings for RAG\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## `nomic-embed-text` — embeddings for RAG\n\n**Why pick it.** It's the default embedding model in Flock's catalog — Apache-2.0 licensed, 137 M parameters, 768-dimensional vectors, 8K-token context window. Beats OpenAI `text-embedding-ada-002` on MTEB benchmarks. If you're wiring Flock into a RAG pipeline (LangChain, LlamaIndex, Haystack, raw vector DB), this is the one to start with.\n\n**Picker fit.** Any machine — it needs ~270 MB on disk and 2 GB RAM. Runs alongside your chat model with no extra hardware budget.\n\n**Install.**\n\n```bash\nflock model add nomic-embed-text\n```\n\n**Use via API (OpenAI embeddings shape).**\n\n```bash\nKEY=\"sk-orc-...\"\n\ncurl http://localhost:8080/v1/embeddings \\\n -H \"Authorization: Bearer $KEY\" \\\n -H \"Content-Type: application/json\" \\\n -d '{\"model\":\"nomic-embed-text\",\"input\":\"the quick brown fox\"}'\n```\n\nResponse (truncated):\n\n```json\n{\n \"object\": \"list\",\n \"data\": [{\"object\":\"embedding\",\"index\":0,\"embedding\":[0.0214, -0.0179, ...768 floats...]}],\n \"model\": \"nomic-embed-text\",\n \"usage\": {\"prompt_tokens\": 5, \"total_tokens\": 5}\n}\n```\n\n**Use via SDK.**\n\n```python\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.embeddings.create(model=\"nomic-embed-text\", input=[\"hello\", \"world\"])\nprint(len(resp.data[0].embedding)) # → 768\n```\n\nDrop-in for any code that calls OpenAI `text-embedding-3-small` / `text-embedding-ada-002` — same request and response shape.\n\n**Use via LangChain / LlamaIndex.** Point the `OpenAIEmbeddings` wrapper at `http://localhost:8080/v1` with your `sk-orc-*` key and `model=\"nomic-embed-text\"`. No other code changes.\n\n**What it doesn't do.** This is an embedding-only model. Don't try `/v1/chat/completions` with it — it won't respond. Pair it with a chat model (Qwen, Llama, Gemma) for the generation half of your RAG stack.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "ba019a063e9c0664d601928e71c2613c603cc414": {"t": 1785770958.6741645, "n": 24101, "h": "cc7187aa5346f3d1", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 10 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## 🎯 Picker table — what to install\n\nScan the column that matches your hardware, then pick by use case. ⭐ = recommended starting point in each row. Catalog has **27 models** as of v0.4.0 (run `flock model search` for the live list).\n\n| Model ID | Size | Min RAM | Chat | Code | Reasoning | Vision | Audio | Long ctx | License | Notes |\n| ------------------------------ | ------ | ------- | :--: | :--: | :-------: | :----: | :---: | :------: | ----------- | -------------------------------------- |\n| **Edge — laptop / Raspberry-Pi-ish** | | | | | | | | | | |\n| `llama-3.2-1b` | 1.3 GB | 2 GB | • | | | | | | Llama 3.2 | Smoke test only |\n| `llama-3.2-3b` ⭐ | 2.0 GB | 4 GB | ● | | | | | | Llama 3.2 | Edge default |\n| **Small — 8-16 GB box** | | | | | | | | | | |\n| `qwen-coder-7b` | 4.7 GB | 8 GB | ● | ● | | | | | Apache-2.0 | FIM-capable, older |\n| `deepseek-r1-8b` | 4.9 GB | 12 GB | ● | | ●● | | | | MIT | Distilled reasoning |\n| `lfm2.5-8b-a1b` ⭐ | 5.0 GB | 8 GB | ● | | ●● | | | ●● | LFM Open | Best on-device MoE (1B active) |\n| `qwen3-8b` | 5.2 GB | 12 GB | ● | | | | | | Apache-2.0 | General chat |\n| `mellum2-12b` | 7.0 GB | 12 GB | ● | ●● | ●● | | | | Apache-2.0 | JetBrains MoE coder (2.5B active) |\n| `mistral-nemo-12b` | 7.1 GB | 12 GB | ● | | | | | ●● | Apache-2.0 | 128K context |\n| `gemma4-12b` | 7.6 GB | 12 GB | ● | | | ●● | ● | ●● | Gemma | Encoder-free any-to-any (T/I/A/V) |\n| `qwen-coder-14b` | 9.0 GB | 16 GB | ● | ●● | | | | | Apache-2.0 | Dense code+agent |\n| `qwen3-14b` | 9.0 GB | 16 GB | ● | | | | | | Apache-2.0 | More capable Qwen3 chat |\n| `phi-4-14b` | 9.1 GB | 12 GB | ● | | ●● | | | | MIT | Strong reasoning per byte |\n| **Mid — 24-32 GB box** | | | | | | | | | | |\n| `gpt-oss-20b` ⭐ | 14 GB | 16 GB | ● | ● | ●● | | | ● | Apache-2.0 | OpenAI open-weight, adjustable thinking |\n| `qwen3.6-27b` ⭐ | 17 GB | 24 GB | ●● | ●● | ● | | | ●● | Apache-2.0 | 77 % SWE-bench, top consumer pick |\n| `gemma4-26b` | 18 GB | 24 GB | ● | | | ● | | ●● | Gemma | MoE 4B-active, multimodal |\n| `qwen3-30b` | 19 GB | 24 GB | ●● | | | | | ●● | Apache-2.0 | MoE 3B-active, very fast |\n| `qwen3-coder-30b` | 19 GB | 24 GB | ● | ●● | | | | ●● | Apache-2.0 | MoE 3.3B-active coder |\n| `qwen-coder-32b` | 20 GB | 32 GB | ● | ●● | | | | | Apache-2.0 | Dense, older but proven |\n| **Power user — single 80 GB GPU / sharded** | | | | | | | | | | |\n| `llama-3.3-70b-sharded` | 43 GB | 48 GB | ●● | | | | | ●● | Llama 3.3 | Needs ≥2 nodes |\n| `gpt-oss-120b` | 65 GB | 80 GB | ●● | ● | ●●● | | | ● | Apache-2.0 | ≈ o4-mini reasoning, single H100 |\n| `llama-4-scout` | 67 GB | 80 GB | ●● | | | ●● | | ●●● | Llama 4 | 10M context, multimodal |\n| **Frontier — multi-machine sharded** | | | | | | | | | | |\n| `step-3.7-flash-sharded` ⭐ | 100 GB | 128 GB | ●● | ●● | ●● | ●● | | ●● | Apache-2.0 | 11B active VLM, fastest frontier MoE |\n| `deepseek-v4-flash-sharded` ⭐ | 150 GB | 160 GB | ●● | ●● | ●●● | | | ●● | MIT | 13B active = fast at frontier quality |\n| `nemotron-3-ultra-sharded` | 280 GB | 320 GB | ●● | ●● | ●●● | | | ●●● | NVIDIA Open | Hybrid Mamba-MoE, 1M ctx, MMLU 89.1 |\n| `glm-5.1-sharded` | 400 GB | 416 GB | ●● | ●●● | ●● | | | ●● | MIT | Best agentic coder |\n| `kimi-k2.6-sharded` | 500 GB | 512 GB | ●● | ●●● | ●● | | | ●● | Mod. MIT | #1 open coding benchmarks |\n\n**Legend** — • basic / ● good / ●● strong / ●●● best-in-tier · ⭐ recommended starting point\n\n> Sizes assume Q4_K_M (Ollama's default). Sharded entries assume Q4 baseline; unsloth's 2-bit dynamic GGUFs cut them roughly in half.\n\n**Quick rules of thumb:**\n- **I just want to try Flock** → `llama-3.2-3b` (2 GB, runs anywhere).\n- **Best on-device edge MoE** → `lfm2.5-8b-a1b` — only 1 B active, MATH500 88.8, MLX-ready.\n- **My laptop coding agent** → `mellum2-12b` (Apache-2.0, MoE, 2.5 B active) on 12 GB or `qwen3-coder-30b` on 24 GB.\n- **Multimodal on a laptop (text/image/audio/video)** → `gemma4-12b` — encoder-free unified architecture.\n- **Best consumer general model** → `qwen3.6-27b` (24 GB) or `gpt-oss-20b` (16 GB).\n- **Reasoning-heavy work** → `gpt-oss-20b` (small) or `gpt-oss-120b` (big).\n- **My team has 4 Mac Studios** → install the same model (e.g. `qwen3.6-27b`) on each — Flock load-balances automatically. No sharding needed for throughput.\n- **A model bigger than any one machine** → sharded tier. Start with `step-3.7-flash-sharded` (Apache-2.0) or `deepseek-v4-flash-sharded` (MIT).\n\nFor the full per-model walkthrough (the 9 original entries below have detailed install + client snippets), keep scrolling. For the 13 newer entries, run `flock model info <id>` — same fields as the table above, plus engine compatibility.\n\n---\n\n## How to choose\n\nUse the [picker table above](#-picker-table--what-to-install). It covers every model in the catalog with size, RAM, capability ratings, and license — pick the row that matches your hardware, then go by use case.\n\nThe per-model walkthroughs below (with full install + client snippets) cover the **9 original v0.1.0 entries**. For the **17 newer entries** added in v0.2 / v0.2.1, run `flock model info <id>` for the same metadata, or open a [catalog issue](https://github.com/hadihonarvar/flock/issues/new?template=catalog_request.yml) if you want a full walkthrough added.\n\n---\n\n## Table of contents (original v0.1.0 walkthroughs)\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "d81a7c9c3ae36a9ed888c55750805a9870936131": {"t": 1785770960.2399564, "n": 24056, "h": "5f7e12fa4b3f7527", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 10 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## 🎯 Picker table — what to install\n\nScan the column that matches your hardware, then pick by use case. ⭐ = recommended starting point in each row. Catalog has **26 models** as of v0.2.1.\n\n| Model ID | Size | Min RAM | Chat | Code | Reasoning | Vision | Audio | Long ctx | License | Notes |\n| ------------------------------ | ------ | ------- | :--: | :--: | :-------: | :----: | :---: | :------: | ----------- | -------------------------------------- |\n| **Edge — laptop / Raspberry-Pi-ish** | | | | | | | | | | |\n| `llama-3.2-1b` | 1.3 GB | 2 GB | • | | | | | | Llama 3.2 | Smoke test only |\n| `llama-3.2-3b` ⭐ | 2.0 GB | 4 GB | ● | | | | | | Llama 3.2 | Edge default |\n| **Small — 8-16 GB box** | | | | | | | | | | |\n| `qwen-coder-7b` | 4.7 GB | 8 GB | ● | ● | | | | | Apache-2.0 | FIM-capable, older |\n| `deepseek-r1-8b` | 4.9 GB | 12 GB | ● | | ●● | | | | MIT | Distilled reasoning |\n| `lfm2.5-8b-a1b` ⭐ | 5.0 GB | 8 GB | ● | | ●● | | | ●● | LFM Open | Best on-device MoE (1B active) |\n| `qwen3-8b` | 5.2 GB | 12 GB | ● | | | | | | Apache-2.0 | General chat |\n| `mellum2-12b` | 7.0 GB | 12 GB | ● | ●● | ●● | | | | Apache-2.0 | JetBrains MoE coder (2.5B active) |\n| `mistral-nemo-12b` | 7.1 GB | 12 GB | ● | | | | | ●● | Apache-2.0 | 128K context |\n| `gemma4-12b` | 7.6 GB | 12 GB | ● | | | ●● | ● | ●● | Gemma | Encoder-free any-to-any (T/I/A/V) |\n| `qwen-coder-14b` | 9.0 GB | 16 GB | ● | ●● | | | | | Apache-2.0 | Dense code+agent |\n| `qwen3-14b` | 9.0 GB | 16 GB | ● | | | | | | Apache-2.0 | More capable Qwen3 chat |\n| `phi-4-14b` | 9.1 GB | 12 GB | ● | | ●● | | | | MIT | Strong reasoning per byte |\n| **Mid — 24-32 GB box** | | | | | | | | | | |\n| `gpt-oss-20b` ⭐ | 14 GB | 16 GB | ● | ● | ●● | | | ● | Apache-2.0 | OpenAI open-weight, adjustable thinking |\n| `qwen3.6-27b` ⭐ | 17 GB | 24 GB | ●● | ●● | ● | | | ●● | Apache-2.0 | 77 % SWE-bench, top consumer pick |\n| `gemma4-26b` | 18 GB | 24 GB | ● | | | ● | | ●● | Gemma | MoE 4B-active, multimodal |\n| `qwen3-30b` | 19 GB | 24 GB | ●● | | | | | ●● | Apache-2.0 | MoE 3B-active, very fast |\n| `qwen3-coder-30b` | 19 GB | 24 GB | ● | ●● | | | | ●● | Apache-2.0 | MoE 3.3B-active coder |\n| `qwen-coder-32b` | 20 GB | 32 GB | ● | ●● | | | | | Apache-2.0 | Dense, older but proven |\n| **Power user — single 80 GB GPU / sharded** | | | | | | | | | | |\n| `llama-3.3-70b-sharded` | 43 GB | 48 GB | ●● | | | | | ●● | Llama 3.3 | Needs ≥2 nodes |\n| `gpt-oss-120b` | 65 GB | 80 GB | ●● | ● | ●●● | | | ● | Apache-2.0 | ≈ o4-mini reasoning, single H100 |\n| `llama-4-scout` | 67 GB | 80 GB | ●● | | | ●● | | ●●● | Llama 4 | 10M context, multimodal |\n| **Frontier — multi-machine sharded** | | | | | | | | | | |\n| `step-3.7-flash-sharded` ⭐ | 100 GB | 128 GB | ●● | ●● | ●● | ●● | | ●● | Apache-2.0 | 11B active VLM, fastest frontier MoE |\n| `deepseek-v4-flash-sharded` ⭐ | 150 GB | 160 GB | ●● | ●● | ●●● | | | ●● | MIT | 13B active = fast at frontier quality |\n| `nemotron-3-ultra-sharded` | 280 GB | 320 GB | ●● | ●● | ●●● | | | ●●● | NVIDIA Open | Hybrid Mamba-MoE, 1M ctx, MMLU 89.1 |\n| `glm-5.1-sharded` | 400 GB | 416 GB | ●● | ●●● | ●● | | | ●● | MIT | Best agentic coder |\n| `kimi-k2.6-sharded` | 500 GB | 512 GB | ●● | ●●● | ●● | | | ●● | Mod. MIT | #1 open coding benchmarks |\n\n**Legend** — • basic / ● good / ●● strong / ●●● best-in-tier · ⭐ recommended starting point\n\n> Sizes assume Q4_K_M (Ollama's default). Sharded entries assume Q4 baseline; unsloth's 2-bit dynamic GGUFs cut them roughly in half.\n\n**Quick rules of thumb:**\n- **I just want to try Flock** → `llama-3.2-3b` (2 GB, runs anywhere).\n- **Best on-device edge MoE** → `lfm2.5-8b-a1b` — only 1 B active, MATH500 88.8, MLX-ready.\n- **My laptop coding agent** → `mellum2-12b` (Apache-2.0, MoE, 2.5 B active) on 12 GB or `qwen3-coder-30b` on 24 GB.\n- **Multimodal on a laptop (text/image/audio/video)** → `gemma4-12b` — encoder-free unified architecture.\n- **Best consumer general model** → `qwen3.6-27b` (24 GB) or `gpt-oss-20b` (16 GB).\n- **Reasoning-heavy work** → `gpt-oss-20b` (small) or `gpt-oss-120b` (big).\n- **My team has 4 Mac Studios** → install the same model (e.g. `qwen3.6-27b`) on each — Flock load-balances automatically. No sharding needed for throughput.\n- **A model bigger than any one machine** → sharded tier. Start with `step-3.7-flash-sharded` (Apache-2.0) or `deepseek-v4-flash-sharded` (MIT).\n\nFor the full per-model walkthrough (the 9 original entries below have detailed install + client snippets), keep scrolling. For the 13 newer entries, run `flock model info <id>` — same fields as the table above, plus engine compatibility.\n\n---\n\n## How to choose\n\nUse the [picker table above](#-picker-table--what-to-install). It covers every model in the catalog with size, RAM, capability ratings, and license — pick the row that matches your hardware, then go by use case.\n\nThe per-model walkthroughs below (with full install + client snippets) cover the **9 original v0.1.0 entries**. For the **17 newer entries** added in v0.2 / v0.2.1, run `flock model info <id>` for the same metadata, or open a [catalog issue](https://github.com/hadihonarvar/flock/issues/new?template=catalog_request.yml) if you want a full walkthrough added.\n\n---\n\n## Table of contents (original v0.1.0 walkthroughs)\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "9a325dd333c8162a1ee21f2b56ae8020df402bfb": {"t": 1785770962.7002435, "n": 25234, "h": "bfa4b5ae563d202d", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 10 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## 🎯 Picker table — what to install\n\nScan the column that matches your hardware, then pick by use case. ⭐ = recommended starting point in each row. Catalog has **26 models** as of v0.2.1.\n\n| Model ID | Size | Min RAM | Chat | Code | Reasoning | Vision | Audio | Long ctx | License | Notes |\n| ------------------------------ | ------ | ------- | :--: | :--: | :-------: | :----: | :---: | :------: | ----------- | -------------------------------------- |\n| **Edge — laptop / Raspberry-Pi-ish** | | | | | | | | | | |\n| `llama-3.2-1b` | 1.3 GB | 2 GB | • | | | | | | Llama 3.2 | Smoke test only |\n| `llama-3.2-3b` ⭐ | 2.0 GB | 4 GB | ● | | | | | | Llama 3.2 | Edge default |\n| **Small — 8-16 GB box** | | | | | | | | | | |\n| `qwen-coder-7b` | 4.7 GB | 8 GB | ● | ● | | | | | Apache-2.0 | FIM-capable, older |\n| `deepseek-r1-8b` | 4.9 GB | 12 GB | ● | | ●● | | | | MIT | Distilled reasoning |\n| `lfm2.5-8b-a1b` ⭐ | 5.0 GB | 8 GB | ● | | ●● | | | ●● | LFM Open | Best on-device MoE (1B active) |\n| `qwen3-8b` | 5.2 GB | 12 GB | ● | | | | | | Apache-2.0 | General chat |\n| `mellum2-12b` | 7.0 GB | 12 GB | ● | ●● | ●● | | | | Apache-2.0 | JetBrains MoE coder (2.5B active) |\n| `mistral-nemo-12b` | 7.1 GB | 12 GB | ● | | | | | ●● | Apache-2.0 | 128K context |\n| `gemma4-12b` | 7.6 GB | 12 GB | ● | | | ●● | ● | ●● | Gemma | Encoder-free any-to-any (T/I/A/V) |\n| `qwen-coder-14b` | 9.0 GB | 16 GB | ● | ●● | | | | | Apache-2.0 | Dense code+agent |\n| `qwen3-14b` | 9.0 GB | 16 GB | ● | | | | | | Apache-2.0 | More capable Qwen3 chat |\n| `phi-4-14b` | 9.1 GB | 12 GB | ● | | ●● | | | | MIT | Strong reasoning per byte |\n| **Mid — 24-32 GB box** | | | | | | | | | | |\n| `gpt-oss-20b` ⭐ | 14 GB | 16 GB | ● | ● | ●● | | | ● | Apache-2.0 | OpenAI open-weight, adjustable thinking |\n| `qwen3.6-27b` ⭐ | 17 GB | 24 GB | ●● | ●● | ● | | | ●● | Apache-2.0 | 77 % SWE-bench, top consumer pick |\n| `gemma4-26b` | 18 GB | 24 GB | ● | | | ● | | ●● | Gemma | MoE 4B-active, multimodal |\n| `qwen3-30b` | 19 GB | 24 GB | ●● | | | | | ●● | Apache-2.0 | MoE 3B-active, very fast |\n| `qwen3-coder-30b` | 19 GB | 24 GB | ● | ●● | | | | ●● | Apache-2.0 | MoE 3.3B-active coder |\n| `qwen-coder-32b` | 20 GB | 32 GB | ● | ●● | | | | | Apache-2.0 | Dense, older but proven |\n| **Power user — single 80 GB GPU / sharded** | | | | | | | | | | |\n| `llama-3.3-70b-sharded` | 43 GB | 48 GB | ●● | | | | | ●● | Llama 3.3 | Needs ≥2 nodes |\n| `gpt-oss-120b` | 65 GB | 80 GB | ●● | ● | ●●● | | | ● | Apache-2.0 | ≈ o4-mini reasoning, single H100 |\n| `llama-4-scout` | 67 GB | 80 GB | ●● | | | ●● | | ●●● | Llama 4 | 10M context, multimodal |\n| **Frontier — multi-machine sharded** | | | | | | | | | | |\n| `step-3.7-flash-sharded` ⭐ | 100 GB | 128 GB | ●● | ●● | ●● | ●● | | ●● | Apache-2.0 | 11B active VLM, fastest frontier MoE |\n| `deepseek-v4-flash-sharded` ⭐ | 150 GB | 160 GB | ●● | ●● | ●●● | | | ●● | MIT | 13B active = fast at frontier quality |\n| `nemotron-3-ultra-sharded` | 280 GB | 320 GB | ●● | ●● | ●●● | | | ●●● | NVIDIA Open | Hybrid Mamba-MoE, 1M ctx, MMLU 89.1 |\n| `glm-5.1-sharded` | 400 GB | 416 GB | ●● | ●●● | ●● | | | ●● | MIT | Best agentic coder |\n| `kimi-k2.6-sharded` | 500 GB | 512 GB | ●● | ●●● | ●● | | | ●● | Mod. MIT | #1 open coding benchmarks |\n\n**Legend** — • basic / ● good / ●● strong / ●●● best-in-tier · ⭐ recommended starting point\n\n> Sizes assume Q4_K_M (Ollama's default). Sharded entries assume Q4 baseline; unsloth's 2-bit dynamic GGUFs cut them roughly in half.\n\n**Quick rules of thumb:**\n- **I just want to try Flock** → `llama-3.2-3b` (2 GB, runs anywhere).\n- **Best on-device edge MoE** → `lfm2.5-8b-a1b` — only 1 B active, MATH500 88.8, MLX-ready.\n- **My laptop coding agent** → `mellum2-12b` (Apache-2.0, MoE, 2.5 B active) on 12 GB or `qwen3-coder-30b` on 24 GB.\n- **Multimodal on a laptop (text/image/audio/video)** → `gemma4-12b` — encoder-free unified architecture.\n- **Best consumer general model** → `qwen3.6-27b` (24 GB) or `gpt-oss-20b` (16 GB).\n- **Reasoning-heavy work** → `gpt-oss-20b` (small) or `gpt-oss-120b` (big).\n- **My team has 4 Mac Studios** → install the same model (e.g. `qwen3.6-27b`) on each — Flock load-balances automatically. No sharding needed for throughput.\n- **A model bigger than any one machine** → sharded tier. Start with `step-3.7-flash-sharded` (Apache-2.0) or `deepseek-v4-flash-sharded` (MIT).\n\nFor the full per-model walkthrough (the 9 original entries below have detailed install + client snippets), keep scrolling. For the 13 newer entries, run `flock model info <id>` — same fields as the table above, plus engine compatibility.\n\n---\n\n## How to choose\n\n```\n ┌──────────────────────────────────┐\n │ How much RAM does your box │\n │ have AFTER OS + Chrome? │\n └──────────────┬───────────────────┘\n │\n ┌──────────────────────┼──────────────────────┐\n │ │ │\n <6 GB 6-20 GB 20-48 GB >48 GB\n │ │ │ │\n ▼ ▼ ▼ ▼\n llama-3.2-1b What's the workload? What's the workload? Same as 20-48,\n (smoke test) │ │ or shard a 70B:\n │ │ llama-3.3-70b\n ┌──────────────┴──────────────┐ │ -sharded\n │ │ │ │\n coding chat/agent reasoning │\n │ │ │ │\n ▼ ▼ ▼ ▼\n qwen-coder-7b qwen3-8b deepseek-r1-8b qwen-coder-32b\n (8 GB) (12 GB) (12 GB) (32 GB) — laptop max\n qwen-coder-14b\n (16 GB)\n qwen3-14b\n (16 GB)\n```\n\n**Quick rules of thumb:**\n- If unsure, start with `qwen-coder-14b` — best general-purpose coder/agent model that fits a 16 GB machine.\n- For Claude-Code-style workflows specifically, prefer Qwen Coder over Qwen3 chat — it's tuned for tool use.\n- Don't pick a 70B model on a 24 GB machine. Even if it loads, every token will swap to disk.\n\n---\n\n## Table of contents\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "8758177c2e31983c940669742e0109bcaeee02a5": {"t": 1785770964.8762283, "n": 24048, "h": "aa711da5e9168e0f", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 10 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## 🎯 Picker table — what to install\n\nScan the column that matches your hardware, then pick by use case. ⭐ = recommended starting point in each row.\n\n| Model ID | Size | Min RAM | Chat | Code | Reasoning | Vision | Long ctx | License | Notes |\n| ------------------------------ | ------ | ------- | :--: | :--: | :-------: | :----: | :------: | ----------- | -------------------------------------- |\n| **Edge — laptop / Raspberry-Pi-ish** | | | | | | | | | |\n| `llama-3.2-1b` | 1.3 GB | 2 GB | • | | | | | Llama 3.2 | Smoke test only |\n| `llama-3.2-3b` ⭐ | 2.0 GB | 4 GB | ● | | | | | Llama 3.2 | Edge default |\n| **Small — 8-16 GB box** | | | | | | | | | |\n| `qwen-coder-7b` | 4.7 GB | 8 GB | ● | ● | | | | Apache-2.0 | FIM-capable, older |\n| `deepseek-r1-8b` | 4.9 GB | 12 GB | ● | | ●● | | | MIT | Distilled reasoning |\n| `qwen3-8b` | 5.2 GB | 12 GB | ● | | | | | Apache-2.0 | General chat |\n| `mistral-nemo-12b` | 7.1 GB | 12 GB | ● | | | | ●● | Apache-2.0 | 128K context |\n| `gemma4-12b` | 7.6 GB | 12 GB | ● | | | | ●● | Gemma | 256K ctx, multilingual |\n| `qwen3-14b` | 9.0 GB | 16 GB | ● | | | | | Apache-2.0 | More capable Qwen3 chat |\n| `qwen-coder-14b` ⭐ | 9.0 GB | 16 GB | ● | ●● | | | | Apache-2.0 | Best 16 GB code+agent pick |\n| `phi-4-14b` | 9.1 GB | 12 GB | ● | | ●● | | | MIT | Strong reasoning per byte |\n| **Mid — 24-32 GB box** | | | | | | | | | |\n| `gpt-oss-20b` ⭐ | 14 GB | 16 GB | ● | ● | ●● | | ● | Apache-2.0 | OpenAI open-weight, adjustable thinking |\n| `qwen3.6-27b` ⭐ | 17 GB | 24 GB | ●● | ●● | ● | | ●● | Apache-2.0 | 77 % SWE-bench, top consumer pick |\n| `gemma4-26b` | 18 GB | 24 GB | ● | | | ● | ●● | Gemma | MoE 4B-active, multimodal |\n| `qwen3-30b` | 19 GB | 24 GB | ●● | | | | ●● | Apache-2.0 | MoE 3B-active, very fast |\n| `qwen3-coder-30b` | 19 GB | 24 GB | ● | ●● | | | ●● | Apache-2.0 | MoE 3.3B-active coder |\n| `qwen-coder-32b` | 20 GB | 32 GB | ● | ●● | | | | Apache-2.0 | Dense, older but proven |\n| **Power user — single 80 GB GPU / sharded** | | | | | | | | | |\n| `llama-3.3-70b-sharded` | 43 GB | 48 GB | ●● | | | | ●● | Llama 3.3 | Needs ≥2 nodes |\n| `gpt-oss-120b` | 65 GB | 80 GB | ●● | ● | ●●● | | ● | Apache-2.0 | ≈ o4-mini reasoning, single H100 |\n| `llama-4-scout` | 67 GB | 80 GB | ●● | | | ●● | ●●● | Llama 4 | 10M context, multimodal |\n| **Frontier — multi-machine sharded** | | | | | | | | | |\n| `deepseek-v4-flash-sharded` ⭐ | 150 GB | 160 GB | ●● | ●● | ●●● | | ●● | MIT | 13B active = fast at frontier quality |\n| `glm-5.1-sharded` | 400 GB | 416 GB | ●● | ●●● | ●● | | ●● | MIT | Best agentic coder |\n| `kimi-k2.6-sharded` | 500 GB | 512 GB | ●● | ●●● | ●● | | ●● | Mod. MIT | #1 open coding benchmarks |\n\n**Legend** — • basic / ● good / ●● strong / ●●● best-in-tier · ⭐ recommended starting point\n\n> Sizes assume Q4_K_M (Ollama's default). Sharded entries assume Q4 baseline; unsloth's 2-bit dynamic GGUFs cut them roughly in half.\n\n**Quick rules of thumb:**\n- **I just want to try Flock** → `llama-3.2-3b` (2 GB, runs anywhere).\n- **My laptop coding agent** → `qwen-coder-14b` if 16 GB box; `qwen3-coder-30b` if 24 GB.\n- **Best consumer general model** → `qwen3.6-27b` (24 GB) or `gpt-oss-20b` (16 GB).\n- **Reasoning-heavy work** → `gpt-oss-20b` (small) or `gpt-oss-120b` (big).\n- **My team has 4 Mac Studios** → install the same model (e.g. `qwen3.6-27b`) on each — Flock load-balances automatically. No sharding needed for throughput.\n- **A model bigger than any one machine** → sharded tier. Start with `deepseek-v4-flash-sharded`.\n\nFor the full per-model walkthrough (the 9 original entries below have detailed install + client snippets), keep scrolling. For the 13 newer entries, run `flock model info <id>` — same fields as the table above, plus engine compatibility.\n\n---\n\n## How to choose\n\n```\n ┌──────────────────────────────────┐\n │ How much RAM does your box │\n │ have AFTER OS + Chrome? │\n └──────────────┬───────────────────┘\n │\n ┌──────────────────────┼──────────────────────┐\n │ │ │\n <6 GB 6-20 GB 20-48 GB >48 GB\n │ │ │ │\n ▼ ▼ ▼ ▼\n llama-3.2-1b What's the workload? What's the workload? Same as 20-48,\n (smoke test) │ │ or shard a 70B:\n │ │ llama-3.3-70b\n ┌──────────────┴──────────────┐ │ -sharded\n │ │ │ │\n coding chat/agent reasoning │\n │ │ │ │\n ▼ ▼ ▼ ▼\n qwen-coder-7b qwen3-8b deepseek-r1-8b qwen-coder-32b\n (8 GB) (12 GB) (12 GB) (32 GB) — laptop max\n qwen-coder-14b\n (16 GB)\n qwen3-14b\n (16 GB)\n```\n\n**Quick rules of thumb:**\n- If unsure, start with `qwen-coder-14b` — best general-purpose coder/agent model that fits a 16 GB machine.\n- For Claude-Code-style workflows specifically, prefer Qwen Coder over Qwen3 chat — it's tuned for tool use.\n- Don't pick a 70B model on a 24 GB machine. Even if it loads, every token will swap to disk.\n\n---\n\n## Table of contents\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "45bff7d270fc7a46c23567694e596155e8530c1e": {"t": 1785770966.5360694, "n": 18487, "h": "5e1af85c18435aea", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> **Tip:** the snippets below are written by hand for reference, but Flock can also generate them automatically with your URL + token already filled in:\n>\n> ```bash\n> flock connect claude-code # or: cursor, aider, continue, zed, cline, qwen-code, openai-sdk, anthropic-sdk, curl\n> flock connect --list # all 10 supported tools\n> ```\n>\n> Or click **Connect** in the dashboard (`http://localhost:8080`).\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## How to choose\n\n```\n ┌──────────────────────────────────┐\n │ How much RAM does your box │\n │ have AFTER OS + Chrome? │\n └──────────────┬───────────────────┘\n │\n ┌──────────────────────┼──────────────────────┐\n │ │ │\n <6 GB 6-20 GB 20-48 GB >48 GB\n │ │ │ │\n ▼ ▼ ▼ ▼\n llama-3.2-1b What's the workload? What's the workload? Same as 20-48,\n (smoke test) │ │ or shard a 70B:\n │ │ llama-3.3-70b\n ┌──────────────┴──────────────┐ │ -sharded\n │ │ │ │\n coding chat/agent reasoning │\n │ │ │ │\n ▼ ▼ ▼ ▼\n qwen-coder-7b qwen3-8b deepseek-r1-8b qwen-coder-32b\n (8 GB) (12 GB) (12 GB) (32 GB) — laptop max\n qwen-coder-14b\n (16 GB)\n qwen3-14b\n (16 GB)\n```\n\n**Quick rules of thumb:**\n- If unsure, start with `qwen-coder-14b` — best general-purpose coder/agent model that fits a 16 GB machine.\n- For Claude-Code-style workflows specifically, prefer Qwen Coder over Qwen3 chat — it's tuned for tool use.\n- Don't pick a 70B model on a 24 GB machine. Even if it loads, every token will swap to disk.\n\n---\n\n## Table of contents\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}, "2ce8a4bcf5a3f730a5cfd985c07c0d2263664fdd": {"t": 1785770968.908246, "n": 18065, "h": "64d9573a57c73e5b", "c": "# Models — the complete walkthrough\n\nEvery model in Flock's catalog, with step-by-step install + use instructions for every common client (curl, Cursor / VS Code, Claude Code, OpenAI SDK, Anthropic SDK). Pick the row that matches your hardware, run the install, copy the snippet for your tool.\n\n> For the catalog summary table see [QUICKSTART](QUICKSTART.md#-use-a-different-model-qwen-llama-deepseek). For \"why those specific models?\" see [README → Supported models](README.md#supported-models).\n\n---\n\n## How to choose\n\n```\n ┌──────────────────────────────────┐\n │ How much RAM does your box │\n │ have AFTER OS + Chrome? │\n └──────────────┬───────────────────┘\n │\n ┌──────────────────────┼──────────────────────┐\n │ │ │\n <6 GB 6-20 GB 20-48 GB >48 GB\n │ │ │ │\n ▼ ▼ ▼ ▼\n llama-3.2-1b What's the workload? What's the workload? Same as 20-48,\n (smoke test) │ │ or shard a 70B:\n │ │ llama-3.3-70b\n ┌──────────────┴──────────────┐ │ -sharded\n │ │ │ │\n coding chat/agent reasoning │\n │ │ │ │\n ▼ ▼ ▼ ▼\n qwen-coder-7b qwen3-8b deepseek-r1-8b qwen-coder-32b\n (8 GB) (12 GB) (12 GB) (32 GB) — laptop max\n qwen-coder-14b\n (16 GB)\n qwen3-14b\n (16 GB)\n```\n\n**Quick rules of thumb:**\n- If unsure, start with `qwen-coder-14b` — best general-purpose coder/agent model that fits a 16 GB machine.\n- For Claude-Code-style workflows specifically, prefer Qwen Coder over Qwen3 chat — it's tuned for tool use.\n- Don't pick a 70B model on a 24 GB machine. Even if it loads, every token will swap to disk.\n\n---\n\n## Table of contents\n\n- [`llama-3.2-1b`](#llama-3-2-1b--smallest-smoke-test) — smallest, smoke test\n- [`llama-3.2-3b`](#llama-3-2-3b--small-fast-chat) — small fast chat\n- [`qwen-coder-7b`](#qwen-coder-7b--coding-baseline) — coding baseline\n- [`qwen-coder-14b`](#qwen-coder-14b--general-purpose-coder) — general-purpose coder ⭐\n- [`qwen-coder-32b`](#qwen-coder-32b--laptop-frontier-coder) — laptop frontier coder\n- [`qwen3-8b`](#qwen3-8b--general-chat-balanced) — general chat, balanced\n- [`qwen3-14b`](#qwen3-14b--general-chat-capable) — general chat, more capable\n- [`deepseek-r1-8b`](#deepseek-r1-8b--reasoning-thinking) — reasoning\n- [`llama-3.3-70b-sharded`](#llama-3-3-70b-sharded--frontier-multi-machine) — frontier, multi-machine\n\n---\n\n## `llama-3.2-1b` — smallest, smoke test\n\n**What it is:** Meta's Llama 3.2 1B Instruct. The tiniest model that's still coherent for short prompts.\n\n**System requirements:** 2 GB RAM. Any laptop made in the last 5 years.\n\n**Best for:** First-run smoke test. \"Does Flock work end-to-end?\" Verifying the CLI / UI / Claude Code wiring. Quick latency benchmarks.\n\n**Not for:** Real coding work. Multi-turn agents. Anything where output quality matters.\n\n**Performance:**\n- Mac M3 (24 GB): ~80–120 tok/s on Ollama, sub-second TTFT\n- Linux + RTX 4090: ~150+ tok/s\n- CPU-only: still usable, ~10–20 tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-1b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-1b\",\"messages\":[{\"role\":\"user\",\"content\":\"say hi\"}]}'\n```\n\n```bash\n# Cursor / VS Code Continue.dev / Aider — OpenAI-compatible base URL\n# Base URL: http://localhost:8080/v1\n# API key: sk-orc-...\n# Model: llama-3.2-1b\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-1b\nclaude\n```\n\n```python\n# Python OpenAI SDK\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8080/v1\", api_key=\"sk-orc-...\")\nresp = client.chat.completions.create(\n model=\"llama-3.2-1b\",\n messages=[{\"role\": \"user\", \"content\": \"say hi\"}],\n)\nprint(resp.choices[0].message.content)\n```\n\n**Switch up when:** you start typing real prompts. Pick `llama-3.2-3b` or `qwen3-8b` next.\n\n---\n\n## `llama-3.2-3b` — small fast chat\n\n**What it is:** Meta's Llama 3.2 3B Instruct. Big enough to hold context, small enough to be snappy on any laptop.\n\n**System requirements:** 4 GB RAM. Comfortable on a M2 MacBook Air.\n\n**Best for:** Personal chat, simple summarization, short-prompt agents, classification. Quick autocomplete-style use cases.\n\n**Not for:** Heavy coding. Long-running agents. Complex multi-tool reasoning.\n\n**Performance:**\n- Mac M3 (24 GB): ~50–80 tok/s\n- Mac M4 Pro: ~80–120 tok/s\n- Linux + RTX 5090: ~200+ tok/s\n\n**Install:**\n```bash\nflock model add llama-3.2-3b\n```\n\n**Use it (pick one):**\n\n```bash\n# curl\ncurl http://localhost:8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.2-3b\",\"messages\":[{\"role\":\"user\",\"content\":\"summarize: …\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=llama-3.2-3b\nclaude\n```\n\n**Switch up when:** you do real coding work — pick a `qwen-coder-*`. For better chat, jump to `qwen3-8b`.\n\n---\n\n## `qwen-coder-7b` — coding baseline\n\n**What it is:** Alibaba's Qwen 2.5 Coder 7B Instruct. Code-tuned, supports fill-in-the-middle (FIM), good tool use.\n\n**System requirements:** 8 GB RAM. Mac M1/M2/M3 with 16 GB sails. M2 Air 8 GB is borderline (close other apps).\n\n**Best for:** Single-line code completion (Continue / Cursor autocomplete). Short coding chats. Quick refactor suggestions.\n\n**Not for:** Multi-turn agentic loops with lots of tool calls — the 14B is meaningfully smarter for that.\n\n**Performance:**\n- Mac M3 24 GB: ~30–50 tok/s\n- Mac M4 Pro 64 GB: ~60–80 tok/s\n- Linux + RTX 5090: ~150+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-7b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-7b\",\"messages\":[{\"role\":\"user\",\"content\":\"write fizzbuzz in rust\"}]}'\n```\n\n```bash\n# Cursor\n# Settings → Models → Override OpenAI Base URL: http://localhost:8080/v1\n# Model id: qwen-coder-7b\n\n# Continue.dev (~/.continue/config.json)\n{\n \"models\": [{\n \"title\": \"Flock — Qwen Coder 7B\",\n \"provider\": \"openai\",\n \"model\": \"qwen-coder-7b\",\n \"apiBase\": \"http://localhost:8080/v1\",\n \"apiKey\": \"sk-orc-...\"\n }]\n}\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-7b\nclaude\n```\n\n**Switch up when:** you start hitting \"this completion is wrong\" too often → try `qwen-coder-14b`.\n\n---\n\n## `qwen-coder-14b` — general-purpose coder ⭐\n\n**What it is:** Alibaba's Qwen 2.5 Coder 14B Instruct. The sweet spot for most laptops — strong code + agent capability while still fitting in 16 GB.\n\n**System requirements:** 16 GB RAM minimum, 24 GB recommended (leaves room for Chrome / your editor / Slack).\n\n**Best for:** Real coding work via Cursor / Claude Code / Aider. Multi-turn refactors. Agentic loops with tool use. Code review. Test generation.\n\n**Not for:** Frontier reasoning tasks where you'd reach for o1 / R1 — use `deepseek-r1-8b` for those. Models bigger than 16 GB if you're tight on RAM.\n\n**Performance:**\n- Mac M3 24 GB: ~20–35 tok/s\n- Mac M4 Pro 64 GB: ~40–60 tok/s\n- Linux + RTX 5090: ~100+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-14b\n```\n\n**Use it (this is the model most people should default to):**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen-coder-14b\",\"messages\":[{\"role\":\"user\",\"content\":\"refactor this …\"}]}'\n```\n\n```bash\n# Claude Code — this is where it shines\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-14b\nclaude\n# Now claude can edit files, run bash, etc. — using your local model\n```\n\n```python\n# Anthropic SDK (works because Flock speaks Anthropic format too)\nfrom anthropic import Anthropic\nclient = Anthropic(base_url=\"http://localhost:8080\", api_key=\"sk-orc-...\")\nresp = client.messages.create(\n model=\"qwen-coder-14b\",\n max_tokens=2048,\n messages=[{\"role\": \"user\", \"content\": \"explain CRDTs in 100 words\"}],\n)\nprint(resp.content[0].text)\n```\n\n**Switch up when:** You have a 64 GB+ machine and want the strongest single-box coder → `qwen-coder-32b`. For frontier-tier you need multi-machine sharding.\n\n---\n\n## `qwen-coder-32b` — laptop frontier coder\n\n**What it is:** Alibaba's Qwen 2.5 Coder 32B Instruct. Approaches GPT-4 / Sonnet quality on coding tasks. The biggest coder that fits a 64 GB Mac without swapping.\n\n**System requirements:** 32 GB RAM minimum (will swap on smaller). 64 GB Mac Mini / MacBook Pro recommended.\n\n**Best for:** Heavy agentic coding (multi-hour Claude Code sessions). Complex refactors. Pull-request review. When the 14B isn't smart enough.\n\n**Not for:** Tight-RAM machines. Latency-sensitive autocomplete (TTFT is noticeably higher).\n\n**Performance:**\n- Mac M3 24 GB: don't (will swap)\n- Mac M4 Pro 64 GB: ~12–20 tok/s\n- Mac Studio M3 Ultra 96 GB: ~25–40 tok/s\n- Linux + 2× RTX 5090: ~80+ tok/s\n\n**Install:**\n```bash\nflock model add qwen-coder-32b\n```\n\n**Use it:**\n\n```bash\n# Claude Code — best target for this size model\nexport ANTHROPIC_BASE_URL=http://localhost:8080\nexport ANTHROPIC_AUTH_TOKEN=sk-orc-...\nexport ANTHROPIC_MODEL=qwen-coder-32b\nclaude\n```\n\n**Switch up when:** You need bigger than 32 GB of model. That means sharding — see `llama-3.3-70b-sharded`.\n\n---\n\n## `qwen3-8b` — general chat, balanced\n\n**What it is:** Alibaba's Qwen3 8B Instruct (general-purpose, not code-tuned).\n\n**System requirements:** 12 GB RAM. Mid-tier laptop comfort.\n\n**Best for:** Mixed workloads where you need general chat + occasional code (Qwen3 isn't as strong on code as Qwen Coder, but it's a better generalist).\n\n**Not for:** Pure code work — pick `qwen-coder-7b` instead at this RAM tier.\n\n**Install:**\n```bash\nflock model add qwen3-8b\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"qwen3-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=qwen3-8b\n```\n\n---\n\n## `qwen3-14b` — general chat, capable\n\n**What it is:** Alibaba's Qwen3 14B Instruct. Strong general-purpose chat — close to GPT-4-class on many benchmarks.\n\n**System requirements:** 16 GB RAM.\n\n**Best for:** Customer-facing chatbots, document Q&A, agent loops where the agent needs broad world knowledge (not just code).\n\n**Not for:** Pure-code workflows (pick `qwen-coder-14b` for those — Qwen Coder is specifically tool-use-tuned).\n\n**Install:**\n```bash\nflock model add qwen3-14b\n```\n\n**Use it:** same patterns as above with `model: qwen3-14b`.\n\n---\n\n## `deepseek-r1-8b` — reasoning (\"thinking\")\n\n**What it is:** DeepSeek-R1-Distill-Qwen-8B. A reasoning model — produces a hidden `<thinking>` block before its answer (where it works through the problem step by step).\n\n**System requirements:** 12 GB RAM.\n\n**Best for:** Math problems, multi-step reasoning, careful code review where correctness matters more than speed, debugging complex logic.\n\n**Not for:** Latency-sensitive autocomplete (every response includes a long internal monologue). Pure chat.\n\n**Performance note:** R1 models are slower per-token because they \"think\" before answering — total time-to-final-answer is what matters, not raw tok/s.\n\n**Install:**\n```bash\nflock model add deepseek-r1-8b\n```\n\n**Use it:**\n\n```bash\n# curl — note that the response will include `<think>...</think>` blocks\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"deepseek-r1-8b\",\"messages\":[{\"role\":\"user\",\"content\":\"if a train leaves Chicago at 3pm…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=deepseek-r1-8b\n```\n\n**Tip:** When a model emits `<think>` tags, most UIs (Claude Code, Cursor) show them as part of the response. If that's distracting, strip them client-side or use a non-reasoning model for that task.\n\n---\n\n## `llama-3.3-70b-sharded` — frontier, multi-machine\n\n**What it is:** Meta Llama 3.3 70B Instruct, split across multiple machines via llama.cpp RPC. The frontier of what's runnable on consumer-grade hardware.\n\n**System requirements:** ~48+ GB of *combined* memory across at least 2 machines. E.g. 2× Mac Mini 32 GB, or 4× Mac Mini 16 GB.\n\n**Best for:** When you need GPT-4 / Sonnet-tier quality and don't want to pay per token. Frontier coding agents. Complex reasoning chains. Long-context document analysis.\n\n**Not for:** Single-machine setups. Latency-sensitive use cases (network hops between shards add overhead). Use Claude Sonnet via fallback if you want this quality without the hardware.\n\n**Prereqs (one-time):**\n- `brew install llama.cpp` on the leader (provides `llama-server`)\n- `rpc-server` on PATH on every worker — currently needs a source build of llama.cpp (the Homebrew bottle doesn't include it)\n- Catalog entry with `sharding.required: true` and a local GGUF path (`catalog/llama-3.3-70b-sharded.yaml` already configured)\n- Place the model GGUF at `/var/lib/flock/models/llama-3.3-70b-q4_k_m.gguf` (or update the YAML's `source.path`)\n- At least 2 workers joined and `ready` (`flock node ls`)\n\n**Install (orchestrates the whole thing):**\n\n```bash\n# splits across 2 workers automatically\nflock shard create llama-3.3-70b-sharded 2\n\n# what Flock does:\n# 1. picks the 2 workers with the most free RAM\n# 2. POSTs /v1/process/start to each → launches `rpc-server -p 50052`\n# 3. waits for both rpc-servers to be TCP-reachable\n# 4. on the leader, launches `llama-server -m <gguf> --rpc w1:50052,w2:50052 --port 9001`\n# 5. persists shard rows + a placements row pointing at the local coordinator\n# 6. Router routes any request for `llama-3.3-70b-sharded` to the coordinator\n```\n\n**Use it:**\n\n```bash\n# curl\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"llama-3.3-70b-sharded\",\"messages\":[{\"role\":\"user\",\"content\":\"…\"}]}'\n```\n\n```bash\n# Claude Code\nexport ANTHROPIC_MODEL=llama-3.3-70b-sharded\nclaude\n```\n\n**Tear down (cleanly):**\n\n```bash\nflock shard remove llama-3.3-70b-sharded\n# stops the coordinator on the leader + every rpc-server on every worker\n# removes shard rows + placements\n```\n\n**Switch up when:** You need bigger than 70B (DeepSeek-V3 671B MoE, Qwen3-Coder-480B) — those need more nodes and more RAM. v0.5 will support auto-scaling shard counts.\n\n---\n\n## Use any other Ollama model (catalog pass-through)\n\nFlock's catalog is curated for UX, but the router will pass through any model name the engine doesn't recognize. Steps:\n\n```bash\n# 1. pull via Ollama directly (Flock's catalog is bypassed)\nollama pull mistral-nemo:12b # or any model from https://ollama.com/library\n\n# 2. use the engine-native name in your API request\ncurl :8080/v1/chat/completions \\\n -H \"Authorization: Bearer sk-orc-...\" \\\n -d '{\"model\":\"mistral-nemo:12b\",\"messages\":[…]}'\n```\n\nOr add a custom catalog entry (so it shows up in `flock model search` + UI): drop a YAML file in `catalog/` matching the schema of the existing entries:\n\n```yaml\n# catalog/my-model.yaml\nid: my-model\ndisplay_name: My Custom Model\nsource:\n type: ollama\n ollama_name: my-model:7b\nsize_bytes: 4000000000\nquant: q4_k_m\ncontext_window: 32768\ncapabilities: [chat]\nrecommended_engines: [ollama]\nhardware:\n min_ram_gb: 8\ntags: [chat]\n```\n\nRestart `flock up` — your model now appears in `flock model search` and the web UI's catalog picker.\n\n---\n\n## Switching models on the fly\n\nFor Claude Code:\n\n```bash\n# different sessions can use different models:\nANTHROPIC_MODEL=llama-3.2-1b claude # smoke test session\nANTHROPIC_MODEL=qwen-coder-14b claude # real work session\nANTHROPIC_MODEL=claude-opus-4-7 claude # falls back to real Anthropic (if fallback configured)\n```\n\nFor curl / SDKs: change the `model` field in each request body. The same Flock instance serves all of them.\n\nFor Cursor / Continue: change the model id in the UI / config file and reload.\n\n---\n\n## Troubleshooting per-model\n\n| Symptom | Likely model | Fix |\n|---|---|---|\n| Slow first response | Any (cold model load) | First chat after `flock up` loads the model into RAM — subsequent ones reuse it |\n| `out of memory` mid-stream | Too big for RAM | Pick a smaller model OR close other apps OR add a worker and offload |\n| Constant `<think>` tags in output | `deepseek-r1-*` | That's normal — it's a reasoning model. Use a non-reasoning model if you don't want them |\n| `model not found` | Typo in catalog id | `flock model ls` to see what's actually installed; `flock model search` to find the right id |\n| Sharded model 404 | Coordinator failed | `flock shard ls` to check status; `flock shard remove …` then `flock shard create …` to redo |\n\n---\n\n## Roadmap (planned models in upcoming versions)\n\nThese have docs and references in the repo but no catalog YAMLs yet. Coming with v0.5+:\n\n- **Llama 3.3 70B** (non-sharded, for boxes with 48+ GB single-machine)\n- **DeepSeek V3 / R1 671B MoE** (frontier, sharded across many nodes)\n- **Qwen3-72B Instruct** + **Qwen3-Coder-30B-A3B**\n- **GLM-4.6**, **Kimi K2**\n- **Vision models** (Qwen2.5-VL, Llama 3.2 Vision)\n- **Embeddings** (BGE-M3) + **Whisper** (transcription)\n\nIf you have a specific model you want in the curated catalog: open an issue at https://github.com/hadihonarvar/flock/issues with the Ollama tag / HF repo + your hardware target, and we'll add a YAML.\n"}}}} |