1 line
19 KiB
JSON
1 line
19 KiB
JSON
{"repo": "ajwcontreras/qdrant-mcp-server", "files": {"SETUP.md": {"ac1f07c249e9b7d005365e72fad56d9f41becd95": {"t": 1785747201.5507863, "n": 9239, "h": "4aad84fe4d9e4894", "c": "# cfcode Setup Guide\n\nComplete setup from zero to indexed codebase. No ambiguity. Follow in order.\n\n## Prerequisites\n\n```bash\n# Node.js 20+\nnode --version # must be >= 20\n\n# npm\nnpm --version\n\n# Wrangler CLI (Cloudflare Workers)\nnpm install -g wrangler\nwrangler --version # should be >= 4.0\n\n# git\ngit --version\n\n# jq (for JSON parsing in scripts, optional)\nbrew install jq # macOS\n```\n\n## 1. Cloudflare Account\n\nSign up at https://dash.cloudflare.com/sign-up. You need:\n\n- **Workers Paid plan** ($5/month) — required for D1, Durable Objects, and 10K subrequests\n- **Workers for Platforms** ($25/month) — required for the dispatch namespace that holds per-codebase workers\n\nOnce created, note your:\n- Account ID: found at `https://dash.cloudflare.com` → right sidebar\n- Email: your Cloudflare login email\n\nCreate an API token at https://dash.cloudflare.com/profile/api-tokens:\n- Template: \"Edit Cloudflare Workers\"\n- Account Resources: your account\n- Zone Resources: All zones\n\nSave the token as `CLOUDFLARE_API_TOKEN`. Alternatively, get your Global API Key from https://dash.cloudflare.com/profile/api-tokens and use `CLOUDFLARE_API_KEY` + `CLOUDFLARE_EMAIL`.\n\n## 2. Vertex AI Service Accounts (Google Cloud)\n\nThe pipeline uses Vertex AI `gemini-embedding-001` for embedding code chunks. You need at least one Google Cloud service account with Vertex AI access.\n\n### Single SA setup (minimum)\n\n1. Go to https://console.cloud.google.com\n2. Create a project (or use existing)\n3. Enable Vertex AI API:\n ```bash\n gcloud services enable aiplatform.googleapis.com --project=YOUR_PROJECT_ID\n ```\n4. Create a service account:\n ```bash\n gcloud iam service-accounts create cfcode-vertex \\\n --project=YOUR_PROJECT_ID \\\n --display-name=\"cfcode Vertex AI\"\n ```\n5. Grant Vertex AI user role:\n ```bash\n gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \\\n --member=\"serviceAccount:cfcode-vertex@YOUR_PROJECT_ID.iam.gserviceaccount.com\" \\\n --role=\"roles/aiplatform.user\"\n ```\n6. Create and download a JSON key:\n ```bash\n gcloud iam service-accounts keys create ~/cfcode-sa.json \\\n --iam-account=cfcode-vertex@YOUR_PROJECT_ID.iam.gserviceaccount.com\n ```\n7. **Store the JSON securely.** The key file is your credential. Mode 0600 recommended:\n ```bash\n chmod 600 ~/cfcode-sa.json\n ```\n\n### Multiple SAs (for higher throughput)\n\nCreate 2-4 SA keys following the same pattern. Each SA gets its own GCP project for independent Vertex quotas. (SA3 and SA4 can share a billing account — quotas are per-project.)\n\nStore all SA JSONs in a dedicated directory:\n```bash\nmkdir -p ~/.config/cfcode/sas\nchmod 700 ~/.config/cfcode/sas\ncp ~/cfcode-sa.json ~/.config/cfcode/sas/\n# repeat for additional SAs\n```\n\n## 3. DeepSeek API Key (for HyDE question generation)\n\n1. Sign up at https://platform.deepseek.com\n2. Create an API key at https://platform.deepseek.com/api_keys\n3. Save the key (starts with `sk-`)\n\nNo subscription needed — pay-per-token. With prompt caching (`deepseek-v4-flash`), one full codebase HyDE pass costs ~$0.50-1.\n\n## 4. Local Configuration\n\n### `.cfapikeys` file\n\nCreate at the repo root (gitignored):\n\n```\nCF_GLOBAL_API_KEY=cfk_your_global_api_key_here\nCF_EMAIL=you@example.com\nCF_ACCOUNT_ID=your_32_char_account_id\nDEEPSEEK_API_KEY=sk-your_deepseek_key_here\n```\n\n**Alternative:** Use environment variables instead:\n```bash\nexport CLOUDFLARE_API_KEY=cfk_...\nexport CLOUDFLARE_EMAIL=you@example.com\nexport CLOUDFLARE_ACCOUNT_ID=...\nexport DEEPSEEK_API_KEY=sk-...\n```\n\n### SA key paths\n\nThe CLI and bench scripts reference SA keys by path. Update these paths in the scripts or use the standard locations:\n- SA1: `~/.config/cfcode/sas/team (1).json`\n- SA2: `~/.config/cfcode/sas/underwriter-agent-479920-af2b45745dac.json`\n\nFor your own SAs, edit the references in `cloudflare-mcp/cli/cfcode.mjs` and the bench scripts.\n\n## 5. Clone and Install\n\n```bash\ngit clone https://github.com/ajwcontreras/qdrant-mcp-server.git\ncd qdrant-mcp-server\n\nnpm install\n\n# Symlink the CLI\nmkdir -p ~/bin\nln -sf \"$PWD/cloudflare-mcp/cli/cfcode.mjs\" ~/bin/cfcode\nchmod +x cloudflare-mcp/cli/cfcode.mjs\n\n# Verify\ncfcode --help\n```\n\n## 6. Deploy the Gateway (one-time)\n\nThe MCP gateway is a single Cloudflare Worker that routes all MCP traffic. Deploy once, never touch again.\n\n```bash\ncd cloudflare-mcp/workers/mcp-gateway\nnpm install\nnpx wrangler deploy --config wrangler.generated.jsonc\n```\n\nNote the URL — it'll be `https://cfcode-gateway.<your-subdomain>.workers.dev/mcp`.\n\n## 7. Deploy the Dispatch Namespace\n\n```bash\n# Create the dispatch namespace (one-time)\nnpx wrangler dispatch-namespace create cfcode-codebases\n```\n\n## 8. Index a Codebase\n\n```bash\n# Full index with HyDE (the full pipeline)\ncfcode index ~/PROJECTS/myrepo\n\n# Code-only (fast, no HyDE questions)\ncfcode index ~/PROJECTS/myrepo --no-hyde\n\n# Check status\ncfcode status myrepo\n```\n\nWhat happens during `cfcode index`:\n1. CLI walks the repo, builds ~1.5K-char chunks per file, creates a JSONL artifact\n2. CLI uploads the artifact to the gateway\n3. Gateway creates CF resources (R2 bucket, D1 database, Vectorize index)\n4. Gateway deploys a per-codebase worker in the dispatch namespace\n5. Worker processes chunks: Vertex embed → Vectorize + D1 insert\n6. Worker generates HyDE questions via DeepSeek → Vertex embed → Vectorize + D1 insert\n7. Code chunks become searchable in ~10s; HyDE completes in ~30-70s\n\n## 9. Install the MCP Gateway in Your Agent\n\n### Claude Code\n\n```bash\nnode -e '\nconst fs = require(\"fs\"), p = process.env.HOME + \"/.claude.json\";\nconst c = JSON.parse(fs.readFileSync(p, \"utf8\"));\nc.mcpServers = c.mcpServers || {};\nc.mcpServers.cfcode = { type: \"http\", url: \"https://cfcode-gateway.<your-subdomain>.workers.dev/mcp\" };\nfs.writeFileSync(p, JSON.stringify(c, null, 2));\n'\n```\n\nThen **fully quit + relaunch** Claude Code. NOT `/reset` — full process restart.\n\n**Gotcha:** MCP config lives in `~/.claude.json`, NOT `~/.claude/settings.json`. Both files exist.\n\n### Cursor / Other MCP Clients\n\nAdd to your MCP configuration:\n```json\n{\n \"mcpServers\": {\n \"cfcode\": {\n \"type\": \"http\",\n \"url\": \"https://cfcode-gateway.<your-subdomain>.workers.dev/mcp\"\n }\n }\n}\n```\n\n## 10. Usage\n\nOnce the gateway is registered, you have these MCP tools:\n```\nlist_codebases\nselect_codebase({ slug: \"myrepo\" })\nsearch({ query: \"how is auth implemented\", topK: 5 })\ncurrent_codebase\n```\n\nIn Claude Code: \"list my codebases\" → \"select myrepo\" → \"search for password reset flow\"\n\n## 11. Reindexing\n\n```bash\n# Reindex after code changes (diff-based, only processes changed files)\ncfcode reindex ~/PROJECTS/myrepo\n\n# Regenerate HyDE questions for existing chunks\n# (useful after updating the HyDE model or fixing generation bugs)\ncurl -X POST https://cfcode-gateway.<your-subdomain>.workers.dev/admin/codebases/myrepo/hyde-enrich \\\n -H \"content-type: application/json\" \\\n -d '{\"job_id\": \"rehyde-1\", \"artifact_key\": \"path/to/artifact.jsonl\"}'\n```\n\n## 12. Uninstall a Codebase\n\n```bash\ncfcode uninstall ~/PROJECTS/myrepo\n# Tears down per-codebase Worker, R2, D1, Vectorize, removes from registry\n```\n\n## 13. Troubleshooting\n\n### Worker deploy fails with D1 binding error\nThe D1 database ID in `wrangler.gen.jsonc` is stale. Delete the generated config and re-run `cfcode index`. It'll regenerate with fresh resource IDs.\n\n### Search returns no results\nVectorize is eventually consistent — new vectors take 30-60s to become queryable. Wait, then retry. Also check `cfcode status` to confirm chunks are published.\n\n### Vertex API 401 / UNAUTHENTICATED\nYour SA key is invalid or the JWT signing is broken. Verify the SA JSON has `private_key` and `client_email` fields. Check that Vertex AI API is enabled on the GCP project.\n\n### DeepSeek API 400\nThe model `deepseek-v4-flash` with `response_format: { type: \"json_object\" }` requires a valid system prompt. If you changed the prompt, verify it produces valid JSON.\n\n### `ctx.waitUntil` work doesn't complete\n`ctx.waitUntil` has a 30-second cap after the response is sent. Long-running work (>30s) must use Durable Object alarms or Cloudflare Queues, not `waitUntil`.\n\n### DO storage `put` silently fails\nDO storage has a 128KB per-key value limit. Storing large objects (like full artifact JSONL) will silently fail. Reference data by key and store config only.\n\n### Wrangler picks wrong account\nIf you have multiple Cloudflare accounts, wrangler may pick the wrong one. Check with `npx wrangler whoami`. Force the correct account with:\n```bash\nCLOUDFLARE_API_KEY=$CF_GLOBAL_API_KEY CLOUDFLARE_EMAIL=$CF_ACCOUNT_EMAIL npx wrangler deploy\n```\nOr set `account_id` in your wrangler config.\n\n## 14. Benchmarks\n\nTo reproduce the Phase 31 benchmarks against the lumae test corpus:\n\n```bash\n# Code-only indexing (no HyDE)\nnode cloudflare-mcp/scripts/poc-31f-code-vertex-bench.mjs\n\n# Full dual fan-out (code + HyDE) \nnode cloudflare-mcp/scripts/poc-31k-2pop-bench.mjs\n\n# Rate limit measurements\nnode cloudflare-mcp/scripts/poc-31i-rate-measure-smoke.mjs\n\n# E2E on a specific codebase\nnode cloudflare-mcp/scripts/poc-31k-e2e-cfpubsub.mjs\n```\n\nEach script creates throwaway resources, runs the benchmark, writes results to a bench JSON file, and cleans up. No production resources touched.\n"}, "5a8f35d45ccd022b629e8a87ca872975578e397c": {"t": 1785771345.583833, "n": 9239, "h": "4aad84fe4d9e4894", "c": "# cfcode Setup Guide\n\nComplete setup from zero to indexed codebase. No ambiguity. Follow in order.\n\n## Prerequisites\n\n```bash\n# Node.js 20+\nnode --version # must be >= 20\n\n# npm\nnpm --version\n\n# Wrangler CLI (Cloudflare Workers)\nnpm install -g wrangler\nwrangler --version # should be >= 4.0\n\n# git\ngit --version\n\n# jq (for JSON parsing in scripts, optional)\nbrew install jq # macOS\n```\n\n## 1. Cloudflare Account\n\nSign up at https://dash.cloudflare.com/sign-up. You need:\n\n- **Workers Paid plan** ($5/month) — required for D1, Durable Objects, and 10K subrequests\n- **Workers for Platforms** ($25/month) — required for the dispatch namespace that holds per-codebase workers\n\nOnce created, note your:\n- Account ID: found at `https://dash.cloudflare.com` → right sidebar\n- Email: your Cloudflare login email\n\nCreate an API token at https://dash.cloudflare.com/profile/api-tokens:\n- Template: \"Edit Cloudflare Workers\"\n- Account Resources: your account\n- Zone Resources: All zones\n\nSave the token as `CLOUDFLARE_API_TOKEN`. Alternatively, get your Global API Key from https://dash.cloudflare.com/profile/api-tokens and use `CLOUDFLARE_API_KEY` + `CLOUDFLARE_EMAIL`.\n\n## 2. Vertex AI Service Accounts (Google Cloud)\n\nThe pipeline uses Vertex AI `gemini-embedding-001` for embedding code chunks. You need at least one Google Cloud service account with Vertex AI access.\n\n### Single SA setup (minimum)\n\n1. Go to https://console.cloud.google.com\n2. Create a project (or use existing)\n3. Enable Vertex AI API:\n ```bash\n gcloud services enable aiplatform.googleapis.com --project=YOUR_PROJECT_ID\n ```\n4. Create a service account:\n ```bash\n gcloud iam service-accounts create cfcode-vertex \\\n --project=YOUR_PROJECT_ID \\\n --display-name=\"cfcode Vertex AI\"\n ```\n5. Grant Vertex AI user role:\n ```bash\n gcloud projects add-iam-policy-binding YOUR_PROJECT_ID \\\n --member=\"serviceAccount:cfcode-vertex@YOUR_PROJECT_ID.iam.gserviceaccount.com\" \\\n --role=\"roles/aiplatform.user\"\n ```\n6. Create and download a JSON key:\n ```bash\n gcloud iam service-accounts keys create ~/cfcode-sa.json \\\n --iam-account=cfcode-vertex@YOUR_PROJECT_ID.iam.gserviceaccount.com\n ```\n7. **Store the JSON securely.** The key file is your credential. Mode 0600 recommended:\n ```bash\n chmod 600 ~/cfcode-sa.json\n ```\n\n### Multiple SAs (for higher throughput)\n\nCreate 2-4 SA keys following the same pattern. Each SA gets its own GCP project for independent Vertex quotas. (SA3 and SA4 can share a billing account — quotas are per-project.)\n\nStore all SA JSONs in a dedicated directory:\n```bash\nmkdir -p ~/.config/cfcode/sas\nchmod 700 ~/.config/cfcode/sas\ncp ~/cfcode-sa.json ~/.config/cfcode/sas/\n# repeat for additional SAs\n```\n\n## 3. DeepSeek API Key (for HyDE question generation)\n\n1. Sign up at https://platform.deepseek.com\n2. Create an API key at https://platform.deepseek.com/api_keys\n3. Save the key (starts with `sk-`)\n\nNo subscription needed — pay-per-token. With prompt caching (`deepseek-v4-flash`), one full codebase HyDE pass costs ~$0.50-1.\n\n## 4. Local Configuration\n\n### `.cfapikeys` file\n\nCreate at the repo root (gitignored):\n\n```\nCF_GLOBAL_API_KEY=cfk_your_global_api_key_here\nCF_EMAIL=you@example.com\nCF_ACCOUNT_ID=your_32_char_account_id\nDEEPSEEK_API_KEY=sk-your_deepseek_key_here\n```\n\n**Alternative:** Use environment variables instead:\n```bash\nexport CLOUDFLARE_API_KEY=cfk_...\nexport CLOUDFLARE_EMAIL=you@example.com\nexport CLOUDFLARE_ACCOUNT_ID=...\nexport DEEPSEEK_API_KEY=sk-...\n```\n\n### SA key paths\n\nThe CLI and bench scripts reference SA keys by path. Update these paths in the scripts or use the standard locations:\n- SA1: `~/.config/cfcode/sas/team (1).json`\n- SA2: `~/.config/cfcode/sas/underwriter-agent-479920-af2b45745dac.json`\n\nFor your own SAs, edit the references in `cloudflare-mcp/cli/cfcode.mjs` and the bench scripts.\n\n## 5. Clone and Install\n\n```bash\ngit clone https://github.com/ajwcontreras/qdrant-mcp-server.git\ncd qdrant-mcp-server\n\nnpm install\n\n# Symlink the CLI\nmkdir -p ~/bin\nln -sf \"$PWD/cloudflare-mcp/cli/cfcode.mjs\" ~/bin/cfcode\nchmod +x cloudflare-mcp/cli/cfcode.mjs\n\n# Verify\ncfcode --help\n```\n\n## 6. Deploy the Gateway (one-time)\n\nThe MCP gateway is a single Cloudflare Worker that routes all MCP traffic. Deploy once, never touch again.\n\n```bash\ncd cloudflare-mcp/workers/mcp-gateway\nnpm install\nnpx wrangler deploy --config wrangler.generated.jsonc\n```\n\nNote the URL — it'll be `https://cfcode-gateway.<your-subdomain>.workers.dev/mcp`.\n\n## 7. Deploy the Dispatch Namespace\n\n```bash\n# Create the dispatch namespace (one-time)\nnpx wrangler dispatch-namespace create cfcode-codebases\n```\n\n## 8. Index a Codebase\n\n```bash\n# Full index with HyDE (the full pipeline)\ncfcode index ~/PROJECTS/myrepo\n\n# Code-only (fast, no HyDE questions)\ncfcode index ~/PROJECTS/myrepo --no-hyde\n\n# Check status\ncfcode status myrepo\n```\n\nWhat happens during `cfcode index`:\n1. CLI walks the repo, builds ~1.5K-char chunks per file, creates a JSONL artifact\n2. CLI uploads the artifact to the gateway\n3. Gateway creates CF resources (R2 bucket, D1 database, Vectorize index)\n4. Gateway deploys a per-codebase worker in the dispatch namespace\n5. Worker processes chunks: Vertex embed → Vectorize + D1 insert\n6. Worker generates HyDE questions via DeepSeek → Vertex embed → Vectorize + D1 insert\n7. Code chunks become searchable in ~10s; HyDE completes in ~30-70s\n\n## 9. Install the MCP Gateway in Your Agent\n\n### Claude Code\n\n```bash\nnode -e '\nconst fs = require(\"fs\"), p = process.env.HOME + \"/.claude.json\";\nconst c = JSON.parse(fs.readFileSync(p, \"utf8\"));\nc.mcpServers = c.mcpServers || {};\nc.mcpServers.cfcode = { type: \"http\", url: \"https://cfcode-gateway.<your-subdomain>.workers.dev/mcp\" };\nfs.writeFileSync(p, JSON.stringify(c, null, 2));\n'\n```\n\nThen **fully quit + relaunch** Claude Code. NOT `/reset` — full process restart.\n\n**Gotcha:** MCP config lives in `~/.claude.json`, NOT `~/.claude/settings.json`. Both files exist.\n\n### Cursor / Other MCP Clients\n\nAdd to your MCP configuration:\n```json\n{\n \"mcpServers\": {\n \"cfcode\": {\n \"type\": \"http\",\n \"url\": \"https://cfcode-gateway.<your-subdomain>.workers.dev/mcp\"\n }\n }\n}\n```\n\n## 10. Usage\n\nOnce the gateway is registered, you have these MCP tools:\n```\nlist_codebases\nselect_codebase({ slug: \"myrepo\" })\nsearch({ query: \"how is auth implemented\", topK: 5 })\ncurrent_codebase\n```\n\nIn Claude Code: \"list my codebases\" → \"select myrepo\" → \"search for password reset flow\"\n\n## 11. Reindexing\n\n```bash\n# Reindex after code changes (diff-based, only processes changed files)\ncfcode reindex ~/PROJECTS/myrepo\n\n# Regenerate HyDE questions for existing chunks\n# (useful after updating the HyDE model or fixing generation bugs)\ncurl -X POST https://cfcode-gateway.<your-subdomain>.workers.dev/admin/codebases/myrepo/hyde-enrich \\\n -H \"content-type: application/json\" \\\n -d '{\"job_id\": \"rehyde-1\", \"artifact_key\": \"path/to/artifact.jsonl\"}'\n```\n\n## 12. Uninstall a Codebase\n\n```bash\ncfcode uninstall ~/PROJECTS/myrepo\n# Tears down per-codebase Worker, R2, D1, Vectorize, removes from registry\n```\n\n## 13. Troubleshooting\n\n### Worker deploy fails with D1 binding error\nThe D1 database ID in `wrangler.gen.jsonc` is stale. Delete the generated config and re-run `cfcode index`. It'll regenerate with fresh resource IDs.\n\n### Search returns no results\nVectorize is eventually consistent — new vectors take 30-60s to become queryable. Wait, then retry. Also check `cfcode status` to confirm chunks are published.\n\n### Vertex API 401 / UNAUTHENTICATED\nYour SA key is invalid or the JWT signing is broken. Verify the SA JSON has `private_key` and `client_email` fields. Check that Vertex AI API is enabled on the GCP project.\n\n### DeepSeek API 400\nThe model `deepseek-v4-flash` with `response_format: { type: \"json_object\" }` requires a valid system prompt. If you changed the prompt, verify it produces valid JSON.\n\n### `ctx.waitUntil` work doesn't complete\n`ctx.waitUntil` has a 30-second cap after the response is sent. Long-running work (>30s) must use Durable Object alarms or Cloudflare Queues, not `waitUntil`.\n\n### DO storage `put` silently fails\nDO storage has a 128KB per-key value limit. Storing large objects (like full artifact JSONL) will silently fail. Reference data by key and store config only.\n\n### Wrangler picks wrong account\nIf you have multiple Cloudflare accounts, wrangler may pick the wrong one. Check with `npx wrangler whoami`. Force the correct account with:\n```bash\nCLOUDFLARE_API_KEY=$CF_GLOBAL_API_KEY CLOUDFLARE_EMAIL=$CF_ACCOUNT_EMAIL npx wrangler deploy\n```\nOr set `account_id` in your wrangler config.\n\n## 14. Benchmarks\n\nTo reproduce the Phase 31 benchmarks against the lumae test corpus:\n\n```bash\n# Code-only indexing (no HyDE)\nnode cloudflare-mcp/scripts/poc-31f-code-vertex-bench.mjs\n\n# Full dual fan-out (code + HyDE) \nnode cloudflare-mcp/scripts/poc-31k-2pop-bench.mjs\n\n# Rate limit measurements\nnode cloudflare-mcp/scripts/poc-31i-rate-measure-smoke.mjs\n\n# E2E on a specific codebase\nnode cloudflare-mcp/scripts/poc-31k-e2e-cfpubsub.mjs\n```\n\nEach script creates throwaway resources, runs the benchmark, writes results to a bench JSON file, and cleans up. No production resources touched.\n"}}}} |