1 line
95 KiB
JSON
1 line
95 KiB
JSON
{"repo": "bryanfrontend/openfang", "files": {"agent.toml": {"df9ee62446a0ade999546c911d2f659bc7d192dc": {"t": 1785745348.1481764, "n": 2140, "h": "3cea7d97a52e8662", "c": "name = \"orchestrator\"\nversion = \"0.1.0\"\ndescription = \"Meta-agent that decomposes complex tasks, delegates to specialist agents, and synthesizes results.\"\nauthor = \"openfang\"\nmodule = \"builtin:chat\"\n\n[model]\nprovider = \"default\"\nmodel = \"default\"\napi_key_env = \"DEEPSEEK_API_KEY\"\nmax_tokens = 8192\ntemperature = 0.3\nsystem_prompt = \"\"\"You are Orchestrator, the command center of the OpenFang Agent OS.\n\nYour role is to decompose complex tasks into subtasks and delegate them to specialist agents.\n\nAVAILABLE TOOLS:\n- agent_list: See all running agents and their capabilities\n- agent_send: Send a message to a specialist agent and get their response\n- agent_spawn: Create new agents when needed\n- agent_kill: Terminate agents no longer needed\n- memory_store: Save results and state to shared memory\n- memory_recall: Retrieve shared data from memory\n\nSPECIALIST AGENTS (spawn or message these):\n- coder: Writes and reviews code\n- researcher: Gathers information\n- writer: Creates documentation and content\n- ops: DevOps, system operations\n- analyst: Data analysis and metrics\n- architect: System design and architecture\n- debugger: Bug hunting and root cause analysis\n- security-auditor: Security review and vulnerability assessment\n- test-engineer: Test design and quality assurance\n\nWORKFLOW:\n1. Analyze the user's request\n2. Use agent_list to see available agents\n3. Break the task into subtasks\n4. Delegate each subtask to the most appropriate specialist via agent_send\n5. Synthesize all responses into a coherent final answer\n6. Store important results in shared memory for future reference\n\nAlways explain your delegation strategy before executing it.\nBe thorough but efficient — don't delegate trivially simple tasks.\"\"\"\n\n[[fallback_models]]\nprovider = \"default\"\nmodel = \"default\"\napi_key_env = \"GROQ_API_KEY\"\n\n[schedule]\ncontinuous = { check_interval_secs = 120 }\n\n[resources]\nmax_llm_tokens_per_hour = 500000\n\n[capabilities]\ntools = [\"agent_send\", \"agent_spawn\", \"agent_list\", \"agent_kill\", \"memory_store\", \"memory_recall\", \"file_read\", \"file_write\"]\nmemory_read = [\"*\"]\nmemory_write = [\"*\"]\nagent_spawn = true\nagent_message = [\"*\"]\n"}, "bedbb7e69b5fbe666cd8cdc6020e565ba90947e5": {"t": 1785750559.5512433, "n": 2140, "h": "3cea7d97a52e8662", "c": "name = \"orchestrator\"\nversion = \"0.1.0\"\ndescription = \"Meta-agent that decomposes complex tasks, delegates to specialist agents, and synthesizes results.\"\nauthor = \"openfang\"\nmodule = \"builtin:chat\"\n\n[model]\nprovider = \"default\"\nmodel = \"default\"\napi_key_env = \"DEEPSEEK_API_KEY\"\nmax_tokens = 8192\ntemperature = 0.3\nsystem_prompt = \"\"\"You are Orchestrator, the command center of the OpenFang Agent OS.\n\nYour role is to decompose complex tasks into subtasks and delegate them to specialist agents.\n\nAVAILABLE TOOLS:\n- agent_list: See all running agents and their capabilities\n- agent_send: Send a message to a specialist agent and get their response\n- agent_spawn: Create new agents when needed\n- agent_kill: Terminate agents no longer needed\n- memory_store: Save results and state to shared memory\n- memory_recall: Retrieve shared data from memory\n\nSPECIALIST AGENTS (spawn or message these):\n- coder: Writes and reviews code\n- researcher: Gathers information\n- writer: Creates documentation and content\n- ops: DevOps, system operations\n- analyst: Data analysis and metrics\n- architect: System design and architecture\n- debugger: Bug hunting and root cause analysis\n- security-auditor: Security review and vulnerability assessment\n- test-engineer: Test design and quality assurance\n\nWORKFLOW:\n1. Analyze the user's request\n2. Use agent_list to see available agents\n3. Break the task into subtasks\n4. Delegate each subtask to the most appropriate specialist via agent_send\n5. Synthesize all responses into a coherent final answer\n6. Store important results in shared memory for future reference\n\nAlways explain your delegation strategy before executing it.\nBe thorough but efficient — don't delegate trivially simple tasks.\"\"\"\n\n[[fallback_models]]\nprovider = \"default\"\nmodel = \"default\"\napi_key_env = \"GROQ_API_KEY\"\n\n[schedule]\ncontinuous = { check_interval_secs = 120 }\n\n[resources]\nmax_llm_tokens_per_hour = 500000\n\n[capabilities]\ntools = [\"agent_send\", \"agent_spawn\", \"agent_list\", \"agent_kill\", \"memory_store\", \"memory_recall\", \"file_read\", \"file_write\"]\nmemory_read = [\"*\"]\nmemory_write = [\"*\"]\nagent_spawn = true\nagent_message = [\"*\"]\n"}}, "model_catalog.rs": {"df9ee62446a0ade999546c911d2f659bc7d192dc": {"t": 1785746230.5896184, "n": 10761, "h": "47167ae0e61ed746", "c": "//! Model catalog types — shared data structures for the model registry.\n\nuse serde::{Deserialize, Serialize};\nuse std::fmt;\n\n// ---------------------------------------------------------------------------\n// Canonical provider base URLs — single source of truth.\n// Referenced by openfang-runtime drivers, model catalog, and embedding modules.\n// ---------------------------------------------------------------------------\n\npub const ANTHROPIC_BASE_URL: &str = \"https://api.anthropic.com\";\npub const OPENAI_BASE_URL: &str = \"https://api.openai.com/v1\";\npub const GEMINI_BASE_URL: &str = \"https://generativelanguage.googleapis.com\";\npub const DEEPSEEK_BASE_URL: &str = \"https://api.deepseek.com/v1\";\npub const GROQ_BASE_URL: &str = \"https://api.groq.com/openai/v1\";\npub const OPENROUTER_BASE_URL: &str = \"https://openrouter.ai/api/v1\";\npub const MISTRAL_BASE_URL: &str = \"https://api.mistral.ai/v1\";\npub const TOGETHER_BASE_URL: &str = \"https://api.together.xyz/v1\";\npub const FIREWORKS_BASE_URL: &str = \"https://api.fireworks.ai/inference/v1\";\npub const OLLAMA_BASE_URL: &str = \"http://localhost:11434/v1\";\npub const VLLM_BASE_URL: &str = \"http://localhost:8000/v1\";\npub const LMSTUDIO_BASE_URL: &str = \"http://localhost:1234/v1\";\npub const PERPLEXITY_BASE_URL: &str = \"https://api.perplexity.ai\";\npub const COHERE_BASE_URL: &str = \"https://api.cohere.com/v2\";\npub const AI21_BASE_URL: &str = \"https://api.ai21.com/studio/v1\";\npub const CEREBRAS_BASE_URL: &str = \"https://api.cerebras.ai/v1\";\npub const SAMBANOVA_BASE_URL: &str = \"https://api.sambanova.ai/v1\";\npub const HUGGINGFACE_BASE_URL: &str = \"https://api-inference.huggingface.co/v1\";\npub const XAI_BASE_URL: &str = \"https://api.x.ai/v1\";\npub const REPLICATE_BASE_URL: &str = \"https://api.replicate.com/v1\";\n\n// ── GitHub Copilot ──────────────────────────────────────────────\npub const GITHUB_COPILOT_BASE_URL: &str = \"https://api.githubcopilot.com\";\n\n// ── Chinese providers ─────────────────────────────────────────────\npub const QWEN_BASE_URL: &str = \"https://dashscope.aliyuncs.com/compatible-mode/v1\";\npub const MINIMAX_BASE_URL: &str = \"https://api.minimax.io/v1\";\npub const ZHIPU_BASE_URL: &str = \"https://open.bigmodel.cn/api/paas/v4\";\npub const ZHIPU_CODING_BASE_URL: &str = \"https://open.bigmodel.cn/api/coding/paas/v4\";\npub const MOONSHOT_BASE_URL: &str = \"https://api.moonshot.cn/v1\";\npub const QIANFAN_BASE_URL: &str = \"https://qianfan.baidubce.com/v2\";\npub const VOLCENGINE_BASE_URL: &str = \"https://ark.cn-beijing.volces.com/api/v3\";\n\n// ── AWS Bedrock ───────────────────────────────────────────────────\npub const BEDROCK_BASE_URL: &str = \"https://bedrock-runtime.us-east-1.amazonaws.com\";\n\n/// A model's capability tier.\n#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]\n#[serde(rename_all = \"lowercase\")]\npub enum ModelTier {\n /// Cutting-edge, most capable models (e.g. Claude Opus, GPT-4.1).\n Frontier,\n /// Smart, cost-effective models (e.g. Claude Sonnet, Gemini 2.5 Flash).\n Smart,\n /// Balanced speed/cost models (e.g. GPT-4o-mini, Groq Llama).\n #[default]\n Balanced,\n /// Fastest, cheapest models for simple tasks.\n Fast,\n /// Local models (Ollama, vLLM, LM Studio).\n Local,\n /// User-defined custom models added at runtime.\n Custom,\n}\n\nimpl fmt::Display for ModelTier {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self {\n ModelTier::Frontier => write!(f, \"frontier\"),\n ModelTier::Smart => write!(f, \"smart\"),\n ModelTier::Balanced => write!(f, \"balanced\"),\n ModelTier::Fast => write!(f, \"fast\"),\n ModelTier::Local => write!(f, \"local\"),\n ModelTier::Custom => write!(f, \"custom\"),\n }\n }\n}\n\n/// Provider authentication status.\n#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]\n#[serde(rename_all = \"snake_case\")]\npub enum AuthStatus {\n /// API key is present in the environment.\n Configured,\n /// API key is missing.\n #[default]\n Missing,\n /// No API key required (local providers).\n NotRequired,\n}\n\nimpl fmt::Display for AuthStatus {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self {\n AuthStatus::Configured => write!(f, \"configured\"),\n AuthStatus::Missing => write!(f, \"missing\"),\n AuthStatus::NotRequired => write!(f, \"not_required\"),\n }\n }\n}\n\n/// A single model entry in the catalog.\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ModelCatalogEntry {\n /// Canonical model identifier (e.g. \"claude-sonnet-4-20250514\").\n pub id: String,\n /// Human-readable display name (e.g. \"Claude Sonnet 4\").\n pub display_name: String,\n /// Provider identifier (e.g. \"anthropic\").\n pub provider: String,\n /// Capability tier.\n pub tier: ModelTier,\n /// Context window size in tokens.\n pub context_window: u64,\n /// Maximum output tokens.\n pub max_output_tokens: u64,\n /// Cost per million input tokens (USD).\n pub input_cost_per_m: f64,\n /// Cost per million output tokens (USD).\n pub output_cost_per_m: f64,\n /// Whether the model supports tool/function calling.\n pub supports_tools: bool,\n /// Whether the model supports vision/image inputs.\n pub supports_vision: bool,\n /// Whether the model supports streaming responses.\n pub supports_streaming: bool,\n /// Aliases for this model (e.g. [\"sonnet\", \"claude-sonnet\"]).\n #[serde(default)]\n pub aliases: Vec<String>,\n}\n\nimpl Default for ModelCatalogEntry {\n fn default() -> Self {\n Self {\n id: String::new(),\n display_name: String::new(),\n provider: String::new(),\n tier: ModelTier::default(),\n context_window: 0,\n max_output_tokens: 0,\n input_cost_per_m: 0.0,\n output_cost_per_m: 0.0,\n supports_tools: false,\n supports_vision: false,\n supports_streaming: false,\n aliases: Vec::new(),\n }\n }\n}\n\n/// Provider metadata.\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ProviderInfo {\n /// Provider identifier (e.g. \"anthropic\").\n pub id: String,\n /// Human-readable display name (e.g. \"Anthropic\").\n pub display_name: String,\n /// Environment variable name for the API key.\n pub api_key_env: String,\n /// Default base URL.\n pub base_url: String,\n /// Whether an API key is required (false for local providers).\n pub key_required: bool,\n /// Runtime-detected authentication status.\n pub auth_status: AuthStatus,\n /// Number of models from this provider in the catalog.\n pub model_count: usize,\n}\n\nimpl Default for ProviderInfo {\n fn default() -> Self {\n Self {\n id: String::new(),\n display_name: String::new(),\n api_key_env: String::new(),\n base_url: String::new(),\n key_required: true,\n auth_status: AuthStatus::default(),\n model_count: 0,\n }\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn test_model_tier_display() {\n assert_eq!(ModelTier::Frontier.to_string(), \"frontier\");\n assert_eq!(ModelTier::Smart.to_string(), \"smart\");\n assert_eq!(ModelTier::Balanced.to_string(), \"balanced\");\n assert_eq!(ModelTier::Fast.to_string(), \"fast\");\n assert_eq!(ModelTier::Local.to_string(), \"local\");\n assert_eq!(ModelTier::Custom.to_string(), \"custom\");\n }\n\n #[test]\n fn test_auth_status_display() {\n assert_eq!(AuthStatus::Configured.to_string(), \"configured\");\n assert_eq!(AuthStatus::Missing.to_string(), \"missing\");\n assert_eq!(AuthStatus::NotRequired.to_string(), \"not_required\");\n }\n\n #[test]\n fn test_model_tier_default() {\n assert_eq!(ModelTier::default(), ModelTier::Balanced);\n }\n\n #[test]\n fn test_auth_status_default() {\n assert_eq!(AuthStatus::default(), AuthStatus::Missing);\n }\n\n #[test]\n fn test_model_catalog_entry_default() {\n let entry = ModelCatalogEntry::default();\n assert!(entry.id.is_empty());\n assert_eq!(entry.tier, ModelTier::Balanced);\n assert!(entry.aliases.is_empty());\n }\n\n #[test]\n fn test_provider_info_default() {\n let info = ProviderInfo::default();\n assert!(info.id.is_empty());\n assert!(info.key_required);\n assert_eq!(info.auth_status, AuthStatus::Missing);\n }\n\n #[test]\n fn test_model_tier_serde_roundtrip() {\n let tier = ModelTier::Frontier;\n let json = serde_json::to_string(&tier).unwrap();\n assert_eq!(json, \"\\\"frontier\\\"\");\n let parsed: ModelTier = serde_json::from_str(&json).unwrap();\n assert_eq!(parsed, tier);\n }\n\n #[test]\n fn test_auth_status_serde_roundtrip() {\n let status = AuthStatus::Configured;\n let json = serde_json::to_string(&status).unwrap();\n assert_eq!(json, \"\\\"configured\\\"\");\n let parsed: AuthStatus = serde_json::from_str(&json).unwrap();\n assert_eq!(parsed, status);\n }\n\n #[test]\n fn test_model_entry_serde_roundtrip() {\n let entry = ModelCatalogEntry {\n id: \"claude-sonnet-4-20250514\".to_string(),\n display_name: \"Claude Sonnet 4\".to_string(),\n provider: \"anthropic\".to_string(),\n tier: ModelTier::Smart,\n context_window: 200_000,\n max_output_tokens: 64_000,\n input_cost_per_m: 3.0,\n output_cost_per_m: 15.0,\n supports_tools: true,\n supports_vision: true,\n supports_streaming: true,\n aliases: vec![\"sonnet\".to_string(), \"claude-sonnet\".to_string()],\n };\n let json = serde_json::to_string(&entry).unwrap();\n let parsed: ModelCatalogEntry = serde_json::from_str(&json).unwrap();\n assert_eq!(parsed.id, entry.id);\n assert_eq!(parsed.tier, ModelTier::Smart);\n assert_eq!(parsed.aliases.len(), 2);\n }\n\n #[test]\n fn test_provider_info_serde_roundtrip() {\n let info = ProviderInfo {\n id: \"anthropic\".to_string(),\n display_name: \"Anthropic\".to_string(),\n api_key_env: \"ANTHROPIC_API_KEY\".to_string(),\n base_url: \"https://api.anthropic.com\".to_string(),\n key_required: true,\n auth_status: AuthStatus::Configured,\n model_count: 3,\n };\n let json = serde_json::to_string(&info).unwrap();\n let parsed: ProviderInfo = serde_json::from_str(&json).unwrap();\n assert_eq!(parsed.id, \"anthropic\");\n assert_eq!(parsed.auth_status, AuthStatus::Configured);\n assert_eq!(parsed.model_count, 3);\n }\n}\n"}, "bedbb7e69b5fbe666cd8cdc6020e565ba90947e5": {"t": 1785750559.6834269, "n": 10761, "h": "47167ae0e61ed746", "c": "//! Model catalog types — shared data structures for the model registry.\n\nuse serde::{Deserialize, Serialize};\nuse std::fmt;\n\n// ---------------------------------------------------------------------------\n// Canonical provider base URLs — single source of truth.\n// Referenced by openfang-runtime drivers, model catalog, and embedding modules.\n// ---------------------------------------------------------------------------\n\npub const ANTHROPIC_BASE_URL: &str = \"https://api.anthropic.com\";\npub const OPENAI_BASE_URL: &str = \"https://api.openai.com/v1\";\npub const GEMINI_BASE_URL: &str = \"https://generativelanguage.googleapis.com\";\npub const DEEPSEEK_BASE_URL: &str = \"https://api.deepseek.com/v1\";\npub const GROQ_BASE_URL: &str = \"https://api.groq.com/openai/v1\";\npub const OPENROUTER_BASE_URL: &str = \"https://openrouter.ai/api/v1\";\npub const MISTRAL_BASE_URL: &str = \"https://api.mistral.ai/v1\";\npub const TOGETHER_BASE_URL: &str = \"https://api.together.xyz/v1\";\npub const FIREWORKS_BASE_URL: &str = \"https://api.fireworks.ai/inference/v1\";\npub const OLLAMA_BASE_URL: &str = \"http://localhost:11434/v1\";\npub const VLLM_BASE_URL: &str = \"http://localhost:8000/v1\";\npub const LMSTUDIO_BASE_URL: &str = \"http://localhost:1234/v1\";\npub const PERPLEXITY_BASE_URL: &str = \"https://api.perplexity.ai\";\npub const COHERE_BASE_URL: &str = \"https://api.cohere.com/v2\";\npub const AI21_BASE_URL: &str = \"https://api.ai21.com/studio/v1\";\npub const CEREBRAS_BASE_URL: &str = \"https://api.cerebras.ai/v1\";\npub const SAMBANOVA_BASE_URL: &str = \"https://api.sambanova.ai/v1\";\npub const HUGGINGFACE_BASE_URL: &str = \"https://api-inference.huggingface.co/v1\";\npub const XAI_BASE_URL: &str = \"https://api.x.ai/v1\";\npub const REPLICATE_BASE_URL: &str = \"https://api.replicate.com/v1\";\n\n// ── GitHub Copilot ──────────────────────────────────────────────\npub const GITHUB_COPILOT_BASE_URL: &str = \"https://api.githubcopilot.com\";\n\n// ── Chinese providers ─────────────────────────────────────────────\npub const QWEN_BASE_URL: &str = \"https://dashscope.aliyuncs.com/compatible-mode/v1\";\npub const MINIMAX_BASE_URL: &str = \"https://api.minimax.io/v1\";\npub const ZHIPU_BASE_URL: &str = \"https://open.bigmodel.cn/api/paas/v4\";\npub const ZHIPU_CODING_BASE_URL: &str = \"https://open.bigmodel.cn/api/coding/paas/v4\";\npub const MOONSHOT_BASE_URL: &str = \"https://api.moonshot.cn/v1\";\npub const QIANFAN_BASE_URL: &str = \"https://qianfan.baidubce.com/v2\";\npub const VOLCENGINE_BASE_URL: &str = \"https://ark.cn-beijing.volces.com/api/v3\";\n\n// ── AWS Bedrock ───────────────────────────────────────────────────\npub const BEDROCK_BASE_URL: &str = \"https://bedrock-runtime.us-east-1.amazonaws.com\";\n\n/// A model's capability tier.\n#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Hash, Serialize, Deserialize)]\n#[serde(rename_all = \"lowercase\")]\npub enum ModelTier {\n /// Cutting-edge, most capable models (e.g. Claude Opus, GPT-4.1).\n Frontier,\n /// Smart, cost-effective models (e.g. Claude Sonnet, Gemini 2.5 Flash).\n Smart,\n /// Balanced speed/cost models (e.g. GPT-4o-mini, Groq Llama).\n #[default]\n Balanced,\n /// Fastest, cheapest models for simple tasks.\n Fast,\n /// Local models (Ollama, vLLM, LM Studio).\n Local,\n /// User-defined custom models added at runtime.\n Custom,\n}\n\nimpl fmt::Display for ModelTier {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self {\n ModelTier::Frontier => write!(f, \"frontier\"),\n ModelTier::Smart => write!(f, \"smart\"),\n ModelTier::Balanced => write!(f, \"balanced\"),\n ModelTier::Fast => write!(f, \"fast\"),\n ModelTier::Local => write!(f, \"local\"),\n ModelTier::Custom => write!(f, \"custom\"),\n }\n }\n}\n\n/// Provider authentication status.\n#[derive(Debug, Default, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]\n#[serde(rename_all = \"snake_case\")]\npub enum AuthStatus {\n /// API key is present in the environment.\n Configured,\n /// API key is missing.\n #[default]\n Missing,\n /// No API key required (local providers).\n NotRequired,\n}\n\nimpl fmt::Display for AuthStatus {\n fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {\n match self {\n AuthStatus::Configured => write!(f, \"configured\"),\n AuthStatus::Missing => write!(f, \"missing\"),\n AuthStatus::NotRequired => write!(f, \"not_required\"),\n }\n }\n}\n\n/// A single model entry in the catalog.\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ModelCatalogEntry {\n /// Canonical model identifier (e.g. \"claude-sonnet-4-20250514\").\n pub id: String,\n /// Human-readable display name (e.g. \"Claude Sonnet 4\").\n pub display_name: String,\n /// Provider identifier (e.g. \"anthropic\").\n pub provider: String,\n /// Capability tier.\n pub tier: ModelTier,\n /// Context window size in tokens.\n pub context_window: u64,\n /// Maximum output tokens.\n pub max_output_tokens: u64,\n /// Cost per million input tokens (USD).\n pub input_cost_per_m: f64,\n /// Cost per million output tokens (USD).\n pub output_cost_per_m: f64,\n /// Whether the model supports tool/function calling.\n pub supports_tools: bool,\n /// Whether the model supports vision/image inputs.\n pub supports_vision: bool,\n /// Whether the model supports streaming responses.\n pub supports_streaming: bool,\n /// Aliases for this model (e.g. [\"sonnet\", \"claude-sonnet\"]).\n #[serde(default)]\n pub aliases: Vec<String>,\n}\n\nimpl Default for ModelCatalogEntry {\n fn default() -> Self {\n Self {\n id: String::new(),\n display_name: String::new(),\n provider: String::new(),\n tier: ModelTier::default(),\n context_window: 0,\n max_output_tokens: 0,\n input_cost_per_m: 0.0,\n output_cost_per_m: 0.0,\n supports_tools: false,\n supports_vision: false,\n supports_streaming: false,\n aliases: Vec::new(),\n }\n }\n}\n\n/// Provider metadata.\n#[derive(Debug, Clone, Serialize, Deserialize)]\npub struct ProviderInfo {\n /// Provider identifier (e.g. \"anthropic\").\n pub id: String,\n /// Human-readable display name (e.g. \"Anthropic\").\n pub display_name: String,\n /// Environment variable name for the API key.\n pub api_key_env: String,\n /// Default base URL.\n pub base_url: String,\n /// Whether an API key is required (false for local providers).\n pub key_required: bool,\n /// Runtime-detected authentication status.\n pub auth_status: AuthStatus,\n /// Number of models from this provider in the catalog.\n pub model_count: usize,\n}\n\nimpl Default for ProviderInfo {\n fn default() -> Self {\n Self {\n id: String::new(),\n display_name: String::new(),\n api_key_env: String::new(),\n base_url: String::new(),\n key_required: true,\n auth_status: AuthStatus::default(),\n model_count: 0,\n }\n }\n}\n\n#[cfg(test)]\nmod tests {\n use super::*;\n\n #[test]\n fn test_model_tier_display() {\n assert_eq!(ModelTier::Frontier.to_string(), \"frontier\");\n assert_eq!(ModelTier::Smart.to_string(), \"smart\");\n assert_eq!(ModelTier::Balanced.to_string(), \"balanced\");\n assert_eq!(ModelTier::Fast.to_string(), \"fast\");\n assert_eq!(ModelTier::Local.to_string(), \"local\");\n assert_eq!(ModelTier::Custom.to_string(), \"custom\");\n }\n\n #[test]\n fn test_auth_status_display() {\n assert_eq!(AuthStatus::Configured.to_string(), \"configured\");\n assert_eq!(AuthStatus::Missing.to_string(), \"missing\");\n assert_eq!(AuthStatus::NotRequired.to_string(), \"not_required\");\n }\n\n #[test]\n fn test_model_tier_default() {\n assert_eq!(ModelTier::default(), ModelTier::Balanced);\n }\n\n #[test]\n fn test_auth_status_default() {\n assert_eq!(AuthStatus::default(), AuthStatus::Missing);\n }\n\n #[test]\n fn test_model_catalog_entry_default() {\n let entry = ModelCatalogEntry::default();\n assert!(entry.id.is_empty());\n assert_eq!(entry.tier, ModelTier::Balanced);\n assert!(entry.aliases.is_empty());\n }\n\n #[test]\n fn test_provider_info_default() {\n let info = ProviderInfo::default();\n assert!(info.id.is_empty());\n assert!(info.key_required);\n assert_eq!(info.auth_status, AuthStatus::Missing);\n }\n\n #[test]\n fn test_model_tier_serde_roundtrip() {\n let tier = ModelTier::Frontier;\n let json = serde_json::to_string(&tier).unwrap();\n assert_eq!(json, \"\\\"frontier\\\"\");\n let parsed: ModelTier = serde_json::from_str(&json).unwrap();\n assert_eq!(parsed, tier);\n }\n\n #[test]\n fn test_auth_status_serde_roundtrip() {\n let status = AuthStatus::Configured;\n let json = serde_json::to_string(&status).unwrap();\n assert_eq!(json, \"\\\"configured\\\"\");\n let parsed: AuthStatus = serde_json::from_str(&json).unwrap();\n assert_eq!(parsed, status);\n }\n\n #[test]\n fn test_model_entry_serde_roundtrip() {\n let entry = ModelCatalogEntry {\n id: \"claude-sonnet-4-20250514\".to_string(),\n display_name: \"Claude Sonnet 4\".to_string(),\n provider: \"anthropic\".to_string(),\n tier: ModelTier::Smart,\n context_window: 200_000,\n max_output_tokens: 64_000,\n input_cost_per_m: 3.0,\n output_cost_per_m: 15.0,\n supports_tools: true,\n supports_vision: true,\n supports_streaming: true,\n aliases: vec![\"sonnet\".to_string(), \"claude-sonnet\".to_string()],\n };\n let json = serde_json::to_string(&entry).unwrap();\n let parsed: ModelCatalogEntry = serde_json::from_str(&json).unwrap();\n assert_eq!(parsed.id, entry.id);\n assert_eq!(parsed.tier, ModelTier::Smart);\n assert_eq!(parsed.aliases.len(), 2);\n }\n\n #[test]\n fn test_provider_info_serde_roundtrip() {\n let info = ProviderInfo {\n id: \"anthropic\".to_string(),\n display_name: \"Anthropic\".to_string(),\n api_key_env: \"ANTHROPIC_API_KEY\".to_string(),\n base_url: \"https://api.anthropic.com\".to_string(),\n key_required: true,\n auth_status: AuthStatus::Configured,\n model_count: 3,\n };\n let json = serde_json::to_string(&info).unwrap();\n let parsed: ProviderInfo = serde_json::from_str(&json).unwrap();\n assert_eq!(parsed.id, \"anthropic\");\n assert_eq!(parsed.auth_status, AuthStatus::Configured);\n assert_eq!(parsed.model_count, 3);\n }\n}\n"}}, "providers.md": {"df9ee62446a0ade999546c911d2f659bc7d192dc": {"t": 1785746559.6998248, "n": 32990, "h": "239ba3b7aae6a01e", "c": "# LLM Providers Guide\n\nOpenFang ships with a comprehensive model catalog covering **3 native LLM drivers**, **20 providers**, **51 builtin models**, and **23 aliases**. Every provider uses one of three battle-tested drivers: the native **Anthropic** driver, the native **Gemini** driver, or the universal **OpenAI-compatible** driver. This guide is the single source of truth for configuring, selecting, and managing LLM providers in OpenFang.\n\n---\n\n## Table of Contents\n\n1. [Quick Setup](#quick-setup)\n2. [Provider Reference](#provider-reference)\n3. [Model Catalog](#model-catalog)\n4. [Model Aliases](#model-aliases)\n5. [Per-Agent Model Override](#per-agent-model-override)\n6. [Model Routing](#model-routing)\n7. [Cost Tracking](#cost-tracking)\n8. [Fallback Providers](#fallback-providers)\n9. [API Endpoints](#api-endpoints)\n10. [Channel Commands](#channel-commands)\n\n---\n\n## Quick Setup\n\nThe fastest path from zero to running:\n\n```bash\n# Pick ONE provider — set its env var — done.\nexport GEMINI_API_KEY=\"your-key\" # Free tier available\n# OR\nexport GROQ_API_KEY=\"your-key\" # Free tier available\n# OR\nexport ANTHROPIC_API_KEY=\"your-key\"\n# OR\nexport OPENAI_API_KEY=\"your-key\"\n```\n\nOpenFang auto-detects which providers have API keys configured at boot. Any model whose provider is authenticated becomes immediately available. Local providers (Ollama, vLLM, LM Studio) require no key at all.\n\nFor Gemini specifically, either `GEMINI_API_KEY` or `GOOGLE_API_KEY` will work.\n\n---\n\n## Provider Reference\n\n### 1. Anthropic\n\n| | |\n|---|---|\n| **Display Name** | Anthropic |\n| **Driver** | Native Anthropic (Messages API) |\n| **Env Var** | `ANTHROPIC_API_KEY` |\n| **Base URL** | `https://api.anthropic.com` |\n| **Key Required** | Yes |\n| **Free Tier** | No |\n| **Auth** | `x-api-key` header |\n| **Models** | 3 |\n\n**Available Models:**\n- `claude-opus-4-20250514` (Frontier)\n- `claude-sonnet-4-20250514` (Smart)\n- `claude-haiku-4-5-20251001` (Fast)\n\n**Setup:**\n1. Sign up at [console.anthropic.com](https://console.anthropic.com)\n2. Create an API key under Settings > API Keys\n3. `export ANTHROPIC_API_KEY=\"sk-ant-...\"`\n\n---\n\n### 2. OpenAI\n\n| | |\n|---|---|\n| **Display Name** | OpenAI |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `OPENAI_API_KEY` |\n| **Base URL** | `https://api.openai.com/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | No |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 6 |\n\n**Available Models:**\n- `gpt-4.1` (Frontier)\n- `gpt-4o` (Smart)\n- `o3-mini` (Smart)\n- `gpt-4.1-mini` (Balanced)\n- `gpt-4o-mini` (Fast)\n- `gpt-4.1-nano` (Fast)\n\n**Setup:**\n1. Sign up at [platform.openai.com](https://platform.openai.com)\n2. Create an API key under API Keys\n3. `export OPENAI_API_KEY=\"sk-...\"`\n\n---\n\n### 3. Google Gemini\n\n| | |\n|---|---|\n| **Display Name** | Google Gemini |\n| **Driver** | Native Gemini (generateContent API) |\n| **Env Var** | `GEMINI_API_KEY` (or `GOOGLE_API_KEY`) |\n| **Base URL** | `https://generativelanguage.googleapis.com` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (generous free tier) |\n| **Auth** | `x-goog-api-key` header |\n| **Models** | 3 |\n\n**Available Models:**\n- `gemini-2.5-pro` (Frontier)\n- `gemini-2.5-flash` (Smart)\n- `gemini-2.0-flash` (Fast)\n\n**Setup:**\n1. Go to [aistudio.google.com](https://aistudio.google.com)\n2. Get an API key (free tier included)\n3. `export GEMINI_API_KEY=\"AIza...\"` or `export GOOGLE_API_KEY=\"AIza...\"`\n\n**Notes:** The Gemini driver is a fully native implementation. It is not OpenAI-compatible. Model goes in the URL path, system prompt via `systemInstruction`, tools via `functionDeclarations`, streaming via `streamGenerateContent?alt=sse`.\n\n---\n\n### 4. DeepSeek\n\n| | |\n|---|---|\n| **Display Name** | DeepSeek |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `DEEPSEEK_API_KEY` |\n| **Base URL** | `https://api.deepseek.com/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | No |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 2 |\n\n**Available Models:**\n- `deepseek-chat` (Smart) -- DeepSeek V3\n- `deepseek-reasoner` (Smart) -- DeepSeek R1, no tool support\n\n**Setup:**\n1. Sign up at [platform.deepseek.com](https://platform.deepseek.com)\n2. Create an API key\n3. `export DEEPSEEK_API_KEY=\"sk-...\"`\n\n---\n\n### 5. Groq\n\n| | |\n|---|---|\n| **Display Name** | Groq |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `GROQ_API_KEY` |\n| **Base URL** | `https://api.groq.com/openai/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (rate-limited) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 4 |\n\n**Available Models:**\n- `llama-3.3-70b-versatile` (Balanced)\n- `mixtral-8x7b-32768` (Balanced)\n- `llama-3.1-8b-instant` (Fast)\n- `gemma2-9b-it` (Fast)\n\n**Setup:**\n1. Sign up at [console.groq.com](https://console.groq.com)\n2. Create an API key\n3. `export GROQ_API_KEY=\"gsk_...\"`\n\n**Notes:** Groq runs open-source models on custom LPU hardware. Extremely fast inference. Free tier has rate limits but is very usable.\n\n---\n\n### 6. OpenRouter\n\n| | |\n|---|---|\n| **Display Name** | OpenRouter |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `OPENROUTER_API_KEY` |\n| **Base URL** | `https://openrouter.ai/api/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (limited credits for some models) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 3 |\n\n**Available Models:**\n- `openrouter/auto` (Smart) -- auto-selects best model\n- `openrouter/optimus` (Balanced) -- cost-optimized\n- `openrouter/nitro` (Fast) -- speed-optimized\n\n**Setup:**\n1. Sign up at [openrouter.ai](https://openrouter.ai)\n2. Create an API key under Keys\n3. `export OPENROUTER_API_KEY=\"sk-or-...\"`\n\n**Notes:** OpenRouter is a unified gateway to 200+ models from many providers. The three builtin entries are OpenRouter's smart-routing endpoints. You can also use any model ID from their catalog directly by specifying the full OpenRouter model path.\n\n---\n\n### 7. Mistral AI\n\n| | |\n|---|---|\n| **Display Name** | Mistral AI |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `MISTRAL_API_KEY` |\n| **Base URL** | `https://api.mistral.ai/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | No |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 3 |\n\n**Available Models:**\n- `mistral-large-latest` (Smart)\n- `codestral-latest` (Smart)\n- `mistral-small-latest` (Fast)\n\n**Setup:**\n1. Sign up at [console.mistral.ai](https://console.mistral.ai)\n2. Create an API key\n3. `export MISTRAL_API_KEY=\"...\"`\n\n---\n\n### 8. Together AI\n\n| | |\n|---|---|\n| **Display Name** | Together AI |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `TOGETHER_API_KEY` |\n| **Base URL** | `https://api.together.xyz/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (limited credits on signup) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 3 |\n\n**Available Models:**\n- `meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo` (Frontier)\n- `Qwen/Qwen2.5-72B-Instruct-Turbo` (Smart)\n- `mistralai/Mixtral-8x22B-Instruct-v0.1` (Balanced)\n\n**Setup:**\n1. Sign up at [api.together.ai](https://api.together.ai)\n2. Create an API key\n3. `export TOGETHER_API_KEY=\"...\"`\n\n---\n\n### 9. Fireworks AI\n\n| | |\n|---|---|\n| **Display Name** | Fireworks AI |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `FIREWORKS_API_KEY` |\n| **Base URL** | `https://api.fireworks.ai/inference/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (limited credits on signup) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 2 |\n\n**Available Models:**\n- `accounts/fireworks/models/llama-v3p1-405b-instruct` (Frontier)\n- `accounts/fireworks/models/mixtral-8x22b-instruct` (Balanced)\n\n**Setup:**\n1. Sign up at [fireworks.ai](https://fireworks.ai)\n2. Create an API key\n3. `export FIREWORKS_API_KEY=\"...\"`\n\n---\n\n### 10. Ollama\n\n| | |\n|---|---|\n| **Display Name** | Ollama |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `OLLAMA_API_KEY` (not required) |\n| **Base URL** | `http://localhost:11434/v1` |\n| **Key Required** | **No** |\n| **Free Tier** | Free (local) |\n| **Auth** | None (local) |\n| **Models** | 3 builtin + auto-discovered |\n\n**Available Models (builtin):**\n- `llama3.2` (Local)\n- `mistral:latest` (Local)\n- `phi3` (Local)\n\n**Setup:**\n1. Install Ollama from [ollama.com](https://ollama.com)\n2. Pull a model: `ollama pull llama3.2`\n3. Start the server: `ollama serve`\n4. No env var needed -- Ollama is always available\n\n**Notes:** OpenFang auto-discovers models from a running Ollama instance and merges them into the catalog with `Local` tier and zero cost. Any model you pull becomes usable immediately.\n\n---\n\n### 11. vLLM\n\n| | |\n|---|---|\n| **Display Name** | vLLM |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `VLLM_API_KEY` (not required) |\n| **Base URL** | `http://localhost:8000/v1` |\n| **Key Required** | **No** |\n| **Free Tier** | Free (self-hosted) |\n| **Auth** | None (local) |\n| **Models** | 1 builtin + auto-discovered |\n\n**Available Models (builtin):**\n- `vllm-local` (Local)\n\n**Setup:**\n1. Install vLLM: `pip install vllm`\n2. Start the server: `python -m vllm.entrypoints.openai.api_server --model <model-name>`\n3. No env var needed\n\n---\n\n### 12. LM Studio\n\n| | |\n|---|---|\n| **Display Name** | LM Studio |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `LMSTUDIO_API_KEY` (not required) |\n| **Base URL** | `http://localhost:1234/v1` |\n| **Key Required** | **No** |\n| **Free Tier** | Free (local) |\n| **Auth** | None (local) |\n| **Models** | 1 builtin + auto-discovered |\n\n**Available Models (builtin):**\n- `lmstudio-local` (Local)\n\n**Setup:**\n1. Download LM Studio from [lmstudio.ai](https://lmstudio.ai)\n2. Download a model from the built-in model browser\n3. Start the local server from the \"Local Server\" tab\n4. No env var needed\n\n---\n\n### 13. Perplexity AI\n\n| | |\n|---|---|\n| **Display Name** | Perplexity AI |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `PERPLEXITY_API_KEY` |\n| **Base URL** | `https://api.perplexity.ai` |\n| **Key Required** | Yes |\n| **Free Tier** | No |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 2 |\n\n**Available Models:**\n- `sonar-pro` (Smart) -- online search-augmented\n- `sonar` (Balanced) -- online search-augmented\n\n**Setup:**\n1. Sign up at [perplexity.ai](https://www.perplexity.ai)\n2. Go to API settings and generate a key\n3. `export PERPLEXITY_API_KEY=\"pplx-...\"`\n\n**Notes:** Perplexity models have built-in web search. They do not support tool use.\n\n---\n\n### 14. Cohere\n\n| | |\n|---|---|\n| **Display Name** | Cohere |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `COHERE_API_KEY` |\n| **Base URL** | `https://api.cohere.com/v2` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (rate-limited trial) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 2 |\n\n**Available Models:**\n- `command-r-plus` (Smart)\n- `command-r` (Balanced)\n\n**Setup:**\n1. Sign up at [dashboard.cohere.com](https://dashboard.cohere.com)\n2. Create an API key\n3. `export COHERE_API_KEY=\"...\"`\n\n---\n\n### 15. AI21 Labs\n\n| | |\n|---|---|\n| **Display Name** | AI21 Labs |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `AI21_API_KEY` |\n| **Base URL** | `https://api.ai21.com/studio/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (limited credits) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 1 |\n\n**Available Models:**\n- `jamba-1.5-large` (Smart)\n\n**Setup:**\n1. Sign up at [studio.ai21.com](https://studio.ai21.com)\n2. Create an API key\n3. `export AI21_API_KEY=\"...\"`\n\n---\n\n### 16. Cerebras\n\n| | |\n|---|---|\n| **Display Name** | Cerebras |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `CEREBRAS_API_KEY` |\n| **Base URL** | `https://api.cerebras.ai/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (generous free tier) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 2 |\n\n**Available Models:**\n- `cerebras/llama3.3-70b` (Balanced)\n- `cerebras/llama3.1-8b` (Fast)\n\n**Setup:**\n1. Sign up at [cloud.cerebras.ai](https://cloud.cerebras.ai)\n2. Create an API key\n3. `export CEREBRAS_API_KEY=\"...\"`\n\n**Notes:** Cerebras runs inference on wafer-scale chips. Ultra-fast and ultra-cheap ($0.06/M tokens for both input and output on the 70B model).\n\n---\n\n### 17. SambaNova\n\n| | |\n|---|---|\n| **Display Name** | SambaNova |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `SAMBANOVA_API_KEY` |\n| **Base URL** | `https://api.sambanova.ai/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (limited credits) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 1 |\n\n**Available Models:**\n- `sambanova/llama-3.3-70b` (Balanced)\n\n**Setup:**\n1. Sign up at [cloud.sambanova.ai](https://cloud.sambanova.ai)\n2. Create an API key\n3. `export SAMBANOVA_API_KEY=\"...\"`\n\n---\n\n### 18. Hugging Face\n\n| | |\n|---|---|\n| **Display Name** | Hugging Face |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `HF_API_KEY` |\n| **Base URL** | `https://api-inference.huggingface.co/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (rate-limited) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 1 |\n\n**Available Models:**\n- `hf/meta-llama/Llama-3.3-70B-Instruct` (Balanced)\n\n**Setup:**\n1. Sign up at [huggingface.co](https://huggingface.co)\n2. Create a token under Settings > Access Tokens\n3. `export HF_API_KEY=\"hf_...\"`\n\n---\n\n### 19. xAI\n\n| | |\n|---|---|\n| **Display Name** | xAI |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `XAI_API_KEY` |\n| **Base URL** | `https://api.x.ai/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (limited free credits) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 2 |\n\n**Available Models:**\n- `grok-2` (Smart) -- supports vision\n- `grok-2-mini` (Fast)\n\n**Setup:**\n1. Sign up at [console.x.ai](https://console.x.ai)\n2. Create an API key\n3. `export XAI_API_KEY=\"xai-...\"`\n\n---\n\n### 20. Replicate\n\n| | |\n|---|---|\n| **Display Name** | Replicate |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `REPLICATE_API_TOKEN` |\n| **Base URL** | `https://api.replicate.com/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | No |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 1 |\n\n**Available Models:**\n- `replicate/meta-llama-3.3-70b-instruct` (Balanced)\n\n**Setup:**\n1. Sign up at [replicate.com](https://replicate.com)\n2. Go to Account > API Tokens\n3. `export REPLICATE_API_TOKEN=\"r8_...\"`\n\n---\n\n## Model Catalog\n\nThe complete catalog of all 51 builtin models, sorted by provider. Pricing is per million tokens.\n\n| # | Model ID | Display Name | Provider | Tier | Context Window | Max Output | Input $/M | Output $/M | Tools | Vision |\n|---|----------|-------------|----------|------|---------------|------------|-----------|------------|-------|--------|\n| 1 | `claude-opus-4-20250514` | Claude Opus 4 | anthropic | Frontier | 200,000 | 32,000 | $15.00 | $75.00 | Yes | Yes |\n| 2 | `claude-sonnet-4-20250514` | Claude Sonnet 4 | anthropic | Smart | 200,000 | 64,000 | $3.00 | $15.00 | Yes | Yes |\n| 3 | `claude-haiku-4-5-20251001` | Claude Haiku 4.5 | anthropic | Fast | 200,000 | 8,192 | $0.25 | $1.25 | Yes | Yes |\n| 4 | `gpt-4.1` | GPT-4.1 | openai | Frontier | 1,047,576 | 32,768 | $2.00 | $8.00 | Yes | Yes |\n| 5 | `gpt-4o` | GPT-4o | openai | Smart | 128,000 | 16,384 | $2.50 | $10.00 | Yes | Yes |\n| 6 | `o3-mini` | o3-mini | openai | Smart | 200,000 | 100,000 | $1.10 | $4.40 | Yes | No |\n| 7 | `gpt-4.1-mini` | GPT-4.1 Mini | openai | Balanced | 1,047,576 | 32,768 | $0.40 | $1.60 | Yes | Yes |\n| 8 | `gpt-4o-mini` | GPT-4o Mini | openai | Fast | 128,000 | 16,384 | $0.15 | $0.60 | Yes | Yes |\n| 9 | `gpt-4.1-nano` | GPT-4.1 Nano | openai | Fast | 1,047,576 | 32,768 | $0.10 | $0.40 | Yes | No |\n| 10 | `gemini-2.5-pro` | Gemini 2.5 Pro | gemini | Frontier | 1,048,576 | 65,536 | $1.25 | $10.00 | Yes | Yes |\n| 11 | `gemini-2.5-flash` | Gemini 2.5 Flash | gemini | Smart | 1,048,576 | 65,536 | $0.15 | $0.60 | Yes | Yes |\n| 12 | `gemini-2.0-flash` | Gemini 2.0 Flash | gemini | Fast | 1,048,576 | 8,192 | $0.10 | $0.40 | Yes | Yes |\n| 13 | `deepseek-chat` | DeepSeek V3 | deepseek | Smart | 64,000 | 8,192 | $0.27 | $1.10 | Yes | No |\n| 14 | `deepseek-reasoner` | DeepSeek R1 | deepseek | Smart | 64,000 | 8,192 | $0.55 | $2.19 | No | No |\n| 15 | `llama-3.3-70b-versatile` | Llama 3.3 70B | groq | Balanced | 128,000 | 32,768 | $0.059 | $0.079 | Yes | No |\n| 16 | `mixtral-8x7b-32768` | Mixtral 8x7B | groq | Balanced | 32,768 | 4,096 | $0.024 | $0.024 | Yes | No |\n| 17 | `llama-3.1-8b-instant` | Llama 3.1 8B | groq | Fast | 128,000 | 8,192 | $0.05 | $0.08 | Yes | No |\n| 18 | `gemma2-9b-it` | Gemma 2 9B | groq | Fast | 8,192 | 4,096 | $0.02 | $0.02 | No | No |\n| 19 | `openrouter/auto` | OpenRouter Auto | openrouter | Smart | 200,000 | 32,000 | $1.00 | $3.00 | Yes | Yes |\n| 20 | `openrouter/optimus` | OpenRouter Optimus | openrouter | Balanced | 200,000 | 32,000 | $0.50 | $1.50 | Yes | No |\n| 21 | `openrouter/nitro` | OpenRouter Nitro | openrouter | Fast | 128,000 | 16,000 | $0.20 | $0.60 | Yes | No |\n| 22 | `mistral-large-latest` | Mistral Large | mistral | Smart | 128,000 | 8,192 | $2.00 | $6.00 | Yes | No |\n| 23 | `codestral-latest` | Codestral | mistral | Smart | 32,000 | 8,192 | $0.30 | $0.90 | Yes | No |\n| 24 | `mistral-small-latest` | Mistral Small | mistral | Fast | 128,000 | 8,192 | $0.10 | $0.30 | Yes | No |\n| 25 | `meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo` | Llama 3.1 405B (Together) | together | Frontier | 130,000 | 4,096 | $3.50 | $3.50 | Yes | No |\n| 26 | `Qwen/Qwen2.5-72B-Instruct-Turbo` | Qwen 2.5 72B (Together) | together | Smart | 32,768 | 4,096 | $0.20 | $0.60 | Yes | No |\n| 27 | `mistralai/Mixtral-8x22B-Instruct-v0.1` | Mixtral 8x22B (Together) | together | Balanced | 65,536 | 4,096 | $0.60 | $0.60 | Yes | No |\n| 28 | `accounts/fireworks/models/llama-v3p1-405b-instruct` | Llama 3.1 405B (Fireworks) | fireworks | Frontier | 131,072 | 16,384 | $3.00 | $3.00 | Yes | No |\n| 29 | `accounts/fireworks/models/mixtral-8x22b-instruct` | Mixtral 8x22B (Fireworks) | fireworks | Balanced | 65,536 | 4,096 | $0.90 | $0.90 | Yes | No |\n| 30 | `llama3.2` | Llama 3.2 (Ollama) | ollama | Local | 128,000 | 4,096 | $0.00 | $0.00 | Yes | No |\n| 31 | `mistral:latest` | Mistral (Ollama) | ollama | Local | 32,768 | 4,096 | $0.00 | $0.00 | Yes | No |\n| 32 | `phi3` | Phi-3 (Ollama) | ollama | Local | 128,000 | 4,096 | $0.00 | $0.00 | No | No |\n| 33 | `vllm-local` | vLLM Local Model | vllm | Local | 32,768 | 4,096 | $0.00 | $0.00 | Yes | No |\n| 34 | `lmstudio-local` | LM Studio Local Model | lmstudio | Local | 32,768 | 4,096 | $0.00 | $0.00 | Yes | No |\n| 35 | `sonar-pro` | Sonar Pro | perplexity | Smart | 200,000 | 8,192 | $3.00 | $15.00 | No | No |\n| 36 | `sonar` | Sonar | perplexity | Balanced | 128,000 | 8,192 | $1.00 | $5.00 | No | No |\n| 37 | `command-r-plus` | Command R+ | cohere | Smart | 128,000 | 4,096 | $2.50 | $10.00 | Yes | No |\n| 38 | `command-r` | Command R | cohere | Balanced | 128,000 | 4,096 | $0.15 | $0.60 | Yes | No |\n| 39 | `jamba-1.5-large` | Jamba 1.5 Large | ai21 | Smart | 256,000 | 4,096 | $2.00 | $8.00 | Yes | No |\n| 40 | `cerebras/llama3.3-70b` | Llama 3.3 70B (Cerebras) | cerebras | Balanced | 128,000 | 8,192 | $0.06 | $0.06 | Yes | No |\n| 41 | `cerebras/llama3.1-8b` | Llama 3.1 8B (Cerebras) | cerebras | Fast | 128,000 | 8,192 | $0.01 | $0.01 | Yes | No |\n| 42 | `sambanova/llama-3.3-70b` | Llama 3.3 70B (SambaNova) | sambanova | Balanced | 128,000 | 8,192 | $0.06 | $0.06 | Yes | No |\n| 43 | `grok-2` | Grok 2 | xai | Smart | 131,072 | 32,768 | $2.00 | $10.00 | Yes | Yes |\n| 44 | `grok-2-mini` | Grok 2 Mini | xai | Fast | 131,072 | 32,768 | $0.30 | $0.50 | Yes | No |\n| 45 | `hf/meta-llama/Llama-3.3-70B-Instruct` | Llama 3.3 70B (HF) | huggingface | Balanced | 128,000 | 4,096 | $0.30 | $0.30 | No | No |\n| 46 | `replicate/meta-llama-3.3-70b-instruct` | Llama 3.3 70B (Replicate) | replicate | Balanced | 128,000 | 4,096 | $0.40 | $0.40 | No | No |\n\n**Model Tiers:**\n\n| Tier | Description | Typical Use |\n|------|------------|------------|\n| **Frontier** | Most capable, highest cost | Orchestration, architecture, security audits |\n| **Smart** | Strong reasoning, moderate cost | Coding, code review, research, analysis |\n| **Balanced** | Good cost/quality tradeoff | Planning, writing, DevOps, day-to-day tasks |\n| **Fast** | Cheapest cloud inference | Ops, translation, simple Q&A, health checks |\n| **Local** | Self-hosted, zero cost | Privacy-first, offline, development |\n\n**Notes:**\n- Local providers (Ollama, vLLM, LM Studio) auto-discover models at runtime. Any model you download and serve will be merged into the catalog with `Local` tier and zero cost.\n- The 46 entries above are the builtin models. The total of 51 referenced in the catalog includes runtime auto-discovered models that vary per installation.\n\n---\n\n## Model Aliases\n\nAll 23 aliases resolve to canonical model IDs. Aliases are case-insensitive.\n\n| Alias | Resolves To |\n|-------|------------|\n| `sonnet` | `claude-sonnet-4-20250514` |\n| `claude-sonnet` | `claude-sonnet-4-20250514` |\n| `haiku` | `claude-haiku-4-5-20251001` |\n| `claude-haiku` | `claude-haiku-4-5-20251001` |\n| `opus` | `claude-opus-4-20250514` |\n| `claude-opus` | `claude-opus-4-20250514` |\n| `gpt4` | `gpt-4o` |\n| `gpt4o` | `gpt-4o` |\n| `gpt4-mini` | `gpt-4o-mini` |\n| `flash` | `gemini-2.5-flash` |\n| `gemini-flash` | `gemini-2.5-flash` |\n| `gemini-pro` | `gemini-2.5-pro` |\n| `deepseek` | `deepseek-chat` |\n| `llama` | `llama-3.3-70b-versatile` |\n| `llama-70b` | `llama-3.3-70b-versatile` |\n| `mixtral` | `mixtral-8x7b-32768` |\n| `mistral` | `mistral-large-latest` |\n| `codestral` | `codestral-latest` |\n| `grok` | `grok-2` |\n| `grok-mini` | `grok-2-mini` |\n| `sonar` | `sonar-pro` |\n| `jamba` | `jamba-1.5-large` |\n| `command-r` | `command-r-plus` |\n\nYou can use aliases anywhere a model ID is accepted: in config files, REST API calls, chat commands, and the model routing configuration.\n\n---\n\n## Per-Agent Model Override\n\nEach agent in your `config.toml` can specify its own model, overriding the global default:\n\n```toml\n# Global default model\n[agents.defaults]\nmodel = \"claude-sonnet-4-20250514\"\n\n# Per-agent override: use an alias or full model ID\n[[agents]]\nname = \"orchestrator\"\nmodel = \"opus\" # alias for claude-opus-4-20250514\n\n[[agents]]\nname = \"ops\"\nmodel = \"llama-3.3-70b-versatile\" # cheap Groq model for simple ops\n\n[[agents]]\nname = \"coder\"\nmodel = \"gemini-2.5-flash\" # fast + cheap + 1M context\n\n[[agents]]\nname = \"researcher\"\nmodel = \"sonar-pro\" # Perplexity with built-in web search\n\n# You can also pin a model in the agent manifest TOML\n[[agents]]\nname = \"production-bot\"\npinned_model = \"claude-sonnet-4-20250514\" # never auto-routed\n```\n\nWhen `pinned_model` is set on an agent manifest, that agent always uses the specified model regardless of routing configuration. This is used in **Stabilisation mode** (`KernelMode::Stable`) where the model is frozen for production reliability.\n\n---\n\n## Model Routing\n\nOpenFang can automatically select the cheapest model capable of handling each query. This is configured per-agent via `ModelRoutingConfig`.\n\n### How It Works\n\n1. The **ModelRouter** scores each incoming `CompletionRequest` based on heuristics\n2. The score maps to a **TaskComplexity** tier: `Simple`, `Medium`, or `Complex`\n3. Each tier has a pre-configured model\n\n### Scoring Heuristics\n\n| Signal | Weight | Logic |\n|--------|--------|-------|\n| Total message length | 1 point per ~4 chars | Rough token proxy |\n| Tool availability | +20 per tool defined | Tools imply multi-step work |\n| Code markers | +30 per marker found | Backticks, `fn`, `def`, `class`, `import`, `function`, `async`, `await`, `struct`, `impl`, `return` |\n| Conversation depth | +15 per message > 10 | Deep context = harder reasoning |\n| System prompt length | +1 per 10 chars > 500 | Long system prompts imply complex tasks |\n\n### Thresholds\n\n| Complexity | Score Range | Default Model |\n|-----------|-------------|---------------|\n| Simple | score < 100 | `claude-haiku-4-5-20251001` |\n| Medium | 100 <= score < 500 | `claude-sonnet-4-20250514` |\n| Complex | score >= 500 | `claude-sonnet-4-20250514` |\n\n### Configuration\n\n```toml\n# In agent manifest or config.toml\n[routing]\nsimple_model = \"claude-haiku-4-5-20251001\"\nmedium_model = \"gemini-2.5-flash\"\ncomplex_model = \"claude-sonnet-4-20250514\"\nsimple_threshold = 100\ncomplex_threshold = 500\n```\n\nThe router also integrates with the model catalog:\n- **`validate_models()`** checks that all configured model IDs exist in the catalog\n- **`resolve_aliases()`** expands aliases to canonical IDs (e.g., `\"sonnet\"` becomes `\"claude-sonnet-4-20250514\"`)\n\n---\n\n## Cost Tracking\n\nOpenFang tracks the cost of every LLM call and can enforce per-agent spending quotas.\n\n### Per-Response Cost Estimation\n\nAfter each LLM call, cost is calculated as:\n\n```\ncost = (input_tokens / 1,000,000) * input_rate + (output_tokens / 1,000,000) * output_rate\n```\n\nThe `MeteringEngine` first checks the **model catalog** for exact pricing. If the model is not found, it falls back to a pattern-matching heuristic.\n\n### Cost Rates (per million tokens)\n\n| Model Pattern | Input $/M | Output $/M |\n|--------------|-----------|------------|\n| `*haiku*` | $0.25 | $1.25 |\n| `*sonnet*` | $3.00 | $15.00 |\n| `*opus*` | $15.00 | $75.00 |\n| `gpt-4o-mini` | $0.15 | $0.60 |\n| `gpt-4o` | $2.50 | $10.00 |\n| `gpt-4.1-nano` | $0.10 | $0.40 |\n| `gpt-4.1-mini` | $0.40 | $1.60 |\n| `gpt-4.1` | $2.00 | $8.00 |\n| `o3-mini` | $1.10 | $4.40 |\n| `gemini-2.5-pro` | $1.25 | $10.00 |\n| `gemini-2.5-flash` | $0.15 | $0.60 |\n| `gemini-2.0-flash` | $0.10 | $0.40 |\n| `deepseek-reasoner` / `deepseek-r1` | $0.55 | $2.19 |\n| `*deepseek*` | $0.27 | $1.10 |\n| `*cerebras*` | $0.06 | $0.06 |\n| `*sambanova*` | $0.06 | $0.06 |\n| `*replicate*` | $0.40 | $0.40 |\n| `*llama*` / `*mixtral*` | $0.05 | $0.10 |\n| `*qwen*` | $0.20 | $0.60 |\n| `mistral-large*` | $2.00 | $6.00 |\n| `*mistral*` (other) | $0.10 | $0.30 |\n| `command-r-plus` | $2.50 | $10.00 |\n| `command-r` | $0.15 | $0.60 |\n| `sonar-pro` | $3.00 | $15.00 |\n| `*sonar*` (other) | $1.00 | $5.00 |\n| `grok-2-mini` / `grok-mini` | $0.30 | $0.50 |\n| `*grok*` (other) | $2.00 | $10.00 |\n| `*jamba*` | $2.00 | $8.00 |\n| Default (unknown) | $1.00 | $3.00 |\n\n### Quota Enforcement\n\nQuotas are checked on every LLM call. If the agent exceeds its hourly limit, the call is rejected with a `QuotaExceeded` error.\n\n```toml\n# Per-agent quota in config.toml\n[[agents]]\nname = \"chatbot\"\n[agents.resources]\nmax_cost_per_hour_usd = 5.00 # cap at $5/hour\n```\n\nThe usage footer (when enabled) appends cost information to each response:\n\n```\n> Cost: $0.0042 | Tokens: 1,200 in / 340 out | Model: claude-sonnet-4-20250514\n```\n\n---\n\n## Fallback Providers\n\nThe `FallbackDriver` wraps multiple LLM drivers in a chain. If the primary driver fails, the next driver in the chain is tried automatically.\n\n### Behavior\n\n- On success: returns immediately\n- On **rate limit / overload** errors (`429`, `529`): bubbles up for retry logic (does NOT failover, because the primary should be retried after backoff)\n- On **all other errors**: logs a warning and tries the next driver in the chain\n- If all drivers fail: returns the last error\n\n### Configuration\n\nFallback chains are configured in your agent manifest or `config.toml`. The `FallbackDriver` is used automatically when an agent is in **Stabilisation mode** (`KernelMode::Stable`) or when multiple providers are configured for reliability.\n\n```toml\n# Example: primary Anthropic, fallback to Gemini, then Groq\n[[agents]]\nname = \"production-bot\"\nmodel = \"claude-sonnet-4-20250514\"\nfallback_models = [\"gemini-2.5-flash\", \"llama-3.3-70b-versatile\"]\n```\n\nThe fallback driver creates a chain: `AnthropicDriver -> GeminiDriver -> OpenAIDriver(Groq)`.\n\n---\n\n## API Endpoints\n\n### List All Models\n\n```\nGET /api/models\n```\n\nReturns the complete model catalog with metadata, pricing, and feature flags.\n\n**Response:**\n```json\n[\n {\n \"id\": \"claude-sonnet-4-20250514\",\n \"display_name\": \"Claude Sonnet 4\",\n \"provider\": \"anthropic\",\n \"tier\": \"Smart\",\n \"context_window\": 200000,\n \"max_output_tokens\": 64000,\n \"input_cost_per_m\": 3.0,\n \"output_cost_per_m\": 15.0,\n \"supports_tools\": true,\n \"supports_vision\": true,\n \"supports_streaming\": true,\n \"aliases\": [\"sonnet\", \"claude-sonnet\"]\n }\n]\n```\n\n### Get Specific Model\n\n```\nGET /api/models/{id}\n```\n\nReturns a single model entry. Supports both canonical IDs and aliases.\n\n```\nGET /api/models/sonnet\nGET /api/models/claude-sonnet-4-20250514\n```\n\n### List Aliases\n\n```\nGET /api/models/aliases\n```\n\nReturns a map of all alias-to-canonical-ID mappings.\n\n**Response:**\n```json\n{\n \"sonnet\": \"claude-sonnet-4-20250514\",\n \"haiku\": \"claude-haiku-4-5-20251001\",\n \"flash\": \"gemini-2.5-flash\",\n \"grok\": \"grok-2\"\n}\n```\n\n### List Providers\n\n```\nGET /api/providers\n```\n\nReturns all 20 providers with auth status and model counts.\n\n**Response:**\n```json\n[\n {\n \"id\": \"anthropic\",\n \"display_name\": \"Anthropic\",\n \"api_key_env\": \"ANTHROPIC_API_KEY\",\n \"base_url\": \"https://api.anthropic.com\",\n \"key_required\": true,\n \"auth_status\": \"Configured\",\n \"model_count\": 3\n },\n {\n \"id\": \"ollama\",\n \"display_name\": \"Ollama\",\n \"api_key_env\": \"OLLAMA_API_KEY\",\n \"base_url\": \"http://localhost:11434/v1\",\n \"key_required\": false,\n \"auth_status\": \"NotRequired\",\n \"model_count\": 5\n }\n]\n```\n\nAuth status values: `Configured`, `Missing`, `NotRequired`.\n\n### Set Provider API Key\n\n```\nPOST /api/providers/{name}/key\nContent-Type: application/json\n\n{ \"api_key\": \"sk-...\" }\n```\n\nConfigures an API key for a provider at runtime (stored as a `Zeroizing<String>`, wiped from memory on drop).\n\n### Remove Provider API Key\n\n```\nDELETE /api/providers/{name}/key\n```\n\nRemoves the configured API key for a provider.\n\n### Test Provider Connection\n\n```\nPOST /api/providers/{name}/test\n```\n\nSends a minimal test request to verify the provider is reachable and the API key is valid.\n\n---\n\n## Channel Commands\n\nTwo chat commands are available in any channel for inspecting models and providers:\n\n### `/models`\n\nLists all available models with their tier, provider, and context window. Only shows models from providers that have authentication configured (or do not require it).\n\n```\n/models\n```\n\nExample output:\n```\nAvailable models (12):\n\nFrontier:\n claude-opus-4-20250514 (Anthropic) — 200K ctx\n gemini-2.5-pro (Google Gemini) — 1M ctx\n\nSmart:\n claude-sonnet-4-20250514 (Anthropic) — 200K ctx\n gemini-2.5-flash (Google Gemini) — 1M ctx\n deepseek-chat (DeepSeek) — 64K ctx\n\nBalanced:\n llama-3.3-70b-versatile (Groq) — 128K ctx\n\nFast:\n claude-haiku-4-5-20251001 (Anthropic) — 200K ctx\n gemini-2.0-flash (Google Gemini) — 1M ctx\n\nLocal:\n llama3.2 (Ollama) — 128K ctx\n```\n\n### `/providers`\n\nLists all 20 providers with their authentication status.\n\n```\n/providers\n```\n\nExample output:\n```\nLLM Providers (20):\n\n Anthropic ANTHROPIC_API_KEY Configured 3 models\n OpenAI OPENAI_API_KEY Missing 6 models\n Google Gemini GEMINI_API_KEY Configured 3 models\n DeepSeek DEEPSEEK_API_KEY Missing 2 models\n Groq GROQ_API_KEY Configured 4 models\n Ollama (no key needed) Ready 3 models\n vLLM (no key needed) Ready 1 model\n LM Studio (no key needed) Ready 1 model\n ...\n```\n\n---\n\n## Environment Variables Summary\n\nQuick reference for all provider environment variables:\n\n| Provider | Env Var | Required |\n|----------|---------|----------|\n| Anthropic | `ANTHROPIC_API_KEY` | Yes |\n| OpenAI | `OPENAI_API_KEY` | Yes |\n| Google Gemini | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | Yes |\n| DeepSeek | `DEEPSEEK_API_KEY` | Yes |\n| Groq | `GROQ_API_KEY` | Yes |\n| OpenRouter | `OPENROUTER_API_KEY` | Yes |\n| Mistral AI | `MISTRAL_API_KEY` | Yes |\n| Together AI | `TOGETHER_API_KEY` | Yes |\n| Fireworks AI | `FIREWORKS_API_KEY` | Yes |\n| Ollama | `OLLAMA_API_KEY` | No |\n| vLLM | `VLLM_API_KEY` | No |\n| LM Studio | `LMSTUDIO_API_KEY` | No |\n| Perplexity AI | `PERPLEXITY_API_KEY` | Yes |\n| Cohere | `COHERE_API_KEY` | Yes |\n| AI21 Labs | `AI21_API_KEY` | Yes |\n| Cerebras | `CEREBRAS_API_KEY` | Yes |\n| SambaNova | `SAMBANOVA_API_KEY` | Yes |\n| Hugging Face | `HF_API_KEY` | Yes |\n| xAI | `XAI_API_KEY` | Yes |\n| Replicate | `REPLICATE_API_TOKEN` | Yes |\n\n---\n\n## Security Notes\n\n- All API keys are stored as `Zeroizing<String>` -- the key material is automatically overwritten with zeros when the value is dropped from memory.\n- Auth detection (`detect_auth()`) only checks `std::env::var()` for presence -- it never reads or logs the actual secret value.\n- Provider API keys set via the REST API (`POST /api/providers/{name}/key`) follow the same zeroization policy.\n- The health endpoint (`/api/health`) never exposes provider auth status or API keys. Detailed info is behind `/api/health/detail` which requires authentication.\n- All `DriverConfig` and `KernelConfig` structs implement `Debug` with secret redaction -- API keys are printed as `\"***\"` in logs.\n"}, "f8ca6086281ebe455c6dbe55fca32620bd112d9d": {"t": 1785750557.7211196, "n": 32990, "h": "239ba3b7aae6a01e", "c": "# LLM Providers Guide\n\nOpenFang ships with a comprehensive model catalog covering **3 native LLM drivers**, **20 providers**, **51 builtin models**, and **23 aliases**. Every provider uses one of three battle-tested drivers: the native **Anthropic** driver, the native **Gemini** driver, or the universal **OpenAI-compatible** driver. This guide is the single source of truth for configuring, selecting, and managing LLM providers in OpenFang.\n\n---\n\n## Table of Contents\n\n1. [Quick Setup](#quick-setup)\n2. [Provider Reference](#provider-reference)\n3. [Model Catalog](#model-catalog)\n4. [Model Aliases](#model-aliases)\n5. [Per-Agent Model Override](#per-agent-model-override)\n6. [Model Routing](#model-routing)\n7. [Cost Tracking](#cost-tracking)\n8. [Fallback Providers](#fallback-providers)\n9. [API Endpoints](#api-endpoints)\n10. [Channel Commands](#channel-commands)\n\n---\n\n## Quick Setup\n\nThe fastest path from zero to running:\n\n```bash\n# Pick ONE provider — set its env var — done.\nexport GEMINI_API_KEY=\"your-key\" # Free tier available\n# OR\nexport GROQ_API_KEY=\"your-key\" # Free tier available\n# OR\nexport ANTHROPIC_API_KEY=\"your-key\"\n# OR\nexport OPENAI_API_KEY=\"your-key\"\n```\n\nOpenFang auto-detects which providers have API keys configured at boot. Any model whose provider is authenticated becomes immediately available. Local providers (Ollama, vLLM, LM Studio) require no key at all.\n\nFor Gemini specifically, either `GEMINI_API_KEY` or `GOOGLE_API_KEY` will work.\n\n---\n\n## Provider Reference\n\n### 1. Anthropic\n\n| | |\n|---|---|\n| **Display Name** | Anthropic |\n| **Driver** | Native Anthropic (Messages API) |\n| **Env Var** | `ANTHROPIC_API_KEY` |\n| **Base URL** | `https://api.anthropic.com` |\n| **Key Required** | Yes |\n| **Free Tier** | No |\n| **Auth** | `x-api-key` header |\n| **Models** | 3 |\n\n**Available Models:**\n- `claude-opus-4-20250514` (Frontier)\n- `claude-sonnet-4-20250514` (Smart)\n- `claude-haiku-4-5-20251001` (Fast)\n\n**Setup:**\n1. Sign up at [console.anthropic.com](https://console.anthropic.com)\n2. Create an API key under Settings > API Keys\n3. `export ANTHROPIC_API_KEY=\"sk-ant-...\"`\n\n---\n\n### 2. OpenAI\n\n| | |\n|---|---|\n| **Display Name** | OpenAI |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `OPENAI_API_KEY` |\n| **Base URL** | `https://api.openai.com/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | No |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 6 |\n\n**Available Models:**\n- `gpt-4.1` (Frontier)\n- `gpt-4o` (Smart)\n- `o3-mini` (Smart)\n- `gpt-4.1-mini` (Balanced)\n- `gpt-4o-mini` (Fast)\n- `gpt-4.1-nano` (Fast)\n\n**Setup:**\n1. Sign up at [platform.openai.com](https://platform.openai.com)\n2. Create an API key under API Keys\n3. `export OPENAI_API_KEY=\"sk-...\"`\n\n---\n\n### 3. Google Gemini\n\n| | |\n|---|---|\n| **Display Name** | Google Gemini |\n| **Driver** | Native Gemini (generateContent API) |\n| **Env Var** | `GEMINI_API_KEY` (or `GOOGLE_API_KEY`) |\n| **Base URL** | `https://generativelanguage.googleapis.com` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (generous free tier) |\n| **Auth** | `x-goog-api-key` header |\n| **Models** | 3 |\n\n**Available Models:**\n- `gemini-2.5-pro` (Frontier)\n- `gemini-2.5-flash` (Smart)\n- `gemini-2.0-flash` (Fast)\n\n**Setup:**\n1. Go to [aistudio.google.com](https://aistudio.google.com)\n2. Get an API key (free tier included)\n3. `export GEMINI_API_KEY=\"AIza...\"` or `export GOOGLE_API_KEY=\"AIza...\"`\n\n**Notes:** The Gemini driver is a fully native implementation. It is not OpenAI-compatible. Model goes in the URL path, system prompt via `systemInstruction`, tools via `functionDeclarations`, streaming via `streamGenerateContent?alt=sse`.\n\n---\n\n### 4. DeepSeek\n\n| | |\n|---|---|\n| **Display Name** | DeepSeek |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `DEEPSEEK_API_KEY` |\n| **Base URL** | `https://api.deepseek.com/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | No |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 2 |\n\n**Available Models:**\n- `deepseek-chat` (Smart) -- DeepSeek V3\n- `deepseek-reasoner` (Smart) -- DeepSeek R1, no tool support\n\n**Setup:**\n1. Sign up at [platform.deepseek.com](https://platform.deepseek.com)\n2. Create an API key\n3. `export DEEPSEEK_API_KEY=\"sk-...\"`\n\n---\n\n### 5. Groq\n\n| | |\n|---|---|\n| **Display Name** | Groq |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `GROQ_API_KEY` |\n| **Base URL** | `https://api.groq.com/openai/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (rate-limited) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 4 |\n\n**Available Models:**\n- `llama-3.3-70b-versatile` (Balanced)\n- `mixtral-8x7b-32768` (Balanced)\n- `llama-3.1-8b-instant` (Fast)\n- `gemma2-9b-it` (Fast)\n\n**Setup:**\n1. Sign up at [console.groq.com](https://console.groq.com)\n2. Create an API key\n3. `export GROQ_API_KEY=\"gsk_...\"`\n\n**Notes:** Groq runs open-source models on custom LPU hardware. Extremely fast inference. Free tier has rate limits but is very usable.\n\n---\n\n### 6. OpenRouter\n\n| | |\n|---|---|\n| **Display Name** | OpenRouter |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `OPENROUTER_API_KEY` |\n| **Base URL** | `https://openrouter.ai/api/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (limited credits for some models) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 3 |\n\n**Available Models:**\n- `openrouter/auto` (Smart) -- auto-selects best model\n- `openrouter/optimus` (Balanced) -- cost-optimized\n- `openrouter/nitro` (Fast) -- speed-optimized\n\n**Setup:**\n1. Sign up at [openrouter.ai](https://openrouter.ai)\n2. Create an API key under Keys\n3. `export OPENROUTER_API_KEY=\"sk-or-...\"`\n\n**Notes:** OpenRouter is a unified gateway to 200+ models from many providers. The three builtin entries are OpenRouter's smart-routing endpoints. You can also use any model ID from their catalog directly by specifying the full OpenRouter model path.\n\n---\n\n### 7. Mistral AI\n\n| | |\n|---|---|\n| **Display Name** | Mistral AI |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `MISTRAL_API_KEY` |\n| **Base URL** | `https://api.mistral.ai/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | No |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 3 |\n\n**Available Models:**\n- `mistral-large-latest` (Smart)\n- `codestral-latest` (Smart)\n- `mistral-small-latest` (Fast)\n\n**Setup:**\n1. Sign up at [console.mistral.ai](https://console.mistral.ai)\n2. Create an API key\n3. `export MISTRAL_API_KEY=\"...\"`\n\n---\n\n### 8. Together AI\n\n| | |\n|---|---|\n| **Display Name** | Together AI |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `TOGETHER_API_KEY` |\n| **Base URL** | `https://api.together.xyz/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (limited credits on signup) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 3 |\n\n**Available Models:**\n- `meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo` (Frontier)\n- `Qwen/Qwen2.5-72B-Instruct-Turbo` (Smart)\n- `mistralai/Mixtral-8x22B-Instruct-v0.1` (Balanced)\n\n**Setup:**\n1. Sign up at [api.together.ai](https://api.together.ai)\n2. Create an API key\n3. `export TOGETHER_API_KEY=\"...\"`\n\n---\n\n### 9. Fireworks AI\n\n| | |\n|---|---|\n| **Display Name** | Fireworks AI |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `FIREWORKS_API_KEY` |\n| **Base URL** | `https://api.fireworks.ai/inference/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (limited credits on signup) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 2 |\n\n**Available Models:**\n- `accounts/fireworks/models/llama-v3p1-405b-instruct` (Frontier)\n- `accounts/fireworks/models/mixtral-8x22b-instruct` (Balanced)\n\n**Setup:**\n1. Sign up at [fireworks.ai](https://fireworks.ai)\n2. Create an API key\n3. `export FIREWORKS_API_KEY=\"...\"`\n\n---\n\n### 10. Ollama\n\n| | |\n|---|---|\n| **Display Name** | Ollama |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `OLLAMA_API_KEY` (not required) |\n| **Base URL** | `http://localhost:11434/v1` |\n| **Key Required** | **No** |\n| **Free Tier** | Free (local) |\n| **Auth** | None (local) |\n| **Models** | 3 builtin + auto-discovered |\n\n**Available Models (builtin):**\n- `llama3.2` (Local)\n- `mistral:latest` (Local)\n- `phi3` (Local)\n\n**Setup:**\n1. Install Ollama from [ollama.com](https://ollama.com)\n2. Pull a model: `ollama pull llama3.2`\n3. Start the server: `ollama serve`\n4. No env var needed -- Ollama is always available\n\n**Notes:** OpenFang auto-discovers models from a running Ollama instance and merges them into the catalog with `Local` tier and zero cost. Any model you pull becomes usable immediately.\n\n---\n\n### 11. vLLM\n\n| | |\n|---|---|\n| **Display Name** | vLLM |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `VLLM_API_KEY` (not required) |\n| **Base URL** | `http://localhost:8000/v1` |\n| **Key Required** | **No** |\n| **Free Tier** | Free (self-hosted) |\n| **Auth** | None (local) |\n| **Models** | 1 builtin + auto-discovered |\n\n**Available Models (builtin):**\n- `vllm-local` (Local)\n\n**Setup:**\n1. Install vLLM: `pip install vllm`\n2. Start the server: `python -m vllm.entrypoints.openai.api_server --model <model-name>`\n3. No env var needed\n\n---\n\n### 12. LM Studio\n\n| | |\n|---|---|\n| **Display Name** | LM Studio |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `LMSTUDIO_API_KEY` (not required) |\n| **Base URL** | `http://localhost:1234/v1` |\n| **Key Required** | **No** |\n| **Free Tier** | Free (local) |\n| **Auth** | None (local) |\n| **Models** | 1 builtin + auto-discovered |\n\n**Available Models (builtin):**\n- `lmstudio-local` (Local)\n\n**Setup:**\n1. Download LM Studio from [lmstudio.ai](https://lmstudio.ai)\n2. Download a model from the built-in model browser\n3. Start the local server from the \"Local Server\" tab\n4. No env var needed\n\n---\n\n### 13. Perplexity AI\n\n| | |\n|---|---|\n| **Display Name** | Perplexity AI |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `PERPLEXITY_API_KEY` |\n| **Base URL** | `https://api.perplexity.ai` |\n| **Key Required** | Yes |\n| **Free Tier** | No |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 2 |\n\n**Available Models:**\n- `sonar-pro` (Smart) -- online search-augmented\n- `sonar` (Balanced) -- online search-augmented\n\n**Setup:**\n1. Sign up at [perplexity.ai](https://www.perplexity.ai)\n2. Go to API settings and generate a key\n3. `export PERPLEXITY_API_KEY=\"pplx-...\"`\n\n**Notes:** Perplexity models have built-in web search. They do not support tool use.\n\n---\n\n### 14. Cohere\n\n| | |\n|---|---|\n| **Display Name** | Cohere |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `COHERE_API_KEY` |\n| **Base URL** | `https://api.cohere.com/v2` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (rate-limited trial) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 2 |\n\n**Available Models:**\n- `command-r-plus` (Smart)\n- `command-r` (Balanced)\n\n**Setup:**\n1. Sign up at [dashboard.cohere.com](https://dashboard.cohere.com)\n2. Create an API key\n3. `export COHERE_API_KEY=\"...\"`\n\n---\n\n### 15. AI21 Labs\n\n| | |\n|---|---|\n| **Display Name** | AI21 Labs |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `AI21_API_KEY` |\n| **Base URL** | `https://api.ai21.com/studio/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (limited credits) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 1 |\n\n**Available Models:**\n- `jamba-1.5-large` (Smart)\n\n**Setup:**\n1. Sign up at [studio.ai21.com](https://studio.ai21.com)\n2. Create an API key\n3. `export AI21_API_KEY=\"...\"`\n\n---\n\n### 16. Cerebras\n\n| | |\n|---|---|\n| **Display Name** | Cerebras |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `CEREBRAS_API_KEY` |\n| **Base URL** | `https://api.cerebras.ai/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (generous free tier) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 2 |\n\n**Available Models:**\n- `cerebras/llama3.3-70b` (Balanced)\n- `cerebras/llama3.1-8b` (Fast)\n\n**Setup:**\n1. Sign up at [cloud.cerebras.ai](https://cloud.cerebras.ai)\n2. Create an API key\n3. `export CEREBRAS_API_KEY=\"...\"`\n\n**Notes:** Cerebras runs inference on wafer-scale chips. Ultra-fast and ultra-cheap ($0.06/M tokens for both input and output on the 70B model).\n\n---\n\n### 17. SambaNova\n\n| | |\n|---|---|\n| **Display Name** | SambaNova |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `SAMBANOVA_API_KEY` |\n| **Base URL** | `https://api.sambanova.ai/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (limited credits) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 1 |\n\n**Available Models:**\n- `sambanova/llama-3.3-70b` (Balanced)\n\n**Setup:**\n1. Sign up at [cloud.sambanova.ai](https://cloud.sambanova.ai)\n2. Create an API key\n3. `export SAMBANOVA_API_KEY=\"...\"`\n\n---\n\n### 18. Hugging Face\n\n| | |\n|---|---|\n| **Display Name** | Hugging Face |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `HF_API_KEY` |\n| **Base URL** | `https://api-inference.huggingface.co/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (rate-limited) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 1 |\n\n**Available Models:**\n- `hf/meta-llama/Llama-3.3-70B-Instruct` (Balanced)\n\n**Setup:**\n1. Sign up at [huggingface.co](https://huggingface.co)\n2. Create a token under Settings > Access Tokens\n3. `export HF_API_KEY=\"hf_...\"`\n\n---\n\n### 19. xAI\n\n| | |\n|---|---|\n| **Display Name** | xAI |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `XAI_API_KEY` |\n| **Base URL** | `https://api.x.ai/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | Yes (limited free credits) |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 2 |\n\n**Available Models:**\n- `grok-2` (Smart) -- supports vision\n- `grok-2-mini` (Fast)\n\n**Setup:**\n1. Sign up at [console.x.ai](https://console.x.ai)\n2. Create an API key\n3. `export XAI_API_KEY=\"xai-...\"`\n\n---\n\n### 20. Replicate\n\n| | |\n|---|---|\n| **Display Name** | Replicate |\n| **Driver** | OpenAI-compatible |\n| **Env Var** | `REPLICATE_API_TOKEN` |\n| **Base URL** | `https://api.replicate.com/v1` |\n| **Key Required** | Yes |\n| **Free Tier** | No |\n| **Auth** | `Authorization: Bearer` header |\n| **Models** | 1 |\n\n**Available Models:**\n- `replicate/meta-llama-3.3-70b-instruct` (Balanced)\n\n**Setup:**\n1. Sign up at [replicate.com](https://replicate.com)\n2. Go to Account > API Tokens\n3. `export REPLICATE_API_TOKEN=\"r8_...\"`\n\n---\n\n## Model Catalog\n\nThe complete catalog of all 51 builtin models, sorted by provider. Pricing is per million tokens.\n\n| # | Model ID | Display Name | Provider | Tier | Context Window | Max Output | Input $/M | Output $/M | Tools | Vision |\n|---|----------|-------------|----------|------|---------------|------------|-----------|------------|-------|--------|\n| 1 | `claude-opus-4-20250514` | Claude Opus 4 | anthropic | Frontier | 200,000 | 32,000 | $15.00 | $75.00 | Yes | Yes |\n| 2 | `claude-sonnet-4-20250514` | Claude Sonnet 4 | anthropic | Smart | 200,000 | 64,000 | $3.00 | $15.00 | Yes | Yes |\n| 3 | `claude-haiku-4-5-20251001` | Claude Haiku 4.5 | anthropic | Fast | 200,000 | 8,192 | $0.25 | $1.25 | Yes | Yes |\n| 4 | `gpt-4.1` | GPT-4.1 | openai | Frontier | 1,047,576 | 32,768 | $2.00 | $8.00 | Yes | Yes |\n| 5 | `gpt-4o` | GPT-4o | openai | Smart | 128,000 | 16,384 | $2.50 | $10.00 | Yes | Yes |\n| 6 | `o3-mini` | o3-mini | openai | Smart | 200,000 | 100,000 | $1.10 | $4.40 | Yes | No |\n| 7 | `gpt-4.1-mini` | GPT-4.1 Mini | openai | Balanced | 1,047,576 | 32,768 | $0.40 | $1.60 | Yes | Yes |\n| 8 | `gpt-4o-mini` | GPT-4o Mini | openai | Fast | 128,000 | 16,384 | $0.15 | $0.60 | Yes | Yes |\n| 9 | `gpt-4.1-nano` | GPT-4.1 Nano | openai | Fast | 1,047,576 | 32,768 | $0.10 | $0.40 | Yes | No |\n| 10 | `gemini-2.5-pro` | Gemini 2.5 Pro | gemini | Frontier | 1,048,576 | 65,536 | $1.25 | $10.00 | Yes | Yes |\n| 11 | `gemini-2.5-flash` | Gemini 2.5 Flash | gemini | Smart | 1,048,576 | 65,536 | $0.15 | $0.60 | Yes | Yes |\n| 12 | `gemini-2.0-flash` | Gemini 2.0 Flash | gemini | Fast | 1,048,576 | 8,192 | $0.10 | $0.40 | Yes | Yes |\n| 13 | `deepseek-chat` | DeepSeek V3 | deepseek | Smart | 64,000 | 8,192 | $0.27 | $1.10 | Yes | No |\n| 14 | `deepseek-reasoner` | DeepSeek R1 | deepseek | Smart | 64,000 | 8,192 | $0.55 | $2.19 | No | No |\n| 15 | `llama-3.3-70b-versatile` | Llama 3.3 70B | groq | Balanced | 128,000 | 32,768 | $0.059 | $0.079 | Yes | No |\n| 16 | `mixtral-8x7b-32768` | Mixtral 8x7B | groq | Balanced | 32,768 | 4,096 | $0.024 | $0.024 | Yes | No |\n| 17 | `llama-3.1-8b-instant` | Llama 3.1 8B | groq | Fast | 128,000 | 8,192 | $0.05 | $0.08 | Yes | No |\n| 18 | `gemma2-9b-it` | Gemma 2 9B | groq | Fast | 8,192 | 4,096 | $0.02 | $0.02 | No | No |\n| 19 | `openrouter/auto` | OpenRouter Auto | openrouter | Smart | 200,000 | 32,000 | $1.00 | $3.00 | Yes | Yes |\n| 20 | `openrouter/optimus` | OpenRouter Optimus | openrouter | Balanced | 200,000 | 32,000 | $0.50 | $1.50 | Yes | No |\n| 21 | `openrouter/nitro` | OpenRouter Nitro | openrouter | Fast | 128,000 | 16,000 | $0.20 | $0.60 | Yes | No |\n| 22 | `mistral-large-latest` | Mistral Large | mistral | Smart | 128,000 | 8,192 | $2.00 | $6.00 | Yes | No |\n| 23 | `codestral-latest` | Codestral | mistral | Smart | 32,000 | 8,192 | $0.30 | $0.90 | Yes | No |\n| 24 | `mistral-small-latest` | Mistral Small | mistral | Fast | 128,000 | 8,192 | $0.10 | $0.30 | Yes | No |\n| 25 | `meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo` | Llama 3.1 405B (Together) | together | Frontier | 130,000 | 4,096 | $3.50 | $3.50 | Yes | No |\n| 26 | `Qwen/Qwen2.5-72B-Instruct-Turbo` | Qwen 2.5 72B (Together) | together | Smart | 32,768 | 4,096 | $0.20 | $0.60 | Yes | No |\n| 27 | `mistralai/Mixtral-8x22B-Instruct-v0.1` | Mixtral 8x22B (Together) | together | Balanced | 65,536 | 4,096 | $0.60 | $0.60 | Yes | No |\n| 28 | `accounts/fireworks/models/llama-v3p1-405b-instruct` | Llama 3.1 405B (Fireworks) | fireworks | Frontier | 131,072 | 16,384 | $3.00 | $3.00 | Yes | No |\n| 29 | `accounts/fireworks/models/mixtral-8x22b-instruct` | Mixtral 8x22B (Fireworks) | fireworks | Balanced | 65,536 | 4,096 | $0.90 | $0.90 | Yes | No |\n| 30 | `llama3.2` | Llama 3.2 (Ollama) | ollama | Local | 128,000 | 4,096 | $0.00 | $0.00 | Yes | No |\n| 31 | `mistral:latest` | Mistral (Ollama) | ollama | Local | 32,768 | 4,096 | $0.00 | $0.00 | Yes | No |\n| 32 | `phi3` | Phi-3 (Ollama) | ollama | Local | 128,000 | 4,096 | $0.00 | $0.00 | No | No |\n| 33 | `vllm-local` | vLLM Local Model | vllm | Local | 32,768 | 4,096 | $0.00 | $0.00 | Yes | No |\n| 34 | `lmstudio-local` | LM Studio Local Model | lmstudio | Local | 32,768 | 4,096 | $0.00 | $0.00 | Yes | No |\n| 35 | `sonar-pro` | Sonar Pro | perplexity | Smart | 200,000 | 8,192 | $3.00 | $15.00 | No | No |\n| 36 | `sonar` | Sonar | perplexity | Balanced | 128,000 | 8,192 | $1.00 | $5.00 | No | No |\n| 37 | `command-r-plus` | Command R+ | cohere | Smart | 128,000 | 4,096 | $2.50 | $10.00 | Yes | No |\n| 38 | `command-r` | Command R | cohere | Balanced | 128,000 | 4,096 | $0.15 | $0.60 | Yes | No |\n| 39 | `jamba-1.5-large` | Jamba 1.5 Large | ai21 | Smart | 256,000 | 4,096 | $2.00 | $8.00 | Yes | No |\n| 40 | `cerebras/llama3.3-70b` | Llama 3.3 70B (Cerebras) | cerebras | Balanced | 128,000 | 8,192 | $0.06 | $0.06 | Yes | No |\n| 41 | `cerebras/llama3.1-8b` | Llama 3.1 8B (Cerebras) | cerebras | Fast | 128,000 | 8,192 | $0.01 | $0.01 | Yes | No |\n| 42 | `sambanova/llama-3.3-70b` | Llama 3.3 70B (SambaNova) | sambanova | Balanced | 128,000 | 8,192 | $0.06 | $0.06 | Yes | No |\n| 43 | `grok-2` | Grok 2 | xai | Smart | 131,072 | 32,768 | $2.00 | $10.00 | Yes | Yes |\n| 44 | `grok-2-mini` | Grok 2 Mini | xai | Fast | 131,072 | 32,768 | $0.30 | $0.50 | Yes | No |\n| 45 | `hf/meta-llama/Llama-3.3-70B-Instruct` | Llama 3.3 70B (HF) | huggingface | Balanced | 128,000 | 4,096 | $0.30 | $0.30 | No | No |\n| 46 | `replicate/meta-llama-3.3-70b-instruct` | Llama 3.3 70B (Replicate) | replicate | Balanced | 128,000 | 4,096 | $0.40 | $0.40 | No | No |\n\n**Model Tiers:**\n\n| Tier | Description | Typical Use |\n|------|------------|------------|\n| **Frontier** | Most capable, highest cost | Orchestration, architecture, security audits |\n| **Smart** | Strong reasoning, moderate cost | Coding, code review, research, analysis |\n| **Balanced** | Good cost/quality tradeoff | Planning, writing, DevOps, day-to-day tasks |\n| **Fast** | Cheapest cloud inference | Ops, translation, simple Q&A, health checks |\n| **Local** | Self-hosted, zero cost | Privacy-first, offline, development |\n\n**Notes:**\n- Local providers (Ollama, vLLM, LM Studio) auto-discover models at runtime. Any model you download and serve will be merged into the catalog with `Local` tier and zero cost.\n- The 46 entries above are the builtin models. The total of 51 referenced in the catalog includes runtime auto-discovered models that vary per installation.\n\n---\n\n## Model Aliases\n\nAll 23 aliases resolve to canonical model IDs. Aliases are case-insensitive.\n\n| Alias | Resolves To |\n|-------|------------|\n| `sonnet` | `claude-sonnet-4-20250514` |\n| `claude-sonnet` | `claude-sonnet-4-20250514` |\n| `haiku` | `claude-haiku-4-5-20251001` |\n| `claude-haiku` | `claude-haiku-4-5-20251001` |\n| `opus` | `claude-opus-4-20250514` |\n| `claude-opus` | `claude-opus-4-20250514` |\n| `gpt4` | `gpt-4o` |\n| `gpt4o` | `gpt-4o` |\n| `gpt4-mini` | `gpt-4o-mini` |\n| `flash` | `gemini-2.5-flash` |\n| `gemini-flash` | `gemini-2.5-flash` |\n| `gemini-pro` | `gemini-2.5-pro` |\n| `deepseek` | `deepseek-chat` |\n| `llama` | `llama-3.3-70b-versatile` |\n| `llama-70b` | `llama-3.3-70b-versatile` |\n| `mixtral` | `mixtral-8x7b-32768` |\n| `mistral` | `mistral-large-latest` |\n| `codestral` | `codestral-latest` |\n| `grok` | `grok-2` |\n| `grok-mini` | `grok-2-mini` |\n| `sonar` | `sonar-pro` |\n| `jamba` | `jamba-1.5-large` |\n| `command-r` | `command-r-plus` |\n\nYou can use aliases anywhere a model ID is accepted: in config files, REST API calls, chat commands, and the model routing configuration.\n\n---\n\n## Per-Agent Model Override\n\nEach agent in your `config.toml` can specify its own model, overriding the global default:\n\n```toml\n# Global default model\n[agents.defaults]\nmodel = \"claude-sonnet-4-20250514\"\n\n# Per-agent override: use an alias or full model ID\n[[agents]]\nname = \"orchestrator\"\nmodel = \"opus\" # alias for claude-opus-4-20250514\n\n[[agents]]\nname = \"ops\"\nmodel = \"llama-3.3-70b-versatile\" # cheap Groq model for simple ops\n\n[[agents]]\nname = \"coder\"\nmodel = \"gemini-2.5-flash\" # fast + cheap + 1M context\n\n[[agents]]\nname = \"researcher\"\nmodel = \"sonar-pro\" # Perplexity with built-in web search\n\n# You can also pin a model in the agent manifest TOML\n[[agents]]\nname = \"production-bot\"\npinned_model = \"claude-sonnet-4-20250514\" # never auto-routed\n```\n\nWhen `pinned_model` is set on an agent manifest, that agent always uses the specified model regardless of routing configuration. This is used in **Stabilisation mode** (`KernelMode::Stable`) where the model is frozen for production reliability.\n\n---\n\n## Model Routing\n\nOpenFang can automatically select the cheapest model capable of handling each query. This is configured per-agent via `ModelRoutingConfig`.\n\n### How It Works\n\n1. The **ModelRouter** scores each incoming `CompletionRequest` based on heuristics\n2. The score maps to a **TaskComplexity** tier: `Simple`, `Medium`, or `Complex`\n3. Each tier has a pre-configured model\n\n### Scoring Heuristics\n\n| Signal | Weight | Logic |\n|--------|--------|-------|\n| Total message length | 1 point per ~4 chars | Rough token proxy |\n| Tool availability | +20 per tool defined | Tools imply multi-step work |\n| Code markers | +30 per marker found | Backticks, `fn`, `def`, `class`, `import`, `function`, `async`, `await`, `struct`, `impl`, `return` |\n| Conversation depth | +15 per message > 10 | Deep context = harder reasoning |\n| System prompt length | +1 per 10 chars > 500 | Long system prompts imply complex tasks |\n\n### Thresholds\n\n| Complexity | Score Range | Default Model |\n|-----------|-------------|---------------|\n| Simple | score < 100 | `claude-haiku-4-5-20251001` |\n| Medium | 100 <= score < 500 | `claude-sonnet-4-20250514` |\n| Complex | score >= 500 | `claude-sonnet-4-20250514` |\n\n### Configuration\n\n```toml\n# In agent manifest or config.toml\n[routing]\nsimple_model = \"claude-haiku-4-5-20251001\"\nmedium_model = \"gemini-2.5-flash\"\ncomplex_model = \"claude-sonnet-4-20250514\"\nsimple_threshold = 100\ncomplex_threshold = 500\n```\n\nThe router also integrates with the model catalog:\n- **`validate_models()`** checks that all configured model IDs exist in the catalog\n- **`resolve_aliases()`** expands aliases to canonical IDs (e.g., `\"sonnet\"` becomes `\"claude-sonnet-4-20250514\"`)\n\n---\n\n## Cost Tracking\n\nOpenFang tracks the cost of every LLM call and can enforce per-agent spending quotas.\n\n### Per-Response Cost Estimation\n\nAfter each LLM call, cost is calculated as:\n\n```\ncost = (input_tokens / 1,000,000) * input_rate + (output_tokens / 1,000,000) * output_rate\n```\n\nThe `MeteringEngine` first checks the **model catalog** for exact pricing. If the model is not found, it falls back to a pattern-matching heuristic.\n\n### Cost Rates (per million tokens)\n\n| Model Pattern | Input $/M | Output $/M |\n|--------------|-----------|------------|\n| `*haiku*` | $0.25 | $1.25 |\n| `*sonnet*` | $3.00 | $15.00 |\n| `*opus*` | $15.00 | $75.00 |\n| `gpt-4o-mini` | $0.15 | $0.60 |\n| `gpt-4o` | $2.50 | $10.00 |\n| `gpt-4.1-nano` | $0.10 | $0.40 |\n| `gpt-4.1-mini` | $0.40 | $1.60 |\n| `gpt-4.1` | $2.00 | $8.00 |\n| `o3-mini` | $1.10 | $4.40 |\n| `gemini-2.5-pro` | $1.25 | $10.00 |\n| `gemini-2.5-flash` | $0.15 | $0.60 |\n| `gemini-2.0-flash` | $0.10 | $0.40 |\n| `deepseek-reasoner` / `deepseek-r1` | $0.55 | $2.19 |\n| `*deepseek*` | $0.27 | $1.10 |\n| `*cerebras*` | $0.06 | $0.06 |\n| `*sambanova*` | $0.06 | $0.06 |\n| `*replicate*` | $0.40 | $0.40 |\n| `*llama*` / `*mixtral*` | $0.05 | $0.10 |\n| `*qwen*` | $0.20 | $0.60 |\n| `mistral-large*` | $2.00 | $6.00 |\n| `*mistral*` (other) | $0.10 | $0.30 |\n| `command-r-plus` | $2.50 | $10.00 |\n| `command-r` | $0.15 | $0.60 |\n| `sonar-pro` | $3.00 | $15.00 |\n| `*sonar*` (other) | $1.00 | $5.00 |\n| `grok-2-mini` / `grok-mini` | $0.30 | $0.50 |\n| `*grok*` (other) | $2.00 | $10.00 |\n| `*jamba*` | $2.00 | $8.00 |\n| Default (unknown) | $1.00 | $3.00 |\n\n### Quota Enforcement\n\nQuotas are checked on every LLM call. If the agent exceeds its hourly limit, the call is rejected with a `QuotaExceeded` error.\n\n```toml\n# Per-agent quota in config.toml\n[[agents]]\nname = \"chatbot\"\n[agents.resources]\nmax_cost_per_hour_usd = 5.00 # cap at $5/hour\n```\n\nThe usage footer (when enabled) appends cost information to each response:\n\n```\n> Cost: $0.0042 | Tokens: 1,200 in / 340 out | Model: claude-sonnet-4-20250514\n```\n\n---\n\n## Fallback Providers\n\nThe `FallbackDriver` wraps multiple LLM drivers in a chain. If the primary driver fails, the next driver in the chain is tried automatically.\n\n### Behavior\n\n- On success: returns immediately\n- On **rate limit / overload** errors (`429`, `529`): bubbles up for retry logic (does NOT failover, because the primary should be retried after backoff)\n- On **all other errors**: logs a warning and tries the next driver in the chain\n- If all drivers fail: returns the last error\n\n### Configuration\n\nFallback chains are configured in your agent manifest or `config.toml`. The `FallbackDriver` is used automatically when an agent is in **Stabilisation mode** (`KernelMode::Stable`) or when multiple providers are configured for reliability.\n\n```toml\n# Example: primary Anthropic, fallback to Gemini, then Groq\n[[agents]]\nname = \"production-bot\"\nmodel = \"claude-sonnet-4-20250514\"\nfallback_models = [\"gemini-2.5-flash\", \"llama-3.3-70b-versatile\"]\n```\n\nThe fallback driver creates a chain: `AnthropicDriver -> GeminiDriver -> OpenAIDriver(Groq)`.\n\n---\n\n## API Endpoints\n\n### List All Models\n\n```\nGET /api/models\n```\n\nReturns the complete model catalog with metadata, pricing, and feature flags.\n\n**Response:**\n```json\n[\n {\n \"id\": \"claude-sonnet-4-20250514\",\n \"display_name\": \"Claude Sonnet 4\",\n \"provider\": \"anthropic\",\n \"tier\": \"Smart\",\n \"context_window\": 200000,\n \"max_output_tokens\": 64000,\n \"input_cost_per_m\": 3.0,\n \"output_cost_per_m\": 15.0,\n \"supports_tools\": true,\n \"supports_vision\": true,\n \"supports_streaming\": true,\n \"aliases\": [\"sonnet\", \"claude-sonnet\"]\n }\n]\n```\n\n### Get Specific Model\n\n```\nGET /api/models/{id}\n```\n\nReturns a single model entry. Supports both canonical IDs and aliases.\n\n```\nGET /api/models/sonnet\nGET /api/models/claude-sonnet-4-20250514\n```\n\n### List Aliases\n\n```\nGET /api/models/aliases\n```\n\nReturns a map of all alias-to-canonical-ID mappings.\n\n**Response:**\n```json\n{\n \"sonnet\": \"claude-sonnet-4-20250514\",\n \"haiku\": \"claude-haiku-4-5-20251001\",\n \"flash\": \"gemini-2.5-flash\",\n \"grok\": \"grok-2\"\n}\n```\n\n### List Providers\n\n```\nGET /api/providers\n```\n\nReturns all 20 providers with auth status and model counts.\n\n**Response:**\n```json\n[\n {\n \"id\": \"anthropic\",\n \"display_name\": \"Anthropic\",\n \"api_key_env\": \"ANTHROPIC_API_KEY\",\n \"base_url\": \"https://api.anthropic.com\",\n \"key_required\": true,\n \"auth_status\": \"Configured\",\n \"model_count\": 3\n },\n {\n \"id\": \"ollama\",\n \"display_name\": \"Ollama\",\n \"api_key_env\": \"OLLAMA_API_KEY\",\n \"base_url\": \"http://localhost:11434/v1\",\n \"key_required\": false,\n \"auth_status\": \"NotRequired\",\n \"model_count\": 5\n }\n]\n```\n\nAuth status values: `Configured`, `Missing`, `NotRequired`.\n\n### Set Provider API Key\n\n```\nPOST /api/providers/{name}/key\nContent-Type: application/json\n\n{ \"api_key\": \"sk-...\" }\n```\n\nConfigures an API key for a provider at runtime (stored as a `Zeroizing<String>`, wiped from memory on drop).\n\n### Remove Provider API Key\n\n```\nDELETE /api/providers/{name}/key\n```\n\nRemoves the configured API key for a provider.\n\n### Test Provider Connection\n\n```\nPOST /api/providers/{name}/test\n```\n\nSends a minimal test request to verify the provider is reachable and the API key is valid.\n\n---\n\n## Channel Commands\n\nTwo chat commands are available in any channel for inspecting models and providers:\n\n### `/models`\n\nLists all available models with their tier, provider, and context window. Only shows models from providers that have authentication configured (or do not require it).\n\n```\n/models\n```\n\nExample output:\n```\nAvailable models (12):\n\nFrontier:\n claude-opus-4-20250514 (Anthropic) — 200K ctx\n gemini-2.5-pro (Google Gemini) — 1M ctx\n\nSmart:\n claude-sonnet-4-20250514 (Anthropic) — 200K ctx\n gemini-2.5-flash (Google Gemini) — 1M ctx\n deepseek-chat (DeepSeek) — 64K ctx\n\nBalanced:\n llama-3.3-70b-versatile (Groq) — 128K ctx\n\nFast:\n claude-haiku-4-5-20251001 (Anthropic) — 200K ctx\n gemini-2.0-flash (Google Gemini) — 1M ctx\n\nLocal:\n llama3.2 (Ollama) — 128K ctx\n```\n\n### `/providers`\n\nLists all 20 providers with their authentication status.\n\n```\n/providers\n```\n\nExample output:\n```\nLLM Providers (20):\n\n Anthropic ANTHROPIC_API_KEY Configured 3 models\n OpenAI OPENAI_API_KEY Missing 6 models\n Google Gemini GEMINI_API_KEY Configured 3 models\n DeepSeek DEEPSEEK_API_KEY Missing 2 models\n Groq GROQ_API_KEY Configured 4 models\n Ollama (no key needed) Ready 3 models\n vLLM (no key needed) Ready 1 model\n LM Studio (no key needed) Ready 1 model\n ...\n```\n\n---\n\n## Environment Variables Summary\n\nQuick reference for all provider environment variables:\n\n| Provider | Env Var | Required |\n|----------|---------|----------|\n| Anthropic | `ANTHROPIC_API_KEY` | Yes |\n| OpenAI | `OPENAI_API_KEY` | Yes |\n| Google Gemini | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | Yes |\n| DeepSeek | `DEEPSEEK_API_KEY` | Yes |\n| Groq | `GROQ_API_KEY` | Yes |\n| OpenRouter | `OPENROUTER_API_KEY` | Yes |\n| Mistral AI | `MISTRAL_API_KEY` | Yes |\n| Together AI | `TOGETHER_API_KEY` | Yes |\n| Fireworks AI | `FIREWORKS_API_KEY` | Yes |\n| Ollama | `OLLAMA_API_KEY` | No |\n| vLLM | `VLLM_API_KEY` | No |\n| LM Studio | `LMSTUDIO_API_KEY` | No |\n| Perplexity AI | `PERPLEXITY_API_KEY` | Yes |\n| Cohere | `COHERE_API_KEY` | Yes |\n| AI21 Labs | `AI21_API_KEY` | Yes |\n| Cerebras | `CEREBRAS_API_KEY` | Yes |\n| SambaNova | `SAMBANOVA_API_KEY` | Yes |\n| Hugging Face | `HF_API_KEY` | Yes |\n| xAI | `XAI_API_KEY` | Yes |\n| Replicate | `REPLICATE_API_TOKEN` | Yes |\n\n---\n\n## Security Notes\n\n- All API keys are stored as `Zeroizing<String>` -- the key material is automatically overwritten with zeros when the value is dropped from memory.\n- Auth detection (`detect_auth()`) only checks `std::env::var()` for presence -- it never reads or logs the actual secret value.\n- Provider API keys set via the REST API (`POST /api/providers/{name}/key`) follow the same zeroization policy.\n- The health endpoint (`/api/health`) never exposes provider auth status or API keys. Detailed info is behind `/api/health/detail` which requires authentication.\n- All `DriverConfig` and `KernelConfig` structs implement `Debug` with secret redaction -- API keys are printed as `\"***\"` in logs.\n"}}}} |