1 line
16 KiB
JSON
1 line
16 KiB
JSON
{"repo": "anonymous-1-dev/liveplan", "files": {"plan_refiner/README.md": {"eef25964fb27ca116d9cf4fbe81f43e4d0f4ac59": {"t": 1785746500.9785147, "n": 7651, "h": "0d16fc734cef6bf1", "c": "# Plan Refiner\n\nExternal LLM-based planning module for Agent-Planner. Integrates with `mini-swe-agent` and `plan_monitor` to critique and refine execution plans.\n\n## Overview\n\nThe plan_refiner provides external planning guidance by:\n1. Receiving issue descriptions and trajectory data from mini-swe-agent\n2. Getting state summaries from plan_monitor (phases, rules triggered, graph signals)\n3. Calling an LLM to critique the current approach\n4. Generating refined high-level plans for next steps\n\n## Architecture\n\n```\n┌─────────────────┐\n│ mini-swe-agent │\n│ (trajectory) │\n└────────┬────────┘\n │\n │ trajectory steps\n │ issue description\n ▼\n┌─────────────────┐ ┌──────────────┐\n│ plan_monitor │────────>│ plan_refiner │\n│ (rules engine) │ state │ (LLM-based) │\n└─────────────────┘ summary └──────────────┘\n │ │\n │ rules triggered │ analysis + new plan\n ▼ ▼\n (when conditions met, trigger refinement)\n```\n\n## Components\n\n### Core Module (`refiner.py`)\n- `PlanRefiner`: Main class for plan refinement\n- `RefinerConfig`: Configuration for refiner behavior\n- Integrates with monitor to build state summaries\n\n### Data Structures (`types.py`)\n- `RefinerInput`: Input to refiner (issue + trajectory + state)\n- `RefinerOutput`: Output from refiner (analysis + new plan)\n- `StateSummary`: Monitor state (phases, rules, graph info)\n- `TrajectoryStep`: Single step in trajectory\n- `RuleTrigger`: Rule trigger information\n\n### Formatters (`formatters.py`)\n- `TrajectoryFormatter`: Formats trajectory for LLM\n- `StateSummaryFormatter`: Formats monitor state for LLM\n- `PromptBuilder`: Builds prompts from templates\n\n### Models (`models.py`)\n- `MockModel`: Mock LLM for testing\n- `OpenAIModel`: OpenAI API interface\n- `AnthropicModel`: Anthropic API interface\n\n### Simulator (`simulator/refiner_simulator.py`)\n- Replays trajectory files through monitor + refiner\n- Triggers refinement when monitor rules are satisfied\n- Demonstrates full pipeline functionality\n\n## Usage\n\n### Basic Simulation\n\n```bash\n# Run on a trajectory file\npython plan_refiner/simulator/refiner_simulator.py path/to/trajectory.json\n\n# With verbose output\npython plan_refiner/simulator/refiner_simulator.py path/to/trajectory.json --verbose\n\n# Adjust minimum steps between refinements\npython plan_refiner/simulator/refiner_simulator.py path/to/trajectory.json --min-steps 10\n```\n\n### Programmatic Usage\n\n```python\nfrom plan_refiner import PlanRefiner, RefinerConfig, RefinerInput\nfrom plan_refiner.models import MockModel\nfrom plan_refiner.types import StateSummary, TrajectoryStep\n\n# Initialize refiner\nmodel = MockModel()\nconfig = RefinerConfig(\n template_path=\"plan_refiner/config/default_template.yaml\",\n max_trajectory_steps=None,\n enable_parsing=True\n)\nrefiner = PlanRefiner(model=model, config=config)\n\n# Prepare input\nrefiner_input = RefinerInput(\n issue_description=\"Bug description here...\",\n trajectory=[\n TrajectoryStep(\n step_index=0,\n thought=\"My reasoning...\",\n action=\"ls -la\",\n observation=\"file1.py file2.py\"\n ),\n # ... more steps\n ],\n state_summary=StateSummary(\n current_phase=\"P\",\n phase_history=[\"L_navigate\", \"L_navigate\", \"P\"],\n unique_phases=[\"L_navigate\", \"P\"],\n rule_triggers=[],\n step_count=3\n )\n)\n\n# Run refinement\noutput = refiner.refine_plan(refiner_input)\n\n# Use results\nprint(output.analysis)\nprint(output.new_plan)\n```\n\n### Integration with Monitor\n\n```python\nfrom plan_monitor.monitor import StatefulPhaseMonitor\nfrom plan_refiner.refiner import PlanRefiner\n\n# Initialize components\nmonitor = StatefulPhaseMonitor(enable_rules=True)\nrefiner = PlanRefiner(model=model, config=config)\n\n# Process trajectory\nfor event, thought, observation in trajectory:\n result = monitor.on_step(event, thought=thought, observation=observation)\n\n # Trigger refinement when rules fire\n if result and result.rule_matches:\n # Build state summary from monitor\n state_summary = PlanRefiner.build_state_summary_from_monitor(\n monitor,\n result.rule_matches\n )\n\n # Build refiner input\n refiner_input = RefinerInput(\n issue_description=issue,\n trajectory=trajectory_steps,\n state_summary=state_summary\n )\n\n # Get refined plan\n output = refiner.refine_plan(refiner_input)\n print(output.new_plan)\n```\n\n## Configuration\n\n### Configuration File (`config/default.yaml`)\n\nThe configuration file contains:\n\n1. **Model Configuration**:\n ```yaml\n model_name: \"openai/gpt-5-mini\" # Model name for OpenRouter\n model_class: \"openrouter\" # Model class to use\n model_kwargs:\n temperature: 0.7\n max_tokens: 2000\n # For reasoning models:\n # extra_body:\n # reasoning:\n # effort: \"medium\"\n ```\n\n2. **Prompt Templates**:\n - System prompt defining the planner's role\n - User prompt template with placeholders:\n - `{{ISSUE_DESCRIPTION}}`: Bug/issue description\n - `{{TRAJECTORY_SO_FAR}}`: Formatted trajectory steps\n - `{{STATE_SUMMARY}}`: Formatted state summary from monitor\n\n### Using Different Models\n\nTo use a different model, edit `plan_refiner/config/default.yaml`:\n\n```yaml\n# For GPT-4\nmodel_name: \"openai/gpt-4\"\nmodel_class: \"openrouter\"\n\n# For Claude Sonnet\nmodel_name: \"anthropic/claude-3-5-sonnet-20241022\"\nmodel_class: \"openrouter\"\n\n# For reasoning models\nmodel_name: \"deepseek/deepseek-reasoner\"\nmodel_class: \"openrouter\"\nmodel_kwargs:\n temperature: 0.7\n max_tokens: 2000\n extra_body:\n reasoning:\n effort: \"high\"\n```\n\n### Environment Variables\n\nSet your OpenRouter API key:\n```bash\nexport OPENROUTER_API_KEY=your_key_here\n```\n\n### Expected Output Format\n\nThe LLM should return:\n\n```xml\n<analysis>\n### 1. Inferred High-Level Plan So Far\n[What the agent has been doing]\n\n### 2. Evaluation of the Plan's Logic\n[Critique of the strategy]\n\n### 3. Review of Implementation and Final Code\n[Assessment of execution quality]\n</analysis>\n\n<new_plan>\n### Customized High-Level Plan\n1. [Phase=L_navigate] Step description...\n2. [Phase=P] Step description...\n3. [Phase=V_regression_test] Step description...\n...\n</new_plan>\n```\n\n## Testing\n\nThe plan_refiner includes a comprehensive simulator that validates:\n1. ✓ Issue description extraction from trajectory\n2. ✓ Monitor integration and rule triggering\n3. ✓ State summary generation\n4. ✓ Trajectory formatting for LLM\n5. ✓ Plan refiner LLM interface\n6. ✓ Analysis and plan output parsing\n\nRun tests with:\n\n```bash\n# Test on sample trajectories\npython plan_refiner/simulator/refiner_simulator.py mini-swe-agent/outputs/swebench/gpt-5-mini/django__django-10914/django__django-10914.traj.json\n\n# Test with verbose output\npython plan_refiner/simulator/refiner_simulator.py mini-swe-agent/outputs/swebench/devstral-small/samples/astropy__astropy-13033/astropy__astropy-13033.traj.json --verbose\n```\n\n## Design Principles\n\n1. **Clean separation**: Refiner is independent of agent implementation\n2. **Protocol-based**: Works with any model implementing `ModelProtocol`\n3. **Configurable**: Template-driven prompts, adjustable parameters\n4. **Observable**: Verbose mode for debugging\n5. **Testable**: Mock model for testing without API calls\n\n## Future Enhancements\n\n- Support for real LLM models (OpenAI, Anthropic)\n- Caching of refinements to avoid redundant calls\n- Multi-turn refinement dialogue\n- Integration with agent's planning loop\n- Metrics tracking (refinement quality, acceptance rate)\n"}, "bc10f5052fd22094d2e618a1ad101a061b3eb285": {"t": 1785764998.6479924, "n": 7651, "h": "0d16fc734cef6bf1", "c": "# Plan Refiner\n\nExternal LLM-based planning module for Agent-Planner. Integrates with `mini-swe-agent` and `plan_monitor` to critique and refine execution plans.\n\n## Overview\n\nThe plan_refiner provides external planning guidance by:\n1. Receiving issue descriptions and trajectory data from mini-swe-agent\n2. Getting state summaries from plan_monitor (phases, rules triggered, graph signals)\n3. Calling an LLM to critique the current approach\n4. Generating refined high-level plans for next steps\n\n## Architecture\n\n```\n┌─────────────────┐\n│ mini-swe-agent │\n│ (trajectory) │\n└────────┬────────┘\n │\n │ trajectory steps\n │ issue description\n ▼\n┌─────────────────┐ ┌──────────────┐\n│ plan_monitor │────────>│ plan_refiner │\n│ (rules engine) │ state │ (LLM-based) │\n└─────────────────┘ summary └──────────────┘\n │ │\n │ rules triggered │ analysis + new plan\n ▼ ▼\n (when conditions met, trigger refinement)\n```\n\n## Components\n\n### Core Module (`refiner.py`)\n- `PlanRefiner`: Main class for plan refinement\n- `RefinerConfig`: Configuration for refiner behavior\n- Integrates with monitor to build state summaries\n\n### Data Structures (`types.py`)\n- `RefinerInput`: Input to refiner (issue + trajectory + state)\n- `RefinerOutput`: Output from refiner (analysis + new plan)\n- `StateSummary`: Monitor state (phases, rules, graph info)\n- `TrajectoryStep`: Single step in trajectory\n- `RuleTrigger`: Rule trigger information\n\n### Formatters (`formatters.py`)\n- `TrajectoryFormatter`: Formats trajectory for LLM\n- `StateSummaryFormatter`: Formats monitor state for LLM\n- `PromptBuilder`: Builds prompts from templates\n\n### Models (`models.py`)\n- `MockModel`: Mock LLM for testing\n- `OpenAIModel`: OpenAI API interface\n- `AnthropicModel`: Anthropic API interface\n\n### Simulator (`simulator/refiner_simulator.py`)\n- Replays trajectory files through monitor + refiner\n- Triggers refinement when monitor rules are satisfied\n- Demonstrates full pipeline functionality\n\n## Usage\n\n### Basic Simulation\n\n```bash\n# Run on a trajectory file\npython plan_refiner/simulator/refiner_simulator.py path/to/trajectory.json\n\n# With verbose output\npython plan_refiner/simulator/refiner_simulator.py path/to/trajectory.json --verbose\n\n# Adjust minimum steps between refinements\npython plan_refiner/simulator/refiner_simulator.py path/to/trajectory.json --min-steps 10\n```\n\n### Programmatic Usage\n\n```python\nfrom plan_refiner import PlanRefiner, RefinerConfig, RefinerInput\nfrom plan_refiner.models import MockModel\nfrom plan_refiner.types import StateSummary, TrajectoryStep\n\n# Initialize refiner\nmodel = MockModel()\nconfig = RefinerConfig(\n template_path=\"plan_refiner/config/default_template.yaml\",\n max_trajectory_steps=None,\n enable_parsing=True\n)\nrefiner = PlanRefiner(model=model, config=config)\n\n# Prepare input\nrefiner_input = RefinerInput(\n issue_description=\"Bug description here...\",\n trajectory=[\n TrajectoryStep(\n step_index=0,\n thought=\"My reasoning...\",\n action=\"ls -la\",\n observation=\"file1.py file2.py\"\n ),\n # ... more steps\n ],\n state_summary=StateSummary(\n current_phase=\"P\",\n phase_history=[\"L_navigate\", \"L_navigate\", \"P\"],\n unique_phases=[\"L_navigate\", \"P\"],\n rule_triggers=[],\n step_count=3\n )\n)\n\n# Run refinement\noutput = refiner.refine_plan(refiner_input)\n\n# Use results\nprint(output.analysis)\nprint(output.new_plan)\n```\n\n### Integration with Monitor\n\n```python\nfrom plan_monitor.monitor import StatefulPhaseMonitor\nfrom plan_refiner.refiner import PlanRefiner\n\n# Initialize components\nmonitor = StatefulPhaseMonitor(enable_rules=True)\nrefiner = PlanRefiner(model=model, config=config)\n\n# Process trajectory\nfor event, thought, observation in trajectory:\n result = monitor.on_step(event, thought=thought, observation=observation)\n\n # Trigger refinement when rules fire\n if result and result.rule_matches:\n # Build state summary from monitor\n state_summary = PlanRefiner.build_state_summary_from_monitor(\n monitor,\n result.rule_matches\n )\n\n # Build refiner input\n refiner_input = RefinerInput(\n issue_description=issue,\n trajectory=trajectory_steps,\n state_summary=state_summary\n )\n\n # Get refined plan\n output = refiner.refine_plan(refiner_input)\n print(output.new_plan)\n```\n\n## Configuration\n\n### Configuration File (`config/default.yaml`)\n\nThe configuration file contains:\n\n1. **Model Configuration**:\n ```yaml\n model_name: \"openai/gpt-5-mini\" # Model name for OpenRouter\n model_class: \"openrouter\" # Model class to use\n model_kwargs:\n temperature: 0.7\n max_tokens: 2000\n # For reasoning models:\n # extra_body:\n # reasoning:\n # effort: \"medium\"\n ```\n\n2. **Prompt Templates**:\n - System prompt defining the planner's role\n - User prompt template with placeholders:\n - `{{ISSUE_DESCRIPTION}}`: Bug/issue description\n - `{{TRAJECTORY_SO_FAR}}`: Formatted trajectory steps\n - `{{STATE_SUMMARY}}`: Formatted state summary from monitor\n\n### Using Different Models\n\nTo use a different model, edit `plan_refiner/config/default.yaml`:\n\n```yaml\n# For GPT-4\nmodel_name: \"openai/gpt-4\"\nmodel_class: \"openrouter\"\n\n# For Claude Sonnet\nmodel_name: \"anthropic/claude-3-5-sonnet-20241022\"\nmodel_class: \"openrouter\"\n\n# For reasoning models\nmodel_name: \"deepseek/deepseek-reasoner\"\nmodel_class: \"openrouter\"\nmodel_kwargs:\n temperature: 0.7\n max_tokens: 2000\n extra_body:\n reasoning:\n effort: \"high\"\n```\n\n### Environment Variables\n\nSet your OpenRouter API key:\n```bash\nexport OPENROUTER_API_KEY=your_key_here\n```\n\n### Expected Output Format\n\nThe LLM should return:\n\n```xml\n<analysis>\n### 1. Inferred High-Level Plan So Far\n[What the agent has been doing]\n\n### 2. Evaluation of the Plan's Logic\n[Critique of the strategy]\n\n### 3. Review of Implementation and Final Code\n[Assessment of execution quality]\n</analysis>\n\n<new_plan>\n### Customized High-Level Plan\n1. [Phase=L_navigate] Step description...\n2. [Phase=P] Step description...\n3. [Phase=V_regression_test] Step description...\n...\n</new_plan>\n```\n\n## Testing\n\nThe plan_refiner includes a comprehensive simulator that validates:\n1. ✓ Issue description extraction from trajectory\n2. ✓ Monitor integration and rule triggering\n3. ✓ State summary generation\n4. ✓ Trajectory formatting for LLM\n5. ✓ Plan refiner LLM interface\n6. ✓ Analysis and plan output parsing\n\nRun tests with:\n\n```bash\n# Test on sample trajectories\npython plan_refiner/simulator/refiner_simulator.py mini-swe-agent/outputs/swebench/gpt-5-mini/django__django-10914/django__django-10914.traj.json\n\n# Test with verbose output\npython plan_refiner/simulator/refiner_simulator.py mini-swe-agent/outputs/swebench/devstral-small/samples/astropy__astropy-13033/astropy__astropy-13033.traj.json --verbose\n```\n\n## Design Principles\n\n1. **Clean separation**: Refiner is independent of agent implementation\n2. **Protocol-based**: Works with any model implementing `ModelProtocol`\n3. **Configurable**: Template-driven prompts, adjustable parameters\n4. **Observable**: Verbose mode for debugging\n5. **Testable**: Mock model for testing without API calls\n\n## Future Enhancements\n\n- Support for real LLM models (OpenAI, Anthropic)\n- Caching of refinements to avoid redundant calls\n- Multi-turn refinement dialogue\n- Integration with agent's planning loop\n- Metrics tracking (refinement quality, acceptance rate)\n"}}}} |