{"repo": "arvindcr4/agentic-grpo-bench-anon", "files": {"ai-scientist-template/README.md": {"dcdb62628587a4ed4ef5c9dbdaad9bc9701b23ce": {"t": 1785746452.083276, "n": 7520, "h": "c487c6cb44179f99", "c": "# AI Scientist Integration for GRPO Research\n\nThis directory contains a complete [Sakana AI Scientist](https://github.com/SakanaAI/AI-Scientist) template for your GRPO math reasoning research. AI Scientist will autonomously generate ideas, run experiments, write papers, and self-review them.\n\n## What AI Scientist Will Do\n\nGiven this template, AI Scientist will:\n\n1. **Generate ideas** — novel variations on GRPO training (reward shaping, curriculum, hyperparameters, etc.)\n2. **Modify `experiment.py`** — implement each idea as code changes\n3. **Run experiments** — train on GSM8K with 3 seeds, collect metrics\n4. **Write a paper** — full LaTeX paper with results, citations, and analysis\n5. **Self-review** — NeurIPS-style review with Accept/Reject decision\n6. **Optionally improve** — revise based on review feedback\n\nCost: ~$15 per generated paper (Claude Sonnet 4.6), ~$5 with DeepSeek. For maximum quality, use Claude Opus 4.6 (`claude-opus-4-6`) at ~$50/paper.\n\n## Setup\n\n### 1. Clone AI Scientist\n\n```bash\ncd ~\ngit clone https://github.com/SakanaAI/AI-Scientist.git\ncd AI-Scientist\nconda create -n ai-scientist python=3.11 -y\nconda activate ai-scientist\npip install -r requirements.txt\n```\n\n### 2. Install GPU dependencies\n\n```bash\n# For local GPU (A10/A100/T4)\npip install torch torchvision --index-url https://download.pytorch.org/whl/cu121\npip install transformers datasets trl peft accelerate bitsandbytes\n```\n\n### 3. Set API keys\n\n```bash\n# At least one LLM provider (Claude recommended for best results)\nexport ANTHROPIC_API_KEY=\"sk-ant-...\"\n\n# OpenAI required for the review stage (hardcoded in AI Scientist)\nexport OPENAI_API_KEY=\"sk-...\"\n\n# Optional: better literature search\nexport S2_API_KEY=\"...\"  # Semantic Scholar\n```\n\n### 4. Install LaTeX\n\n```bash\n# Ubuntu/Debian\nsudo apt-get install texlive-full\n\n# Or minimal (faster install)\nsudo apt-get install texlive-latex-base texlive-latex-extra texlive-bibtex-extra biber\n```\n\n### 5. Copy template into AI Scientist\n\n```bash\n# From tinker-rl-lab root\ncp -r ai-scientist-template ~/AI-Scientist/templates/grpo_gsm8k\n```\n\n### 6. Generate real baseline (recommended)\n\nThe provided `run_0/final_info.json` has placeholder values. For best results, run the actual baseline:\n\n```bash\ncd ~/AI-Scientist/templates/grpo_gsm8k\npython experiment.py --out_dir run_0\npython plot.py\n```\n\nThis takes ~30-90 min on a single GPU depending on hardware.\n\n### 7. Copy LaTeX style files\n\n```bash\n# Copy from an existing AI Scientist template\ncp ~/AI-Scientist/templates/nanoGPT/latex/iclr2024_conference.sty ~/AI-Scientist/templates/grpo_gsm8k/latex/\ncp ~/AI-Scientist/templates/nanoGPT/latex/iclr2024_conference.bst ~/AI-Scientist/templates/grpo_gsm8k/latex/\ncp ~/AI-Scientist/templates/nanoGPT/latex/fancyhdr.sty ~/AI-Scientist/templates/grpo_gsm8k/latex/\n```\n\n### 8. Launch\n\n```bash\ncd ~/AI-Scientist\n\n# Best quality: Claude Opus 4.6 (deepest reasoning)\npython launch_scientist.py \\\n    --model \"claude-opus-4-6\" \\\n    --experiment grpo_gsm8k \\\n    --num-ideas 2\n\n# Good balance: Claude Sonnet 4.6 (best coding model, cheaper)\npython launch_scientist.py \\\n    --model \"claude-sonnet-4-6\" \\\n    --experiment grpo_gsm8k \\\n    --num-ideas 4\n\n# Budget option: DeepSeek\npython launch_scientist.py \\\n    --model \"deepseek-chat\" \\\n    --experiment grpo_gsm8k \\\n    --num-ideas 5\n\n# With improvement loop (paper gets revised after review)\npython launch_scientist.py \\\n    --model \"claude-opus-4-6\" \\\n    --experiment grpo_gsm8k \\\n    --num-ideas 2 \\\n    --improvement\n```\n\n## What Ideas Will It Generate?\n\nBased on the seed ideas and task description, expect ideas like:\n\n- **Reward shaping**: partial credit for format compliance, step-level rewards\n- **Curriculum learning**: easy-first scheduling based on problem difficulty\n- **GRPO variants**: different group sizes, KL penalties, advantage normalization\n- **Prompt engineering**: chain-of-thought templates, few-shot examples in prompts\n- **Learning rate schedules**: warmup, cosine decay, cyclical LR\n- **Data augmentation**: rephrasing problems, augmenting with similar problems\n\n## How to Use Results for Your Thesis/Paper\n\n### Incorporating AI Scientist findings into your existing paper\n\n1. **Check `results/grpo_gsm8k/` for generated papers** — each idea gets its own folder with:\n   - `{idea_name}.pdf` — the generated paper\n   - `review.txt` — automated review with scores\n   - `experiment.py` — the modified code (shows exactly what was changed)\n   - `final_info.json` — quantitative results\n\n2. **Mine for ideas, not papers** — the generated papers are drafts. The value is in:\n   - Novel experiment configurations that improve your metric\n   - Ablation results you can cite or reproduce at scale on Tinker\n   - Literature connections you may have missed (via auto-citations)\n\n3. **Scale winners on Tinker** — when AI Scientist finds a configuration that improves accuracy:\n   ```bash\n   # Adapt the winning experiment.py changes back to your Tinker script\n   # e.g., if partial credit reward worked, add it to grpo_gsm8k_base.py\n   ```\n\n4. **Mandatory disclosure** — AI Scientist's license requires disclosing AI use in any resulting manuscript. Add a note like:\n   > \"Preliminary experiment configurations were explored using the AI Scientist framework (Lu et al., 2024). All results reported in this paper were independently reproduced and validated by the authors.\"\n\n### For the capstone thesis\n\nThe thesis can include a section on \"AI-assisted research methodology\" describing how AI Scientist was used for hyperparameter exploration and idea generation, with the human researchers validating and scaling the best findings.\n\n## Running on Cloud GPUs\n\n### vast.ai (cheapest)\n\n```bash\n# In your vast.ai instance\ngit clone https://github.com/SakanaAI/AI-Scientist.git\ncd AI-Scientist && pip install -r requirements.txt\npip install torch trl peft transformers datasets accelerate bitsandbytes\ncp -r /path/to/ai-scientist-template templates/grpo_gsm8k\npython launch_scientist.py --model \"deepseek-chat\" --experiment grpo_gsm8k --num-ideas 5\n```\n\n### Google Colab\n\nUse the template with a Colab notebook:\n```python\n!git clone https://github.com/SakanaAI/AI-Scientist.git\n%cd AI-Scientist\n!pip install -r requirements.txt\n!pip install trl peft accelerate bitsandbytes\n# Copy template files, set API keys, then launch\n```\n\n## Template File Reference\n\n| File | Purpose | Modify? |\n|------|---------|---------|\n| `experiment.py` | GRPO training script (AI Scientist modifies this) | Yes — AI Scientist's core target |\n| `plot.py` | Visualization (AI Scientist updates plots) | Yes — updated per experiment |\n| `prompt.json` | System prompt + domain description | Rarely — tune if ideas are off-target |\n| `seed_ideas.json` | Example ideas (few-shot) | Rarely — add domain-specific ideas |\n| `latex/template.tex` | Paper template with embedded references | No — AI Scientist fills in content |\n| `run_0/final_info.json` | Baseline results | Once — regenerate with real baseline |\n\n## Open-Ended Evolution Mode\n\nAI Scientist also supports an **open-ended evolution** mode where it builds on previous results continuously. This is ideal for a sustained research campaign:\n\n```bash\ncd ~/AI-Scientist\npython experimental/launch_oe_scientist.py \\\n    --model \"claude-sonnet-4-6\" \\\n    --experiment grpo_gsm8k \\\n    --num-ideas 10 \\\n    --num-rounds 5\n```\n\nThis generates ideas → runs experiments → uses results to generate _better_ ideas → repeat. After 5 rounds, you'll have a chain of progressively refined experiments.\n"}}}}