1 line
13 KiB
JSON
1 line
13 KiB
JSON
{"repo": "amap-ml/gpg", "files": {"VisualThinker-R1-Zero/README.md": {"858eafad2b2d79dfde58d77205bd2b4d923baa00": {"t": 1785747256.048964, "n": 6270, "h": "b1684faf5a3eaaaf", "c": "# VisualThinker-R1-Zero: First ever R1-Zero's Aha Moment on just a 2B non-SFT Model\n[](https://turningpointai.notion.site/the-multimodal-aha-moment-on-2b-model)\n\n \n \n \n\n \n\nVisualThinker-R1-Zero is a replication of [DeepSeek-R1-Zero](https://arxiv.org/abs/2501.12948) in visual reasoning. We are **the first** to successfully observe **the emergent “aha moment”** and **increased response length** in **visual reasoning** on just a **2B non-SFT models**.\n\nFor more details, please refer to the notion [report](https://turningpointai.notion.site/the-multimodal-aha-moment-on-2b-model).\n\n<div align=\"center\">\n<img src=\"https://multimodal-r1.s3.us-west-1.amazonaws.com/Training_Steps.png\" width=\"700\" alt=\"visualthinking-intro-figure_00\">\n</div>\n\n> Training dynamics of our VisualThinker-R1-Zero training starting from the Qwen-VL-2B, without SFT or reward models. An aha moment and increasing response length is ever observed at a multimodal model.\n\n## 🔮 Highlights\n1. We are the **first to successfully produce the emergent “aha moment” and increased response length** for multimodal reasoning on just a **non-SFT 2B model**.\n2. We showed that **vision-centric** tasks could also benefit from improved reasoning capabilities. \n\nSimilar to DeepSeek R1, self reflection behavior is also observed during our RL training on vision-centric reasoning tasks. The model exhibits an emergent ability to rethink and correct its mistakes:\n\n```\n. . .\nTherefore, dark brown wooden bed with white blanket is not above the doorway.\nBut wait! I can think of something else.\nMaybe it's just higher than above the doorway, but slightly lower than above the doorway.\n. . .\n```\n\n## 📢 Updates\n- 2025-03-16: 🤗We released the model [checkpoint](https://huggingface.co/turningpoint-ai/VisualThinker-R1-Zero) at huggingface!\n- 2025-02-26: 🔥We share our main findings in this [notion blog](https://turningpointai.notion.site/the-multimodal-aha-moment-on-2b-model).\n- 2025-02-26: 🔥We release the VisualThinker R1 Zero repo.\n\n## 💻 Hardware Requirements\n\n\\* *estimated*\n\n| Method | Bits | 2B |\n| ------------------------ | ---- | ------ |\n| GRPO Full Fine-Tuning | AMP | 4*80GB |\n\n## 🧱 Setup\n\n```bash\nbash setup.sh\n```\n## 🤗 Prepare Dataset\n\n```bash\ncd src/data/SAT\nbash prepare_dataset.sh\n```\n\n## 🏋️ Training\n\n### GRPO Training\nTo reproduce the multimodal aha moment, run the following code to train the non-SFT model with GRPO on SAT:\n```bash\ncd src/open-r1-multimodal\nbash run_grpo_SAT.sh # Adjust open-r1-multimodal/configs/zero3.yaml or zero2.yaml accordingly\n```\n\n### SFT Training\nTo obtain SFT model for comparison, run the following code to train the non-SFT model on SAT:\n```bash\ncd src/open-r1-multimodal\nbash run_sft.sh # Adjust open-r1-multimodal/configs/zero3.yaml or zero2.yaml accordingly\n```\n\n## 📈 Evaluation\n\n### CVBench Evaluation\nWe provide following commands to reproduce our evaluation results on the CVBench. First change to evaluation directory:\n```bash\ncd src/eval \n```\n\nTo evaluate Base + GRPO (VisualThinker R1 Zero) model:\n```bash\npython evaluate_Qwen2_VL_CVBench-base.py --model_path <path_to_your_model> \\\n --bs 8 \\\n --use_reasoning_prompt\n```\nTo evaluate Base model:\n```bash\npython evaluate_Qwen2_VL_CVBench-base.py --model_path <path_to_your_model> \\\n --bs 8 \\\n --no-use_reasoning_prompt\n```\nTo evaluate Instruct + GRPO model:\n```bash\npython evaluate_Qwen2_VL_CVBench.py --model_path <path_to_your_model> \\\n --bs 8 \\\n --use_reasoning_prompt\n```\nTo evaluate Instruct model:\n```bash\npython evaluate_Qwen2_VL_CVBench.py --model_path <path_to_your_model> \\\n --bs 8 \\\n --no-use_reasoning_prompt\n```\n## 🔍 Resources\n\n**Full experiment log:** Upcoming\n\n**Models CKPT:** [🤗VisualThinker-R1-Zero](https://huggingface.co/turningpoint-ai/VisualThinker-R1-Zero) at huggingface\n\n## :coffee: Stay Connected!\n\nWe are always open to engaging discussions, collaborations, or even just sharing a virtual coffee. To get in touch or join our team, visit [TurningPoint AI](https://www.turningpoint-ai.com/)'s homepage for contact information.\n\n## 📖 Acknowledgements\n\nWe sincerely thank [DeepSeek](https://github.com/deepseek-ai/DeepSeek-R1), [Open-R1](https://github.com/huggingface/open-r1), [QwenVL](https://github.com/QwenLM/Qwen2.5-VL), [Open-R1-Multimodal](https://github.com/EvolvingLMMs-Lab/open-r1-multimodal), [R1-V](https://github.com/Deep-Agent/R1-V), [SAT](https://arxiv.org/abs/2412.07755), and [CV-Bench](https://cambrian-mllm.github.io/) for providing open source resources that laid the foundation of our project. \n\n## 🤝 Contributors\n\nHere are the key contributors from [TurningPoint AI](https://www.turningpoint-ai.com/) to this project:\n\n[Hengguang Zhou](https://hengguangzhou.github.io/)<sup>1</sup><sup>* </sup>, [Xirui Li](https://xirui-li.github.io/)<sup>1</sup><sup>* </sup>, [Ruochen Wang](https://ruocwang.github.io/)<sup>1</sup><sup>† </sup>, [Minhao Cheng](https://cmhcbb.github.io/)<sup>2</sup>, [Tianyi Zhou](https://tianyizhou.github.io/)<sup>3</sup> and [Cho-Jui Hsieh](https://web.cs.ucla.edu/~chohsieh/)<sup>1</sup><sup>4</sup>\n\n<sup>*</sup> Project Leads, <sup>†</sup> Main Advisor\n<sup>1</sup>University of California, Los Angeles, <sup>2</sup>Penn State University, <sup>3</sup>University of Maryland and <sup>4</sup>Google Research\n\n\n## :white_check_mark: Cite\n\nIf you find our work useful for your projects, please kindly cite the following BibTeX:\n\n```latex\n@misc{zhou2025r1zerosahamomentvisual,\n title={R1-Zero's \"Aha Moment\" in Visual Reasoning on a 2B Non-SFT Model}, \n author={Hengguang Zhou and Xirui Li and Ruochen Wang and Minhao Cheng and Tianyi Zhou and Cho-Jui Hsieh},\n year={2025},\n eprint={2503.05132},\n archivePrefix={arXiv},\n primaryClass={cs.AI},\n url={https://arxiv.org/abs/2503.05132}, \n}\n```\n\n"}, "a888899d34b940832b25cf2315b2f93d9d1fc6f4": {"t": 1785771869.7218263, "n": 6270, "h": "b1684faf5a3eaaaf", "c": "# VisualThinker-R1-Zero: First ever R1-Zero's Aha Moment on just a 2B non-SFT Model\n[](https://turningpointai.notion.site/the-multimodal-aha-moment-on-2b-model)\n\n \n \n \n\n \n\nVisualThinker-R1-Zero is a replication of [DeepSeek-R1-Zero](https://arxiv.org/abs/2501.12948) in visual reasoning. We are **the first** to successfully observe **the emergent “aha moment”** and **increased response length** in **visual reasoning** on just a **2B non-SFT models**.\n\nFor more details, please refer to the notion [report](https://turningpointai.notion.site/the-multimodal-aha-moment-on-2b-model).\n\n<div align=\"center\">\n<img src=\"https://multimodal-r1.s3.us-west-1.amazonaws.com/Training_Steps.png\" width=\"700\" alt=\"visualthinking-intro-figure_00\">\n</div>\n\n> Training dynamics of our VisualThinker-R1-Zero training starting from the Qwen-VL-2B, without SFT or reward models. An aha moment and increasing response length is ever observed at a multimodal model.\n\n## 🔮 Highlights\n1. We are the **first to successfully produce the emergent “aha moment” and increased response length** for multimodal reasoning on just a **non-SFT 2B model**.\n2. We showed that **vision-centric** tasks could also benefit from improved reasoning capabilities. \n\nSimilar to DeepSeek R1, self reflection behavior is also observed during our RL training on vision-centric reasoning tasks. The model exhibits an emergent ability to rethink and correct its mistakes:\n\n```\n. . .\nTherefore, dark brown wooden bed with white blanket is not above the doorway.\nBut wait! I can think of something else.\nMaybe it's just higher than above the doorway, but slightly lower than above the doorway.\n. . .\n```\n\n## 📢 Updates\n- 2025-03-16: 🤗We released the model [checkpoint](https://huggingface.co/turningpoint-ai/VisualThinker-R1-Zero) at huggingface!\n- 2025-02-26: 🔥We share our main findings in this [notion blog](https://turningpointai.notion.site/the-multimodal-aha-moment-on-2b-model).\n- 2025-02-26: 🔥We release the VisualThinker R1 Zero repo.\n\n## 💻 Hardware Requirements\n\n\\* *estimated*\n\n| Method | Bits | 2B |\n| ------------------------ | ---- | ------ |\n| GRPO Full Fine-Tuning | AMP | 4*80GB |\n\n## 🧱 Setup\n\n```bash\nbash setup.sh\n```\n## 🤗 Prepare Dataset\n\n```bash\ncd src/data/SAT\nbash prepare_dataset.sh\n```\n\n## 🏋️ Training\n\n### GRPO Training\nTo reproduce the multimodal aha moment, run the following code to train the non-SFT model with GRPO on SAT:\n```bash\ncd src/open-r1-multimodal\nbash run_grpo_SAT.sh # Adjust open-r1-multimodal/configs/zero3.yaml or zero2.yaml accordingly\n```\n\n### SFT Training\nTo obtain SFT model for comparison, run the following code to train the non-SFT model on SAT:\n```bash\ncd src/open-r1-multimodal\nbash run_sft.sh # Adjust open-r1-multimodal/configs/zero3.yaml or zero2.yaml accordingly\n```\n\n## 📈 Evaluation\n\n### CVBench Evaluation\nWe provide following commands to reproduce our evaluation results on the CVBench. First change to evaluation directory:\n```bash\ncd src/eval \n```\n\nTo evaluate Base + GRPO (VisualThinker R1 Zero) model:\n```bash\npython evaluate_Qwen2_VL_CVBench-base.py --model_path <path_to_your_model> \\\n --bs 8 \\\n --use_reasoning_prompt\n```\nTo evaluate Base model:\n```bash\npython evaluate_Qwen2_VL_CVBench-base.py --model_path <path_to_your_model> \\\n --bs 8 \\\n --no-use_reasoning_prompt\n```\nTo evaluate Instruct + GRPO model:\n```bash\npython evaluate_Qwen2_VL_CVBench.py --model_path <path_to_your_model> \\\n --bs 8 \\\n --use_reasoning_prompt\n```\nTo evaluate Instruct model:\n```bash\npython evaluate_Qwen2_VL_CVBench.py --model_path <path_to_your_model> \\\n --bs 8 \\\n --no-use_reasoning_prompt\n```\n## 🔍 Resources\n\n**Full experiment log:** Upcoming\n\n**Models CKPT:** [🤗VisualThinker-R1-Zero](https://huggingface.co/turningpoint-ai/VisualThinker-R1-Zero) at huggingface\n\n## :coffee: Stay Connected!\n\nWe are always open to engaging discussions, collaborations, or even just sharing a virtual coffee. To get in touch or join our team, visit [TurningPoint AI](https://www.turningpoint-ai.com/)'s homepage for contact information.\n\n## 📖 Acknowledgements\n\nWe sincerely thank [DeepSeek](https://github.com/deepseek-ai/DeepSeek-R1), [Open-R1](https://github.com/huggingface/open-r1), [QwenVL](https://github.com/QwenLM/Qwen2.5-VL), [Open-R1-Multimodal](https://github.com/EvolvingLMMs-Lab/open-r1-multimodal), [R1-V](https://github.com/Deep-Agent/R1-V), [SAT](https://arxiv.org/abs/2412.07755), and [CV-Bench](https://cambrian-mllm.github.io/) for providing open source resources that laid the foundation of our project. \n\n## 🤝 Contributors\n\nHere are the key contributors from [TurningPoint AI](https://www.turningpoint-ai.com/) to this project:\n\n[Hengguang Zhou](https://hengguangzhou.github.io/)<sup>1</sup><sup>* </sup>, [Xirui Li](https://xirui-li.github.io/)<sup>1</sup><sup>* </sup>, [Ruochen Wang](https://ruocwang.github.io/)<sup>1</sup><sup>† </sup>, [Minhao Cheng](https://cmhcbb.github.io/)<sup>2</sup>, [Tianyi Zhou](https://tianyizhou.github.io/)<sup>3</sup> and [Cho-Jui Hsieh](https://web.cs.ucla.edu/~chohsieh/)<sup>1</sup><sup>4</sup>\n\n<sup>*</sup> Project Leads, <sup>†</sup> Main Advisor\n<sup>1</sup>University of California, Los Angeles, <sup>2</sup>Penn State University, <sup>3</sup>University of Maryland and <sup>4</sup>Google Research\n\n\n## :white_check_mark: Cite\n\nIf you find our work useful for your projects, please kindly cite the following BibTeX:\n\n```latex\n@misc{zhou2025r1zerosahamomentvisual,\n title={R1-Zero's \"Aha Moment\" in Visual Reasoning on a 2B Non-SFT Model}, \n author={Hengguang Zhou and Xirui Li and Ruochen Wang and Minhao Cheng and Tianyi Zhou and Cho-Jui Hsieh},\n year={2025},\n eprint={2503.05132},\n archivePrefix={arXiv},\n primaryClass={cs.AI},\n url={https://arxiv.org/abs/2503.05132}, \n}\n```\n\n"}}}} |