{"repo": "ai4co/reevo", "files": {"README.md": {"6dce18257da5e11db2d138e417a2fffc5c72d05f": {"t": 1785674104.5166345, "n": 9715, "h": "ae1f63a5a46d87e2", "c": "# [NeurIPS 2024] ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution\n\n<p align=\"center\">\n    <img src=\"./assets/reevo-logo.png\" alt=\"logo\" width=\"25%\">\n</p>\n\n\n\n🥳 **Welcome!** This is a codebase that accompanies the paper [*ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution*](https://arxiv.org/abs/2402.01145).\n\n**Give ReEvo 5 minutes, and get a state-of-the-art algorithm in return!**\n\n## Table of Contents\n\n* 1\\. [ News 📰](#News)\n* 2\\. [ Introduction 🚀](#Introduction)\n* 3\\. [ Exciting Highlights 🌟](#ExcitingHighlights)\n* 4\\. [ Usage 🔑](#Usage)\n  * 4.1. [Dependency](#Dependency)\n  * 4.2. [To run ReEvo](#TorunReEvo)\n  * 4.3. [Available problems](#Availableproblems)\n  * 4.4. [Simple steps to apply ReEvo to your problem](#SimplestepstoapplyReEvotoyourproblem)\n  * 4.5. [Use Alternative LLMs](#UseAlternativeLLMs)\n* 5\\. [ Citation 🤩](#Citation)\n* 6\\. [ Acknowledgments 🫡](#Acknowledgments)\n\n\n##  1. <a name='News'></a> News 📰\n\n- **Sep. 2024:** *ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution* has been accepted at NeurIPS 2024 🥳\n- **May 2024:** We release a new paper version\n- **Apr. 2024:** Novel use cases for Neural Combinatorial Optimization (NCO) and Electronic Design Automation (EDA)\n- **Feb. 2024:** We are excited to release *ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution* 🚀\n\n\n##  2. <a name='Introduction'></a> Introduction 🚀\n\n![Diagram of ReEvo](./assets/reevo.png)\n\nWe introduce **Language Hyper-Heuristics (LHHs)**, an emerging variant of Hyper-Heuristics (HHs) that leverages LLMs for heuristic generation, featuring **minimal manual intervention and open-ended heuristic spaces**.\n\nTo empower LHHs, we present **Reflective Evolution (ReEvo)**, a generic searching framework that emulates the reflective design approach of human experts while much surpassing human capabilities with its scalable LLM inference, Internet-scale domain knowledge, and powerful evolutionary search.\n\n\n##  3. <a name='ExcitingHighlights'></a> Exciting Highlights 🌟\n\nWe can improve the following types of algorithms:\n- Neural Combinatorial Optimization (NCO)\n- Genetic Algorithm (GA)\n- Ant Colony Optimization (ACO)\n- Guided Local Search (GLS)\n- Constructive Heuristics\n\non the following problems:\n- Traveling Salesman Problem (TSP)\n- Capacitated Vehicle Routing Problem (CVRP)\n- Orienteering Problem (OP)\n- Multiple Knapsack Problems (MKP)\n- Bin Packing Problem (BPP)\n- Decap Placement Problem (DPP)\n\nwith both black-box and white-box settings.\n\n##  4. <a name='Usage'></a> Usage 🔑\n\n- Set your LLM API key (OpenAI API, ZhiPu API, Llama API) as an environment variable or like this:\n    ```bash\n    $ python main.py llm_client=openai llm_client.api_key=\"<Your API key>\" # see more options in ./cfg/llm_client\n    ```\n- Running logs and intermediate results are saved in `./outputs/main/` by default.\n- Datasets are generated on the fly.\n- Some test notebooks are provided in `./problems/*/test.ipynb`.\n\n####  4.1. <a name='Dependency'></a>Installation\n\n\n> [!TIP] \n> We recommend using [uv](https://github.com/astral-sh/uv) for lightning fast installation and dependency management (see below details), otherwise read on!\n\n<details>\n<summary>See details</summary>\n\nWe recommend using [uv](https://github.com/astral-sh/uv) for faster installation and dependency management. To install it, run:\n\n```bash\ncurl -LsSf https://astral.sh/uv/install.sh | sh\n```\n\nThen, clone the repository and cd into it:\n```bash\ngit clone git@github.com:ai4co/reevo.git\ncd reevo\n```\n\nCreate a new virtual environment and activate it:\n```bash\nuv venv --python 3.12\nsource .venv/bin/activate\n```\n\nThen synchronize the dependencies:\n```bash\nuv sync --all-extras\n```\n\nThis will install all the optional required dependencies; you can remove `--all-extras` to install only the default dependencies and add e.g. `--extra aco --extra gls` to install only the ACO and GLS problem-specific ones respectively and similarly (see the [`pyproject.toml`](pyproject.toml) for more details).\n\n</details>\n\n\nAfter cloning the repo, you can install dependencies locally on Python>=3.11 as follows:\n\n```bash\npip install -e \".[gls,aco,nco]\"\n```\n\nwhere `gls`, `aco` and `nco` are the optional dependencies for GLS, ACO and NCO problems (remove them if not needed). \n\n\n\n\n\n\n####  4.2. <a name='TorunReEvo'></a>To run ReEvo\n```bash\n# e.g., for tsp_aco\npython main.py \\\n    problem=tsp_aco \\  # problem name\n    init_pop_size=4 \\  # initial population size\n    pop_size=4 \\  # population size\n    max_fe=20 \\  # maximum number of heuristic evaluations\n    timeout=20  # allowed evaluation time for one generation\n```\n\nCheck out `./cfg/` for more options.\n\n####  4.3. <a name='Availableproblems'></a>Available problems\n- Traveling Salesman Problem (TSP): `tsp_aco`, `tsp_aco_black_box`, `tsp_constructive`, `tsp_gls`, `tsp_pomo`, `tsp_lehd`\n- Capacitated Vehicle Routing Problem (CVRP): `cvrp_aco`, `cvrp_aco_black_box`, `cvrp_pomo`, `cvrp_lehd`\n- Bin Packing Problem (BPP): `bpp_offline_aco`, `bpp_offline_aco_black_box`, `bpp_online`\n- Multiple Knapsack Problems (MKP): `mkp_aco`, `mkp_aco_black_box`\n- Orienteering Problem (OP): `op_aco`, `op_aco_black_box`\n- Decap Placement Problem (DPP): `dpp_ga`\n\n####  4.4. <a name='SimplestepstoapplyReEvotoyourproblem'></a>Simple steps to apply ReEvo to your problem\n\n- Define your problem in `./cfg/problem/`.\n- Generate problem instances and implement the evaluation pipeline in `./problems/`.\n- Add function_description, function_signature, and seed_function in `./prompts/`.\n\nBy default:\n\n- The LLM-generated heuristic is written into `f\"./problems/YOUR_PROBLEM/gpt.py\"`, and will be imported into `./problems/YOUR_PROBLEM/eval.py` (e.g. for [TSP_ACO](https://github.com/ai4co/reevo/blob/e4e479f44b9fc4cd47af91faffc0977a58b635ab/problems/tsp_aco/eval.py#L10)), which is called by [`reevo._run_code`](https://github.com/ai4co/reevo/blob/e4e479f44b9fc4cd47af91faffc0977a58b635ab/reevo.py#L229) during ReEvo.\n- In training mode, `./problems/YOUR_PROBLEM/eval.py` (e.g. for [TSP_ACO](https://github.com/ai4co/reevo/blob/e4e479f44b9fc4cd47af91faffc0977a58b635ab/problems/tsp_aco/eval.py#L58)) should print out the meta-objective value as the last line of stdout, which is parsed by [`reevo.evaluate_population`](https://github.com/ai4co/reevo/blob/e4e479f44b9fc4cd47af91faffc0977a58b635ab/reevo.py#L206) for heuristic evaluation.\n\n#### 4.5. <a name='UseAlternativeLLMs'></a>Use Alternative LLMs\n\nUse the cli parameter `llm_client` to designate an LLM API provider, and `llm_client.model` to determine the model to use. For example,\n\n```bash\n$ export LLAMA_API_KEY=xxxxxxxxxxxxxxxxxxxx\n$ python main.py llm_client=llama_api llm_client.model=gemma2-9b\n```\n\nSupported LLM API providers and models include (note that only chat models are supported):\n- [**OpenAI**](https://openai.com/api/): gpt-3.5-turbo (default), gpt-4o, gpt-4o-mini, gpt-4-turbo, etc.\n- [**Zhipu AI**](https://open.bigmodel.cn/): GLM-3-Turbo, GLM-4-Air, GLM-4-0520, etc. ([full list](https://open.bigmodel.cn/dev/howuse/model))\n- [**DeepSeek**](https://www.deepseek.com/): deepseek-chat\n- [**Moonshot AI**](https://www.moonshot.cn/): moonshot-v1-8k/32k/128k\n- [**Llama API**](https://www.llama-api.com/): llama3.1-8b/70b/405b, gemma2-9b/27b, Qwen2-72B, etc. ([full list](https://docs.llama-api.com/quickstart#available-models))\n- And [more **providers**](https://docs.litellm.ai/docs/providers) supported via [**LiteLLM**](https://docs.litellm.ai/docs/).\n\n\n##  5. <a name='Citation'></a> Citation 🤩\n\nIf you encounter any difficulty using our code, please do not hesitate to submit an issue or directly contact us! \n\nWe are also on [Slack](https://join.slack.com/t/ai4co-community/shared_invite/zt-3jsdjs3ec-3KHdV3HwanL884mq_9tyYw) if you have any questions or would like to discuss ReEvo with us. We are open to collaborations and would love to hear from you 🚀\n\nIf you find our work helpful (or if you are so kind as to offer us some encouragement), please consider giving us a star, and citing our paper.\n```bibtex\n@inproceedings{ye2024reevo,\n    title={ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution}, \n    author={Ye, Haoran and Wang, Jiarui and Cao, Zhiguang and Berto, Federico and Hua, Chuanbo and Kim, Haeyeon and Park, Jinkyoo and Song, Guojie},\n    booktitle={Advances in Neural Information Processing Systems},\n    year={2024},\n    note={\\url{https://github.com/ai4co/reevo}}\n}\n```\n\n##  6. <a name='Acknowledgments'></a> Acknowledgments 🫡\nWe are very grateful to [Yuan Jiang](https://github.com/jiang-yuan), [Yining Ma](https://github.com/yining043), [Yifan Yang](https://scholar.google.com/citations?hl=en&user=dO8kmG4AAAAJ), and [AI4CO community](https://github.com/ai4co) for valuable discussions and feedback.\n\nAlso, our work is built upon the following projects, among others:\n- [DeepACO: Neural-enhanced Ant Systems for Combinatorial Optimization](https://github.com/henry-yeh/DeepACO)\n- [Eureka: Human-Level Reward Design via Coding Large Language Models](https://github.com/eureka-research/Eureka)\n- [Algorithm Evolution Using Large Language Model](https://arxiv.org/abs/2311.15249)\n- [Mathematical discoveries from program search with large language models](https://github.com/google-deepmind/funsearch)\n- [An Example of Evolutionary Computation + Large Language Model Beating Human: Design of Efficient Guided Local Search](https://arxiv.org/abs/2401.02051)\n- [Evolution of Heuristics: Towards Efficient Automatic Algorithm Design Using Large Language Model](https://openreview.net/forum?id=BwAkaxqiLB)\n- [DevFormer: A Symmetric Transformer for Context-Aware Device Placement](https://arxiv.org/abs/2205.13225)\n"}}}}