1 line
142 KiB
JSON
1 line
142 KiB
JSON
{"repo": "akatrainning/streamguard", "files": {"README.md": {"01686d2b6f176b2ca64517fc0842eb5890ddbd65": {"t": 1785674035.5014544, "n": 2484, "h": "aec59707e93cff53", "c": "# StreamGuard\n\nStreamGuard is a live-stream compliance monitoring demo. It contains a FastAPI\nbackend and a React/Vite frontend for monitoring live-stream scripts, chat\nsentiment, risk alerts, rule checks, history reports, and Douyin live-room\ndiscovery.\n\n## Project Layout\n\n```text\nstreamguard/\n streamguard-backend/ FastAPI backend, WebSocket streams, ASR/LLM analysis\n streamguard-web/ React + Vite frontend dashboard\n start-backend.ps1 Windows helper for the backend\n start-frontend.ps1 Windows helper for the frontend\n start-dev.ps1 Starts backend and frontend in two PowerShell windows\n```\n\nRemoved legacy AgentDojo research code, experiment outputs, notebooks, and old\ndocumentation so this repository only keeps StreamGuard application content.\n\n## Requirements\n\n- Python 3.10+\n- Node.js 18+\n- Chrome, if you use Douyin live-room discovery or live capture\n\n## Backend\n\n```powershell\ncd streamguard-backend\npip install -r requirements.txt\ncopy .env.example .env\npython -m uvicorn app:app --reload --host 0.0.0.0 --port 8011\n```\n\nThe backend listens on `http://localhost:8011`.\n\nOptional `.env` values:\n\n```env\nLLM_PROVIDER=deepseek\nDEEPSEEK_API_KEY=sk-...\nLLM_BASE_URL=https://api.deepseek.com\nLLM_MODEL=deepseek-v4-flash\n\nASR_OPENAI_API_KEY=sk-...\nLOCAL_WHISPER_MODEL=tiny\nWHISPER_CPU_THREADS=2\nWHISPER_BEAM_SIZE=1\nAUDIO_CAPTURE_WINDOW_SECS=5\nENABLE_LIVE_AUDIO_ASR=1\nDOUYIN_SHOW_BROWSER=0\n```\n\nWithout API keys, the app still supports local/mock workflows and rule-based\nfallback behavior where available.\n\n## Frontend\n\n```powershell\ncd streamguard-web\nnpm install\nnpm run dev\n```\n\nOpen the Vite URL printed in the terminal, usually `http://localhost:5173`.\nThe frontend proxies API and WebSocket calls to backend port `8011`.\n\n## Quick Start On Windows\n\nFrom the repository root:\n\n```powershell\n.\\start-backend.ps1\n.\\start-frontend.ps1\n```\n\nOr start both in separate PowerShell windows:\n\n```powershell\n.\\start-dev.ps1\n```\n\n## Useful Checks\n\n```powershell\ncd streamguard-web\nnpm run build\n\ncd ..\\streamguard-backend\npython -m py_compile app.py douyin_cdp.py douyin_search.py douyin_selenium.py douyin_ws_client.py\n```\n\n## RAG Knowledge Architecture\n\nThe RAG layer is organized as a layered knowledge base: real-time transcript hot memory, historical records, rule graph, and citation-ready evidence documents. See [docs/rag_knowledge_architecture.md](docs/rag_knowledge_architecture.md) for the module design, retrieval policy, and API surface.\n"}, "d2098e6c6e508de536cb0cd437260e9c9f277cf3": {"t": 1785763900.8972054, "n": 2484, "h": "aec59707e93cff53", "c": "# StreamGuard\n\nStreamGuard is a live-stream compliance monitoring demo. It contains a FastAPI\nbackend and a React/Vite frontend for monitoring live-stream scripts, chat\nsentiment, risk alerts, rule checks, history reports, and Douyin live-room\ndiscovery.\n\n## Project Layout\n\n```text\nstreamguard/\n streamguard-backend/ FastAPI backend, WebSocket streams, ASR/LLM analysis\n streamguard-web/ React + Vite frontend dashboard\n start-backend.ps1 Windows helper for the backend\n start-frontend.ps1 Windows helper for the frontend\n start-dev.ps1 Starts backend and frontend in two PowerShell windows\n```\n\nRemoved legacy AgentDojo research code, experiment outputs, notebooks, and old\ndocumentation so this repository only keeps StreamGuard application content.\n\n## Requirements\n\n- Python 3.10+\n- Node.js 18+\n- Chrome, if you use Douyin live-room discovery or live capture\n\n## Backend\n\n```powershell\ncd streamguard-backend\npip install -r requirements.txt\ncopy .env.example .env\npython -m uvicorn app:app --reload --host 0.0.0.0 --port 8011\n```\n\nThe backend listens on `http://localhost:8011`.\n\nOptional `.env` values:\n\n```env\nLLM_PROVIDER=deepseek\nDEEPSEEK_API_KEY=sk-...\nLLM_BASE_URL=https://api.deepseek.com\nLLM_MODEL=deepseek-v4-flash\n\nASR_OPENAI_API_KEY=sk-...\nLOCAL_WHISPER_MODEL=tiny\nWHISPER_CPU_THREADS=2\nWHISPER_BEAM_SIZE=1\nAUDIO_CAPTURE_WINDOW_SECS=5\nENABLE_LIVE_AUDIO_ASR=1\nDOUYIN_SHOW_BROWSER=0\n```\n\nWithout API keys, the app still supports local/mock workflows and rule-based\nfallback behavior where available.\n\n## Frontend\n\n```powershell\ncd streamguard-web\nnpm install\nnpm run dev\n```\n\nOpen the Vite URL printed in the terminal, usually `http://localhost:5173`.\nThe frontend proxies API and WebSocket calls to backend port `8011`.\n\n## Quick Start On Windows\n\nFrom the repository root:\n\n```powershell\n.\\start-backend.ps1\n.\\start-frontend.ps1\n```\n\nOr start both in separate PowerShell windows:\n\n```powershell\n.\\start-dev.ps1\n```\n\n## Useful Checks\n\n```powershell\ncd streamguard-web\nnpm run build\n\ncd ..\\streamguard-backend\npython -m py_compile app.py douyin_cdp.py douyin_search.py douyin_selenium.py douyin_ws_client.py\n```\n\n## RAG Knowledge Architecture\n\nThe RAG layer is organized as a layered knowledge base: real-time transcript hot memory, historical records, rule graph, and citation-ready evidence documents. See [docs/rag_knowledge_architecture.md](docs/rag_knowledge_architecture.md) for the module design, retrieval policy, and API surface.\n"}, "56700f17c4f036bc118269e2a11e3708e0cef933": {"t": 1785763903.1998281, "n": 2157, "h": "4f919ee6fa70e932", "c": "# StreamGuard\n\nStreamGuard is a live-stream compliance monitoring demo. It contains a FastAPI\nbackend and a React/Vite frontend for monitoring live-stream scripts, chat\nsentiment, risk alerts, rule checks, history reports, and Douyin live-room\ndiscovery.\n\n## Project Layout\n\n```text\nstreamguard/\n streamguard-backend/ FastAPI backend, WebSocket streams, ASR/LLM analysis\n streamguard-web/ React + Vite frontend dashboard\n start-backend.ps1 Windows helper for the backend\n start-frontend.ps1 Windows helper for the frontend\n start-dev.ps1 Starts backend and frontend in two PowerShell windows\n```\n\nRemoved legacy AgentDojo research code, experiment outputs, notebooks, and old\ndocumentation so this repository only keeps StreamGuard application content.\n\n## Requirements\n\n- Python 3.10+\n- Node.js 18+\n- Chrome, if you use Douyin live-room discovery or live capture\n\n## Backend\n\n```powershell\ncd streamguard-backend\npip install -r requirements.txt\ncopy .env.example .env\npython -m uvicorn app:app --reload --host 0.0.0.0 --port 8011\n```\n\nThe backend listens on `http://localhost:8011`.\n\nOptional `.env` values:\n\n```env\nLLM_PROVIDER=deepseek\nDEEPSEEK_API_KEY=sk-...\nLLM_BASE_URL=https://api.deepseek.com\nLLM_MODEL=deepseek-v4-flash\n\nASR_OPENAI_API_KEY=sk-...\nLOCAL_WHISPER_MODEL=tiny\nWHISPER_CPU_THREADS=2\nWHISPER_BEAM_SIZE=1\nAUDIO_CAPTURE_WINDOW_SECS=5\nENABLE_LIVE_AUDIO_ASR=1\nDOUYIN_SHOW_BROWSER=0\n```\n\nWithout API keys, the app still supports local/mock workflows and rule-based\nfallback behavior where available.\n\n## Frontend\n\n```powershell\ncd streamguard-web\nnpm install\nnpm run dev\n```\n\nOpen the Vite URL printed in the terminal, usually `http://localhost:5173`.\nThe frontend proxies API and WebSocket calls to backend port `8011`.\n\n## Quick Start On Windows\n\nFrom the repository root:\n\n```powershell\n.\\start-backend.ps1\n.\\start-frontend.ps1\n```\n\nOr start both in separate PowerShell windows:\n\n```powershell\n.\\start-dev.ps1\n```\n\n## Useful Checks\n\n```powershell\ncd streamguard-web\nnpm run build\n\ncd ..\\streamguard-backend\npython -m py_compile app.py douyin_cdp.py douyin_search.py douyin_selenium.py douyin_ws_client.py\n```\n"}, "2d4c27572ac42e9ca597f8364dbb66b16ba0a089": {"t": 1785763904.6921685, "n": 2109, "h": "bfc5430d9661ec1d", "c": "# StreamGuard\n\nStreamGuard is a live-stream compliance monitoring demo. It contains a FastAPI\nbackend and a React/Vite frontend for monitoring live-stream scripts, chat\nsentiment, risk alerts, rule checks, history reports, and Douyin live-room\ndiscovery.\n\n## Project Layout\n\n```text\nstreamguard/\n streamguard-backend/ FastAPI backend, WebSocket streams, ASR/LLM analysis\n streamguard-web/ React + Vite frontend dashboard\n start-backend.ps1 Windows helper for the backend\n start-frontend.ps1 Windows helper for the frontend\n start-dev.ps1 Starts backend and frontend in two PowerShell windows\n```\n\nRemoved legacy AgentDojo research code, experiment outputs, notebooks, and old\ndocumentation so this repository only keeps StreamGuard application content.\n\n## Requirements\n\n- Python 3.10+\n- Node.js 18+\n- Chrome, if you use Douyin live-room discovery or live capture\n\n## Backend\n\n```powershell\ncd streamguard-backend\npip install -r requirements.txt\ncopy .env.example .env\npython -m uvicorn app:app --reload --host 0.0.0.0 --port 8011\n```\n\nThe backend listens on `http://localhost:8011`.\n\nOptional `.env` values:\n\n```env\nLLM_PROVIDER=deepseek\nDEEPSEEK_API_KEY=sk-...\nLLM_BASE_URL=https://api.deepseek.com\nLLM_MODEL=deepseek-v4-flash\n\nASR_OPENAI_API_KEY=sk-...\nLOCAL_WHISPER_MODEL=base\nWHISPER_CPU_THREADS=2\nENABLE_LIVE_AUDIO_ASR=0\nDOUYIN_SHOW_BROWSER=0\n```\n\nWithout API keys, the app still supports local/mock workflows and rule-based\nfallback behavior where available.\n\n## Frontend\n\n```powershell\ncd streamguard-web\nnpm install\nnpm run dev\n```\n\nOpen the Vite URL printed in the terminal, usually `http://localhost:5173`.\nThe frontend proxies API and WebSocket calls to backend port `8011`.\n\n## Quick Start On Windows\n\nFrom the repository root:\n\n```powershell\n.\\start-backend.ps1\n.\\start-frontend.ps1\n```\n\nOr start both in separate PowerShell windows:\n\n```powershell\n.\\start-dev.ps1\n```\n\n## Useful Checks\n\n```powershell\ncd streamguard-web\nnpm run build\n\ncd ..\\streamguard-backend\npython -m py_compile app.py douyin_cdp.py douyin_search.py douyin_selenium.py douyin_ws_client.py\n```\n"}, "6c2cc5f59ca09e2660b14bd4de92dd6090b3c176": {"t": 1785763905.9768803, "n": 4776, "h": "6b3957536df342ec", "c": "# StreamGuard — 直播语义鉴别系统\n\n> 实时感知直播间话术风险、弹幕舆情与合规异常的 AI 监控平台\n\n---\n\n## 项目简介\n\nStreamGuard 是一套面向直播电商场景的实时合规监控系统。系统通过接入直播音频，将主播话术转写为文本后，联合大语言模型（LLM）与规则引擎进行语义合规分析；同时对弹幕舆情进行轻量级情感意图识别，最终在可视化仪表盘上展示风险评分、违规告警与趋势分析，帮助运营团队在直播进行中实时发现陷阱话术、虚假宣传等违规内容。\n\n---\n\n## 核心功能\n\n### 1. 实时话术分析\n- **音频自动转写**：优先调用 OpenAI Whisper API（云端），网络不可用时自动切换为本地 faster-whisper 模型（无需 API Key）\n- **LLM + 规则双引擎**：DeepSeek / OpenAI 大模型对话术进行语义合规评分，同时以规则引擎作为零延迟基线兜底\n- **三类话术分类**：事实陈述（fact）/ 夸大话术（hype）/ 陷阱话术（trap）\n- **四维子评分**：语义一致性、事实核验度、合规分、主观性指数\n- **润色与关键词提取**：LLM 自动将 ASR 原始转写整理为通顺句子并提取核心关键词\n\n### 2. 弹幕舆情分析\n- **零延迟规则识别**：纯关键词规则引擎，无需 LLM 调用，即时输出\n- **七类意图分类**：购买意向 / 提问咨询 / 客诉投诉 / 质疑话术 / 支持主播 / 广告刷屏 / 普通弹幕\n- **情感极性判断**：正向 / 负向 / 中性\n- **弹幕-话术关联性**：识别弹幕是否在支持或质疑当前主播话术\n- **刷屏过滤**：检测重复字符集刷屏行为\n\n### 3. 可视化仪表盘\n- **理性指数仪表盘（RationalityGauge）**：实时显示当前直播间整体话术合规水位\n- **风险雷达图（RiskRadar）**：多维度合规评分雷达可视化\n- **语义拓扑图（TopologyGraph）**：话术节点与弹幕关联网络图\n- **语义信息流（SemanticFeed）**：按时间线滚动展示话术与弹幕分析结果\n- **高危告警横幅（AlertBanner）**：检测到高风险内容时自动弹出告警\n\n### 4. 直播发现与对比\n- **抖音直播间搜索**：通过关键词搜索抖音在播直播间，展示主播信息、观众数、直播标题等\n- **AI 评分对比**：同时选取多个直播间，由 AI 进行综合评级与横向对比\n- **Cookie 鉴权**：支持扫码登录或粘贴 Cookie 方式实现账号授权\n- **直播流接入**：通过 Selenium + Chrome CDP 自动发现 m3u8/flv 直播媒体流，免手动配置\n\n### 5. 消费者顾问\n- **商品搜索**：输入关键词，搜索当前直播间相关商品\n- **个性化分析**：结合用户预算、核心需求与当前直播话术证据，由 AI 生成购买建议\n- **风险提示**：将直播话术中检测到的违规内容作为负面证据纳入购买建议报告\n\n### 6. 规则中心\n- **30 条合规规则**，覆盖四大法规领域：\n - 《广告法》7 条（极限词、虚假广告、医疗广告等）\n - 《消费者权益保护法》6 条（知情权、虚假承诺等）\n - 《电子商务法》6 条（强制搭售、平台责任等）\n - 直播行业规范 10 条（主播资质、数据造假、未成年人保护等）\n- 每条规则含：风险等级 / 违规次数统计 / 监测关键词 / 违规示例 / 详细条文说明\n- 支持按法规分类筛选与全文关键词搜索\n\n### 7. 历史归档与会话报告\n- **会话存储**：每次监控会话结束后自动生成并保存报告（基于 localStorage）\n- **报告内容**：话术总数、三类分布、合规评分、高危话术摘录、弹幕舆情统计\n- **历史列表**：支持重命名、查看详情、删除单条及清空全部\n- **可视化报告**：以柱状图展示历史会话合规分趋势\n\n### 8. 数据统计分析\n- 7 日话术类型趋势折线图\n- 话术类型分布饼图\n- 多维合规能力雷达图（价格透明度 / 话术压力值 / 描述真实度 / 时间紧迫感 / 证据充分度 / 合规得分）\n- KPI 卡片：总话术数 / 高危话术数 / 平均合规分 / 监测时长\n\n### 9. 指挥中心（Command Center）\n- WebSocket 连接状态实时监控（已连接 / 连接中 / 重试中 / 已断开）\n- 消息吞吐量统计（总消息数 / 每分钟消息率）\n- 实时日志滚动面板（终端风格，自动滚底）\n- 快速导出当前会话 JSON 快照\n\n---\n\n## 技术栈\n\n| 层级 | 技术 |\n|------|------|\n| 前端框架 | React 18 + Vite |\n| 动效库 | Framer Motion |\n| 图表库 | Recharts |\n| 后端框架 | FastAPI（Python 3.10+） |\n| 大语言模型 | DeepSeek（默认）/ OpenRouter / OpenAI |\n| 语音转写（云端）| OpenAI Whisper API |\n| 语音转写（本地）| faster-whisper（CPU int8 量化） |\n| 音频采集 | imageio-ffmpeg（内置 ffmpeg，无需手动安装） |\n| 浏览器自动化 | Selenium + webdriver-manager + Chrome CDP |\n| 环境配置 | python-dotenv |\n\n---\n\n## 快速启动\n\n### 环境要求\n- Python 3.10+\n- Node.js 18+\n- （可选）Conda 环境\n\n### 1. 后端启动\n\n```bash\ncd streamguard-backend\n\n# 安装依赖\npip install -r requirements.txt\n\n# 配置环境变量（复制示例文件后填写 API Key）\n# 编辑 .env，填入 DEEPSEEK_API_KEY 等\n\n# 启动后端服务（默认端口 8011）\nuvicorn app:app --host 0.0.0.0 --port 8011 --reload\n```\n\n或使用项目根目录的启动脚本：\n\n```powershell\n.\\start-backend.ps1\n```\n\n### 2. 前端启动\n\n```bash\ncd streamguard-web\nnpm install\nnpm run dev\n# 打开 http://localhost:5173\n```\n\n### 3. 环境变量配置\n\n在 `streamguard-backend/.env` 中配置以下变量：\n\n```env\n# 大语言模型（三选一，优先级：DeepSeek > OpenRouter > OpenAI）\nDEEPSEEK_API_KEY=sk-xxxxxxxx\nLLM_BASE_URL=https://api.deepseek.com/v1\nLLM_MODEL=deepseek-chat\n\n# 语音转写（可选，不填则自动使用本地 faster-whisper）\nASR_OPENAI_API_KEY=sk-xxxxxxxx\n\n# 本地 Whisper 模型大小（tiny/base/small/medium，默认 base）\nLOCAL_WHISPER_MODEL=base\n\n# ASR 性能调优（可选）\nWHISPER_CPU_THREADS=2\nWHISPER_BEAM_SIZE=1\nAUDIO_CAPTURE_IDLE=2\n```\n\n---\n\n## 数据源模式\n\n| 模式 | 说明 |\n|------|------|\n| 模拟数据 | 内置 15 条话术 + 20 条弹幕随机循环播放，无需任何账号或网络 |\n| 抖音直播间 | 输入直播间 ID，通过 Selenium 自动发现媒体流，采集真实音频进行转写分析 |\n\n---\n\n## 项目结构\n\n```\nstreamguard/\n├── streamguard-backend/ # FastAPI 后端\n│ ├── app.py # 主入口：API 路由、ASR、LLM 分析、WebSocket\n│ ├── douyin_search.py # 抖音直播间搜索与 Cookie 管理\n│ ├── requirements.txt # Python 依赖\n│ └── .env # 环境变量（需自行创建）\n│\n└── streamguard-web/ # React 前端\n └── src/\n ├── pages/\n │ ├── WelcomePage.jsx # 欢迎引导页\n │ ├── AnalyticsPage.jsx # 数据统计分析\n │ ├── HistoryPage.jsx # 历史会话归档\n │ ├── RulesPage.jsx # 合规规则中心（30 条）\n │ ├── ConsumerAdvisorPage.jsx # 消费者顾问\n │ └── LiveDiscoverPage.jsx # 抖音直播间发现\n └── components/\n ├── Header.jsx # 顶部导航与会话控制栏\n ├── CommandCenter.jsx # 指挥中心（连接状态与日志）\n ├── SemanticFeed.jsx # 话术/弹幕实时信息流\n ├── RationalityGauge.jsx # 理性指数仪表盘\n ├── RiskRadar.jsx # 风险雷达图\n ├── TopologyGraph.jsx # 语义拓扑图\n ├── AlertBanner.jsx # 高危告警横幅\n ├── LiveVideoPlayer.jsx # 直播视频播放器\n └── SessionReportModal.jsx # 会话报告弹窗\n```\n\n---\n\n## 合规覆盖范围\n\n| 法规 | 条数 | 典型违规场景 |\n|------|------|------------|\n| 广告法 | 7 条 | 极限词、虚假广告、医疗夸大、未成年人广告 |\n| 消费者权益保护法 | 6 条 | 知情权缺失、虚假承诺、强制搭售 |\n| 电子商务法 | 6 条 | 平台连带责任、价格欺诈、刷单造假 |\n| 直播行业规范 | 10 条 | 主播资质、数据造假、打赏诱导、内容低俗、未成年人保护 |\n\n---\n\n## License\n\n本项目仅供学习与研究使用。\n"}, "4826c4d6a64efcdea7ac2e4500ce1965419574ac": {"t": 1785763907.2972639, "n": 3186, "h": "6349ef886bc81da4", "c": "<center>\n\n# AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents\n\n \n   \n\n\n\n[Edoardo Debenedetti](https://edoardo.science)<sup>1</sup>, [Jie Zhang](https://zj-jayzhang.github.io)<sup>1</sup>, [Mislav Balunović](https://www.sri.inf.ethz.ch/people/mislav)<sup>1,2</sup>, [Luca Beurer-Kellner](https://www.sri.inf.ethz.ch/people/luca)<sup>1,2</sup>, [Marc Fischer](https://marcfischer.at)<sup>1,2</sup>, [Florian Tramèr](https://floriantramer.com)<sup>1</sup>\n\n<sup>1</sup>ETH Zurich and <sup>2</sup>Invariant Labs\n\n[Read Paper](https://arxiv.org/abs/2406.13352) | [Inspect Results](https://agentdojo.spylab.ai/results/)\n</center>\n\n## Quickstart\n\n```bash\npip install agentdojo\n```\n\n> [!IMPORTANT]\n> Note that the API of the package is still under development and might change in the future.\n\nIf you want to use the prompt injection detector, you need to install the `transformers` extra:\n\n```bash\npip install \"agentdojo[transformers]\"\n```\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attack with tool knowlege, run the following command:\n\n```bash\npython -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \\\n -ut user_task_1 --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\n## Inspect the results\n\nTo inspect the results, go to the dedicated [results page](https://agentdojo.spylab.ai/results/) of the documentation. AgentDojo results are also listed in the [Invariant Benchmark Registry](https://explorer.invariantlabs.ai/benchmarks/).Agent\n\n## Documentation of the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.\n\n## Citing\n\nIf you use AgentDojo in your research, please consider citing our paper:\n\n```bibtex\n@inproceedings{\n debenedetti2024agentdojo,\n title={AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for {LLM} Agents},\n author={Edoardo Debenedetti and Jie Zhang and Mislav Balunovic and Luca Beurer-Kellner and Marc Fischer and Florian Tram{\\`e}r},\n booktitle={The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track},\n year={2024},\n url={https://openreview.net/forum?id=m1YYAQjO3w}\n}\n```\n"}, "fb0818e665a14ce2f44b293fb974fd70a6ca457b": {"t": 1785763909.3727388, "n": 3182, "h": "ed8be5ed5335b9ad", "c": "<center>\n\n# AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents\n\n \n   \n\n\n\n[Edoardo Debenedetti](https://edoardo.science)<sup>1</sup>, [Jie Zhang](https://zj-jayzhang.github.io)<sup>1</sup>, [Mislav Balunović](https://www.sri.inf.ethz.ch/people/mislav)<sup>1,2</sup>, [Luca Beurer-Kellner](https://www.sri.inf.ethz.ch/people/luca)<sup>1,2</sup>, [Marc Fischer](https://marcfischer.at)<sup>1,2</sup>, [Florian Tramèr](https://floriantramer.com)<sup>1</sup>\n\n<sup>1</sup>ETH Zurich and <sup>2</sup>Invariant Labs\n\n[Read Paper](https://arxiv.org/abs/2406.13352) | [Inspect Results](https://agentdojo.spylab.ai/results/)\n</center>\n\n## Quickstart\n\n```bash\npip install agentdojo\n```\n\n> [!IMPORTANT]\n> Note that the API of the package is still under development and might change in the future.\n\nIf you want to use the prompt injection detector, you need to install the `transformers` extra:\n\n```bash\npip install \"agentdojo[transformers]\"\n```\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attack with tool knowlege, run the following command:\n\n```bash\npython -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \\\n -ut user_task_1 --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\n## Inspect the results\n\nTo inspect the results, go to the dedicated [results page](https://agentdojo.spylab.ai/results/) of the documentation. Agent Dojo results are also listed in the [Invariant Benchmark Registry](https://explorer.invariantlabs.ai/benchmarks/).\n\n## Documentation of the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.\n\n## Citing\n\nIf you use AgentDojo in your research, please consider citing our paper:\n\n```bibtex\n@inproceedings{\n debenedetti2024agentdojo,\n title={AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for {LLM} Agents},\n author={Edoardo Debenedetti and Jie Zhang and Mislav Balunovic and Luca Beurer-Kellner and Marc Fischer and Florian Tram{\\`e}r},\n booktitle={The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track},\n year={2024},\n url={https://openreview.net/forum?id=m1YYAQjO3w}\n}\n```\n"}, "c587f6acf348f38cdba81f5197dcc44468163624": {"t": 1785763910.730894, "n": 2869, "h": "0145c6c751bc0506", "c": "<center>\n\n# AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents\n\n \n   \n\n\n\n[Edoardo Debenedetti](https://edoardo.science)<sup>1</sup>, [Jie Zhang](https://zj-jayzhang.github.io)<sup>1</sup>, [Mislav Balunović](https://www.sri.inf.ethz.ch/people/mislav)<sup>1,2</sup>, [Luca Beurer-Kellner](https://www.sri.inf.ethz.ch/people/luca)<sup>1,2</sup>, [Marc Fischer](https://marcfischer.at)<sup>1,2</sup>, [Florian Tramèr](https://floriantramer.com)<sup>1</sup>\n\n<sup>1</sup>ETH Zurich and <sup>2</sup>Invariant Labs\n\nRead our paper [here](https://arxiv.org/abs/2406.13352).\n</center>\n\n## Quickstart\n\n```bash\npip install agentdojo\n```\n\n> [!IMPORTANT]\n> Note that the API of the package is still under development and might change in the future.\n\nIf you want to use the prompt injection detector, you need to install the `transformers` extra:\n\n```bash\npip install \"agentdojo[transformers]\"\n```\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attack with tool knowlege, run the following command:\n\n```bash\npython -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \\\n -ut user_task_1 --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\n## Documentation of the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.\n\n## Citing\n\nIf you use AgentDojo in your research, please consider citing our paper:\n\n```bibtex\n@inproceedings{\n debenedetti2024agentdojo,\n title={AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for {LLM} Agents},\n author={Edoardo Debenedetti and Jie Zhang and Mislav Balunovic and Luca Beurer-Kellner and Marc Fischer and Florian Tram{\\`e}r},\n booktitle={The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track},\n year={2024},\n url={https://openreview.net/forum?id=m1YYAQjO3w}\n}\n```\n"}, "fd469f209f6bd20d5c853e5ae875055014f564b7": {"t": 1785763913.1579108, "n": 2754, "h": "8863be6f8f1e476a", "c": "<center>\n\n# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n \n   \n\n\n\n[Edoardo Debenedetti](https://edoardo.science)<sup>1</sup>, [Jie Zhang](https://zj-jayzhang.github.io)<sup>1</sup>, [Mislav Balunović](https://www.sri.inf.ethz.ch/people/mislav)<sup>1,2</sup>, [Luca Beurer-Kellner](https://www.sri.inf.ethz.ch/people/luca)<sup>1,2</sup>, [Marc Fischer](https://marcfischer.at)<sup>1,2</sup>, [Florian Tramèr](https://floriantramer.com)<sup>1</sup>\n\n<sup>1</sup>ETH Zurich and <sup>2</sup>Invariant Labs\n\nRead our paper [here](https://arxiv.org/abs/2406.13352).\n</center>\n\n## Quickstart\n\n```bash\npip install agentdojo\n```\n\n> [!IMPORTANT]\n> Note that the API of the package is still under development and might change in the future.\n\nIf you want to use the prompt injection detector, you need to install the `transformers` extra:\n\n```bash\npip install \"agentdojo[transformers]\"\n```\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attack with tool knowlege, run the following command:\n\n```bash\npython -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \\\n -ut user_task_1 --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\n## Documentation of the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.\n\n## Citing\n\nIf you use AgentDojo in your research, please consider citing our paper:\n\n```bibtex\n@misc{debenedetti2024agentdojo,\n title={AgentDojo: A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents},\n author={Edoardo Debenedetti and Jie Zhang and Mislav Balunović and Luca Beurer-Kellner and Marc Fischer and Florian Tramèr},\n year={2024},\n eprint={2406.13352},\n archivePrefix={arXiv},\n primaryClass={cs.CR}\n}\n```\n"}, "ef848f3df4f082564b54a730721f4bbe05244202": {"t": 1785763915.1293833, "n": 2349, "h": "02d1f9317739a76e", "c": "<center>\n\n# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n[Edoardo Debenedetti](https://edoardo.science)<sup>1</sup>, [Jie Zhang](https://zj-jayzhang.github.io)<sup>1</sup>, [Mislav Balunović](https://www.sri.inf.ethz.ch/people/mislav)<sup>1,2</sup>, [Luca Beurer-Kellner](https://www.sri.inf.ethz.ch/people/luca)<sup>1,2</sup>, [Marc Fischer](https://marcfischer.at)<sup>1,2</sup>, [Florian Tramèr](https://floriantramer.com)<sup>1</sup>\n\n<sup>1</sup>ETH Zurich and <sup>2</sup>Invariant Labs\n\nRead our paper [here](https://arxiv.org/abs/2406.13352).\n</center>\n\n## Quickstart\n\n```bash\npip install agentdojo\n```\n\n> [!IMPORTANT]\n> Note that the API of the package is still under development and might change in the future.\n\nIf you want to use the prompt injection detector, you need to install the `transformers` extra:\n\n```bash\npip install \"agentdojo[transformers]\"\n```\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attack with tool knowlege, run the following command:\n\n```bash\npython -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \\\n -ut user_task_1 --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\n## Documentation of the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.\n\n## Citing\n\nIf you use AgentDojo in your research, please consider citing our paper:\n\n```bibtex\n@misc{debenedetti2024agentdojo,\n title={AgentDojo: A Dynamic Environment to Evaluate Attacks and Defenses for LLM Agents},\n author={Edoardo Debenedetti and Jie Zhang and Mislav Balunović and Luca Beurer-Kellner and Marc Fischer and Florian Tramèr},\n year={2024},\n eprint={2406.13352},\n archivePrefix={arXiv},\n primaryClass={cs.CR}\n}\n```\n"}, "81f1e964261dc8538e2ff27a08c4b90d27f3820a": {"t": 1785763916.3266783, "n": 1963, "h": "be5c25cf8c13aa30", "c": "<center>\n\n# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n[Edoardo Debenedetti](https://edoardo.science)<sup>1</sup>, [Jie Zhang](https://zj-jayzhang.github.io)<sup>1</sup>, [Mislav Balunović](https://www.sri.inf.ethz.ch/people/mislav)<sup>1,2</sup>, [Luca Beurer-Kellner](https://www.sri.inf.ethz.ch/people/luca)<sup>1,2</sup>, [Marc Fischer](https://marcfischer.at)<sup>1,2</sup>, [Florian Tramèr](https://floriantramer.com)<sup>1</sup>\n\n<sup>1</sup>ETH Zurich and <sup>2</sup>Invariant Labs\n\nRead our paper [here](TODO).\n</center>\n\n## Quickstart\n\n```bash\npip install agentdojo\n```\n\n> [!IMPORTANT]\n> Note that the API of the package is still under development and might change in the future.\n\nIf you want to use the prompt injection detector, you need to install the `transformers` extra:\n\n```bash\npip install \"agentdojo[transformers]\"\n```\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attack with tool knowlege, run the following command:\n\n```bash\npython -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \\\n -ut user_task_1 --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\n## Documentation of the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.\n\n## Citing\n\nIf you use AgentDojo in your research, please consider citing our paper:\n\n```bibtex\nTODO\n```\n"}, "aa43300153dc92568d88b724c4f637390de3bcc7": {"t": 1785763917.8139768, "n": 1853, "h": "b149402a00a8d10b", "c": "<center>\n\n# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n[Edoardo Debenedetti](https://edoardo.science)<sup>1</sup>, [Jie Zhang](https://zj-jayzhang.github.io)<sup>1</sup>, [Mislav Balunović](https://www.sri.inf.ethz.ch/people/mislav)<sup>1,2</sup>, [Luca Beurer-Kellner](https://www.sri.inf.ethz.ch/people/luca)<sup>1,2</sup>, [Marc Fischer](https://marcfischer.at)<sup>1,2</sup>, [Florian Tramèr](https://floriantramer.com)<sup>1</sup>\n\n<sup>1</sup>ETH Zurich and <sup>2</sup>Invariant Labs\n\nRead our paper [here](TODO).\n</center>\n\n## Quickstart\n\n```bash\npip install agentdojo\n```\n\nIf you want to use the prompt injection detector, you need to install the `transformers` extra:\n\n```bash\npip install \"agentdojo[transformers]\"\n```\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attack with tool knowlege, run the following command:\n\n```bash\npython -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \\\n -ut user_task_1 --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\n## Documentation of the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.\n\n## Citing\n\nIf you use AgentDojo in your research, please consider citing our paper:\n\n```bibtex\nTODO\n```\n"}, "d4b1b9533e73541ecdffc5f27df0677398d6fdd2": {"t": 1785763919.3711536, "n": 1822, "h": "6cb83965e0349493", "c": "# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n[Edoardo Debenedetti](https://edoardo.science)<sup>1</sup>, [Jie Zhang](https://zj-jayzhang.github.io)<sup>1</sup>, [Mislav Balunović](https://www.sri.inf.ethz.ch/people/mislav)<sup>1,2</sup>, [Luca Beurer-Kellner](https://www.sri.inf.ethz.ch/people/luca)<sup>1,2</sup>, [Marc Fischer](https://marcfischer.at)<sup>1,2</sup>, [Florian Tramèr](https://floriantramer.com)<sup>1</sup> | <sup>1</sup>ETH Zurich and <sup>2</sup>Invariant Labs\n\n**Paper**: TODO.\n\n## Quickstart\n\n```bash\npip install agentdojo\n```\n\nIf you want to use the prompt injection detector, you need to install the `transformers` extra:\n\n```bash\npip install \"agentdojo[transformers]\"\n```\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attack with tool knowlege, run the following command:\n\n```bash\npython -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \\\n -ut user_task_1 --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attack tool_knowledge\n```\n\n## Documentation of the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.\n\n## Citing\n\nIf you use AgentDojo in your research, please consider citing our paper:\n\n```bibtex\nTODO\n```\n"}, "14cc9058e844eaf16d0f82aac5c17dc240fdb465": {"t": 1785763920.7285326, "n": 1695, "h": "1827fbb1250c4ca0", "c": "# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n[Edoardo Debenedetti](https://edoardo.science), [Jie Zhang](https://zj-jayzhang.github.io), [Mislav Balunović](https://www.sri.inf.ethz.ch/people/mislav), [Luca Beurer-Kellner](https://www.sri.inf.ethz.ch/people/luca), [Marc Fischer](https://marcfischer.at), [Florian Tramèr](https://floriantramer.com).\n\n**Paper**: TODO.\n\n## Quickstart\n\n```bash\npip install agentdojo\n```\n\nIf you want to use the prompt injection detector, you need to install the `transformers` extra:\n\n```bash\npip install \"agentdojo[transformers]\"\n```\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \\\n -ut user_task_1 --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attacker tool_knowledge\n```\n\n## Documentation of the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.\n\n## Citing\n\nIf you use AgentDojo in your research, please consider citing our paper:\n\n```bibtex\nTODO\n```\n"}, "d294103b928cc1862e411295634ff9ddc6bac767": {"t": 1785763921.9561167, "n": 1267, "h": "71de523673987b1f", "c": "# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Quickstart\n\n```bash\npip install agentdojo\n```\n\nIf you want to use the prompt injection detector, you need to install the `transformers` extra:\n\n```bash\npip install \"agentdojo[transformers]\"\n```\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \\\n -ut user_task_1 --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attacker tool_knowledge\n```\n\n## Documentation of the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.\n"}, "c8fad6facba6fb008c8a507252937b3c92aa40ee": {"t": 1785763923.1085448, "n": 1069, "h": "69de31b2a51ff7f7", "c": "# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 \\\n -ut user_task_1 --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 \\\n --defense tool_filter --attacker tool_knowledge\n```\n\n## Documentation on the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.\n"}, "967fb5afe2a36799ca88dac2f4d6182731b4f1e5": {"t": 1785763925.8596904, "n": 1051, "h": "e6eefaddcfa75859", "c": "# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](src/agentdojo/scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython -m agentdojo.scripts.benchmark -s workspace -ut user_task_0 -ut user_task_1 --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython -m agentdojo.scripts.benchmark --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\n## Documentation on the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.\n"}, "ec2519ffb0d0ec31117a867f40d4656186d204dc": {"t": 1785763927.0956583, "n": 1017, "h": "35860e4409697021", "c": "# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython scripts/benchmark.py -s workspace -ut user_task_0 -ut user_task_1 --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython scripts/benchmark.py --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\n## Documentation on the Dojo\n\nTake a look at our [documentation](https://agentdojo.spylab.ai/).\n\n## Development set-up\n\nTake a look at the [development set-up](https://agentdojo.spylab.ai/docs/development) docs.\n"}, "22c017f19215e385a8edd2e432e948ab20b8c836": {"t": 1785763928.3193846, "n": 1573, "h": "ae862dc5171d559a", "c": "# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython scripts/benchmark.py -s workspace -ut user_task_0 -ut user_task_1 --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython scripts/benchmark.py --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\n## How to use the dojo?\n\nTake a look at our [documentation](https://agentdojo.spylab.ai).\n\n## Development set-up\n\nWe use `rye`. You can follow the instructions on the [website](https://rye-up.com) to set it up.\n\n### Install the dev packages, lint, test, typecheck\n\n- Install all the packages with `rye sync --features transformers` (include `--features transformers` if you want to work on the PI detector)\n- Lint with `rye lint --fix`\n- Test with `rye test`\n- Run the static type checker with `rye run pyright`\n\n### What if I don't want to install `rye`?\n\nThen you can set up the dev environment via `uv` or `pip` by running\n`[uv] pip install -r requirements-dev.lock`.\n\nThen, you can lint with `ruff check . --fix`, test with `pytest`, and typecheck with `pyright`.\n"}, "17b52b975ccdc84ffd89b1ee9b95a6689e420d02": {"t": 1785763929.840266, "n": 1565, "h": "c927f31e4b96ea73", "c": "# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython scripts/benchmark.py -s workspace -ut user_task_0 -ut user_task_1 --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython scripts/benchmark.py --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\n## How to use the dojo?\n\nTake a look at our [documentation](agentdojo.spylab.ai).\n\n## Development set-up\n\nWe use `rye`. You can follow the instructions on the [website](https://rye-up.com) to set it up.\n\n### Install the dev packages, lint, test, typecheck\n\n- Install all the packages with `rye sync --features transformers` (include `--features transformers` if you want to work on the PI detector)\n- Lint with `rye lint --fix`\n- Test with `rye test`\n- Run the static type checker with `rye run pyright`\n\n### What if I don't want to install `rye`?\n\nThen you can set up the dev environment via `uv` or `pip` by running\n`[uv] pip install -r requirements-dev.lock`.\n\nThen, you can lint with `ruff check . --fix`, test with `pytest`, and typecheck with `pyright`.\n"}, "ad9ff9e93f1f5e2df816b6a938ef136950e05cdc": {"t": 1785763931.0824094, "n": 9194, "h": "ed74f6562718d2c2", "c": "# AgentDojo: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython scripts/benchmark.py -s workspace -ut user_task_0 -ut user_task_1 --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython scripts/benchmark.py --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\n## Tasks definition\n\nThe (benign) user tasks are defined by subclassing `BaseUserTask`, while the (malicious) injection tasks are defined by subclassing `BaseInjectionTask`. All tasks need to be registered with the task suite they belong to.\n\n### Task suite\n\nEach group of tasks is defined by a `TaskSuite` instantiation. The task suite is resposnible for defining:\n\n- The name of the suite\n- The tasks that belong to the suite\n- The tools avaliable for the suite tasks\n- The type for the environment used by the suite\n\nAn example is availabe in the [`workspace`](src/agentdojo/task_suites/workspace/task_suite.py) task suite. Each suite is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks.\n\nThe task suite has also the ability to run tasks with the `run_task` method. This method takes as argument the user task to run and the injection task to check for. If no injection task is passed, then the method will only run the user task. The method returns booleans indicating if the task and/or the injection were successful. The method also takes as an optional argument the environment to use for the tasks. If no environment is passed, then the method will use the default environment for the suite. Moreover, the method takes as argument the injections to inject within the (default environment).\n\nThe environment for the task is defined in `src/agentdojo/data/suites/{suite_name}/environment.yaml` and the injection vectors for the environment are defined in `src/function_calling_pi/data/suites/{suite_name}/injection_vectors.yaml`.\n\n### User tasks\n\nEach suite has a set of (benign) user tasks. Each user task is an instantiation of the [`BaseUserTask`](src/agentdojo/task_suites/base_tasks.py) class. Also each user task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each user task **must** define:\n\n- A `PROMPT`: which is the prompt that will be passed to the LLM\n- A `COMMENT`: which is a comment useful for humans that need to understand the task\n- A `GROUND_TRUTH_OUTPUT`: which is a portion of the expected output of the task.\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should solve the task. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `utility` method, used to calculate the utility of the task. It takes as argument the original environment and the environment after the LLM executed the task. Returns a boolean indicating if the task was executed successfully.\n\nAdditionally, each user task _may_ define an `init_environment` static method, which is used to initialize the environment for the task. This is useful for tasks that need to set up the environment before the task is executed. If the original environment is enough for the task, then this method is not needed.\n\nEach user task should be decorated with the `@task_suite.register_user_task`, where `task_suite` is the task suite that the task belongs to.\n\nAn example implementation of a user task is available in the [`workspace`](src/agentdojo/task_suites/workspace/user_tasks.py) module.\n\n### Injection tasks\n\nEach suite has a set of (malicious) injection tasks. Each injection task is an instantiation of the [`BaseInjectionTask`](src/agentdojo/task_suites/base_tasks.py) class. Also each injection task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each injection task **must** define:\n\n- A `GOAL`: which is the goal that needs to be achieved via the injection\n- A `COMMENT`: which is a comment useful for humans that need to understand the injection goal\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should execute the injection. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `security` method, used to calculate if the injection was successful. It takes as argument the original environment and the environment after the LLM executed the task. It returns a boolean indicating if the injection was successful.\n\n### Checking the correctness of the task suite\n\nThe correctness of the task suite can be checked with the [check_suite.py](scripts/check_suites.py) script. By default it checks all the suites, but, optionally takes as arguments the name of the selected suites to check, each after the argument `-s`, and checks if:\n\n- The `ground_truth` method can be correctly executed.\n- The `ground_truth` and `utility`/`security` methods are consistent with each other.\n- If all the user tasks are injectable, i.e., if at least an injection placeholder is part of what is return by a tool and seen by the agent.\n\nThe script can be run with the following command (e.g., for the `workspace` suite):\n\n```bash\npython scripts/check_suites.py -s workspace\n```\n\nOr for all suites:\n\n```bash\npython scripts/check_suites.py\n```\n\n## Adding Tools\n\nTools are functions that can be used by the tasks to perform some operations. Tools can be stateless or stateful.\n\nEach tool must have type hints for both the input arguments and the return value. The type hints are used to generate the documentation for the tools. Moreover, tools **must** contain documentation for all the input argument (with the exception of the state argument for statefult tools, which **must not** be documented).\n\nAll tools can be tested as regular functions.\n\n### Stateless tools\n\nTools that do not need to access any state, can be registered with the `@register_function` decorator. Each tool can be defined as a function that takes any number of arguments and returns the result of the tool.\n\nAn example implementation of stateless tool is\n\n```python\nfrom agentdojo import register_function\n\n\n@register_function\ndef sum(a: int, b: int) -> int:\n \"\"\"Sums to numbers.\n\n :param a: First number\n :param b: Second number\n \"\"\"\n return a + b\n```\n\n### Stateful tools\n\nTools that need to access some state, must also be registered with the `@register_function` decorator. Each tool can be defined as a function that takes an arbitrary number of arguments.\n\nArguments that need to be injected from the environment are called _dependencies_ and work similarly to [FastAPI dependencies](https://fastapi.tiangolo.com/tutorial/dependencies/). Assuming a dependency as type `ExampleType`, and the dependency is inside of benchmark environments with the name `\"example_name\"`, then the argument needs to be type hinted using the type `Annotated[ExampleType, Depends(\"example_name\")]`.\n\n> [!NOTE]\n> The tool is responsible for updating the state (and need to make sure that the update happens in-place!)\n\nAn example implementation of stateful tool is\n\n```python\nfrom typing import Annotated\nfrom agentdojo import Depends, register_function\n\nfrom pydantic import BaseModel\n\n\nclass Counter:\n def __init__(self):\n self.counter = 0\n\nclass Environment(BaseModel):\n counter: Counter\n\n\n@register_function\ndef add_to_counter(\n counter: Annotated[Counter, Depends(\"counter\")], n: int = 1,\n) -> int:\n \"\"\"Adds an arbitrary number `n` to the counter and returns the new value.\n\n :param n: The number to add to the counter. Default is 1.\n \"\"\"\n counter.counter += 1\n return counter.counter\n\n```\n\n> [!NOTE]\n> Note that the `counter` argument is a dependency and is updated in-place, and is **not** documented in the docstring.\n> This is because the agent does not need to know about this argument as it is injected by the engine executing the function.\n\n## Development set-up\n\nWe use `rye`. You can follow the instructions on the [website](https://rye-up.com) to set it up.\n\n### Install the dev packages, lint, test, typecheck\n\n- Install all the packages with `rye sync --features transformers` (include `--features transformers` if you want to work on the PI detector)\n- Lint with `rye lint --fix`\n- Test with `rye test`\n- Run the static type checker with `rye run pyright`\n\n### What if I don't want to install `rye`?\n\nThen you can set up the dev environment via `uv` or `pip` by running\n`[uv] pip install -r requirements-dev.lock`.\n\nThen, you can lint with `ruff check . --fix`, test with `pytest`, and typecheck with `pyright`.\n"}, "23559ab3d3b45a26fe375c303576345221956d17": {"t": 1785763933.4316578, "n": 9186, "h": "32d6290928cf0be6", "c": "# AgentGym: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython scripts/benchmark.py -s workspace -ut user_task_0 -ut user_task_1 --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython scripts/benchmark.py --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\n## Tasks definition\n\nThe (benign) user tasks are defined by subclassing `BaseUserTask`, while the (malicious) injection tasks are defined by subclassing `BaseInjectionTask`. All tasks need to be registered with the task suite they belong to.\n\n### Task suite\n\nEach group of tasks is defined by a `TaskSuite` instantiation. The task suite is resposnible for defining:\n\n- The name of the suite\n- The tasks that belong to the suite\n- The tools avaliable for the suite tasks\n- The type for the environment used by the suite\n\nAn example is availabe in the [`workspace`](src/agentgym/task_suites/workspace/task_suite.py) task suite. Each suite is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks.\n\nThe task suite has also the ability to run tasks with the `run_task` method. This method takes as argument the user task to run and the injection task to check for. If no injection task is passed, then the method will only run the user task. The method returns booleans indicating if the task and/or the injection were successful. The method also takes as an optional argument the environment to use for the tasks. If no environment is passed, then the method will use the default environment for the suite. Moreover, the method takes as argument the injections to inject within the (default environment).\n\nThe environment for the task is defined in `src/agentgym/data/suites/{suite_name}/environment.yaml` and the injection vectors for the environment are defined in `src/function_calling_pi/data/suites/{suite_name}/injection_vectors.yaml`.\n\n### User tasks\n\nEach suite has a set of (benign) user tasks. Each user task is an instantiation of the [`BaseUserTask`](src/agentgym/task_suites/base_tasks.py) class. Also each user task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each user task **must** define:\n\n- A `PROMPT`: which is the prompt that will be passed to the LLM\n- A `COMMENT`: which is a comment useful for humans that need to understand the task\n- A `GROUND_TRUTH_OUTPUT`: which is a portion of the expected output of the task.\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should solve the task. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `utility` method, used to calculate the utility of the task. It takes as argument the original environment and the environment after the LLM executed the task. Returns a boolean indicating if the task was executed successfully.\n\nAdditionally, each user task _may_ define an `init_environment` static method, which is used to initialize the environment for the task. This is useful for tasks that need to set up the environment before the task is executed. If the original environment is enough for the task, then this method is not needed.\n\nEach user task should be decorated with the `@task_suite.register_user_task`, where `task_suite` is the task suite that the task belongs to.\n\nAn example implementation of a user task is available in the [`workspace`](src/agentgym/task_suites/workspace/user_tasks.py) module.\n\n### Injection tasks\n\nEach suite has a set of (malicious) injection tasks. Each injection task is an instantiation of the [`BaseInjectionTask`](src/agentgym/task_suites/base_tasks.py) class. Also each injection task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each injection task **must** define:\n\n- A `GOAL`: which is the goal that needs to be achieved via the injection\n- A `COMMENT`: which is a comment useful for humans that need to understand the injection goal\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should execute the injection. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `security` method, used to calculate if the injection was successful. It takes as argument the original environment and the environment after the LLM executed the task. It returns a boolean indicating if the injection was successful.\n\n### Checking the correctness of the task suite\n\nThe correctness of the task suite can be checked with the [check_suite.py](scripts/check_suites.py) script. By default it checks all the suites, but, optionally takes as arguments the name of the selected suites to check, each after the argument `-s`, and checks if:\n\n- The `ground_truth` method can be correctly executed.\n- The `ground_truth` and `utility`/`security` methods are consistent with each other.\n- If all the user tasks are injectable, i.e., if at least an injection placeholder is part of what is return by a tool and seen by the agent.\n\nThe script can be run with the following command (e.g., for the `workspace` suite):\n\n```bash\npython scripts/check_suites.py -s workspace\n```\n\nOr for all suites:\n\n```bash\npython scripts/check_suites.py\n```\n\n## Adding Tools\n\nTools are functions that can be used by the tasks to perform some operations. Tools can be stateless or stateful.\n\nEach tool must have type hints for both the input arguments and the return value. The type hints are used to generate the documentation for the tools. Moreover, tools **must** contain documentation for all the input argument (with the exception of the state argument for statefult tools, which **must not** be documented).\n\nAll tools can be tested as regular functions.\n\n### Stateless tools\n\nTools that do not need to access any state, can be registered with the `@register_function` decorator. Each tool can be defined as a function that takes any number of arguments and returns the result of the tool.\n\nAn example implementation of stateless tool is\n\n```python\nfrom agentgym import register_function\n\n\n@register_function\ndef sum(a: int, b: int) -> int:\n \"\"\"Sums to numbers.\n\n :param a: First number\n :param b: Second number\n \"\"\"\n return a + b\n```\n\n### Stateful tools\n\nTools that need to access some state, must also be registered with the `@register_function` decorator. Each tool can be defined as a function that takes an arbitrary number of arguments.\n\nArguments that need to be injected from the environment are called _dependencies_ and work similarly to [FastAPI dependencies](https://fastapi.tiangolo.com/tutorial/dependencies/). Assuming a dependency as type `ExampleType`, and the dependency is inside of benchmark environments with the name `\"example_name\"`, then the argument needs to be type hinted using the type `Annotated[ExampleType, Depends(\"example_name\")]`.\n\n> [!NOTE]\n> The tool is responsible for updating the state (and need to make sure that the update happens in-place!)\n\nAn example implementation of stateful tool is\n\n```python\nfrom typing import Annotated\nfrom agentgym import Depends, register_function\n\nfrom pydantic import BaseModel\n\n\nclass Counter:\n def __init__(self):\n self.counter = 0\n\nclass Environment(BaseModel):\n counter: Counter\n\n\n@register_function\ndef add_to_counter(\n counter: Annotated[Counter, Depends(\"counter\")], n: int = 1,\n) -> int:\n \"\"\"Adds an arbitrary number `n` to the counter and returns the new value.\n\n :param n: The number to add to the counter. Default is 1.\n \"\"\"\n counter.counter += 1\n return counter.counter\n\n```\n\n> [!NOTE]\n> Note that the `counter` argument is a dependency and is updated in-place, and is **not** documented in the docstring.\n> This is because the agent does not need to know about this argument as it is injected by the engine executing the function.\n\n## Development set-up\n\nWe use `rye`. You can follow the instructions on the [website](https://rye-up.com) to set it up.\n\n### Install the dev packages, lint, test, typecheck\n\n- Install all the packages with `rye sync --features transformers` (include `--features transformers` if you want to work on the PI detector)\n- Lint with `rye lint --fix`\n- Test with `rye test`\n- Run the static type checker with `rye run pyright`\n\n### What if I don't want to install `rye`?\n\nThen you can set up the dev environment via `uv` or `pip` by running\n`[uv] pip install -r requirements-dev.lock`.\n\nThen, you can lint with `ruff check . --fix`, test with `pytest`, and typecheck with `pyright`.\n"}, "4581861b4f1cc2b2c3fd970d5929fceb15c57d7a": {"t": 1785763934.7071555, "n": 9187, "h": "7a6874a064d3e88c", "c": "# AgentGym: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython scripts/benchmark.py -s workspace -ut user_task_0 -ut user_task_1 --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython scripts/benchmark.py --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\n## Tasks definition\n\nThe (benign) user tasks are defined by subclassing `BaseUserTask`, while the (malicious) injection tasks are defined by subclassing `BaseInjectionTask`. All tasks need to be registered with the task suite they belong to.\n\n### Task suite\n\nEach group of tasks is defined by a `TaskSuite` instantiation. The task suite is resposnible for defining:\n\n- The name of the suite\n- The tasks that belong to the suite\n- The tools avaliable for the suite tasks\n- The type for the environment used by the suite\n\nAn example is availabe in the [`workspace`](src/agentgym/task_suites/workspace/task_suite.py) task suite. Each suite is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks.\n\nThe task suite has also the ability to run tasks with the `run_task` method. This method takes as argument the user task to run and the injection task to check for. If no injection task is passed, then the method will only run the user task. The method returns booleans indicating if the task and/or the injection were successful. The method also takes as an optional argument the environment to use for the tasks. If no environment is passed, then the method will use the default environment for the suite. Moreover, the method takes as argument the injections to inject within the (default environment).\n\nThe environment for the task is defined in `src/agentgym/data/suites/{suite_name}/environment.yaml` and the injection vectors for the environment are defined in `src/function_calling_pi/data/suites/{suite_name}/injection_vectors.yaml`.\n\n### User tasks\n\nEach suite has a set of (benign) user tasks. Each user task is an instantiation of the [`BaseUserTask`](src/agentgym/task_suites/base_tasks.py) class. Also each user task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each user task **must** define:\n\n- A `PROMPT`: which is the prompt that will be passed to the LLM\n- A `COMMENT`: which is a comment useful for humans that need to understand the task\n- A `GROUND_TRUTH_OUTPUT`: which is a portion of the expected output of the task.\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should solve the task. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `utility` method, used to calculate the utility of the task. It takes as argument the original environment and the environment after the LLM executed the task. Returns a boolean indicating if the task was executed successfully.\n\nAdditionally, each user task _may_ define an `init_environment` static method, which is used to initialize the environment for the task. This is useful for tasks that need to set up the environment before the task is executed. If the original environment is enough for the task, then this method is not needed.\n\nEach user task should be decorated with the `@task_suite.register_user_task`, where `task_suite` is the task suite that the task belongs to.\n\nAn example implementation of a user task is available in the [`workspace`](src/agentgym/task_suites/workspace/user_tasks.py) module.\n\n### Injection tasks\n\nEach suite has a set of (malicious) injection tasks. Each injection task is an instantiation of the [`BaseInjectionTask`](src/agentgym/task_suites/base_tasks.py) class. Also each injection task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each injection task **must** define:\n\n- A `GOAL`: which is the goal that needs to be achieved via the injection\n- A `COMMENT`: which is a comment useful for humans that need to understand the injection goal\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should execute the injection. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `security` method, used to calculate if the injection was successful. It takes as argument the original environment and the environment after the LLM executed the task. It returns a boolean indicating if the injection was successful.\n\n### Checking the correctness of the task suite\n\nThe correctness of the task suite can be checked with the [check_suite.py](scripts/check_suites.py) script. By default it checks all the suites, but, optionally takes as arguments the name of the selected suites to check, each after the argument `-s`, and checks if:\n\n- The `ground_truth` method can be correctly executed.\n- The `ground_truth` and `utility`/`security` methods are consistent with each other.\n- If all the user tasks are injectable, i.e., if at least an injection placeholder is part of what is return by a tool and seen by the agent.\n\nThe script can be run with the following command (e.g., for the `workspace` suite):\n\n```bash\npython scripts/check_suites.py -s workspace\n```\n\nOr for all suites:\n\n```bash\npython scripts/check_suites.py\n```\n\n## Adding Tools\n\nTools are functions that can be used by the tasks to perform some operations. Tools can be stateless or stateful.\n\nEach tool must have type hints for both the input arguments and the return value. The type hints are used to generate the documentation for the tools. Moreover, tools **must** contain documentation for all the input argument (with the exception of the state argument for statefult tools, which **must not** be documented).\n\nAll tools can be tested as regular functions.\n\n### Stateless tools\n\nTools that do not need to access any state, can be registered with the `@register_function` decorator. Each tool can be defined as a function that takes any number of arguments and returns the result of the tool.\n\nAn example implementation of stateless tool is\n\n```python\nfrom agentgym import register_function\n\n\n@register_function\ndef sum(a: int, b: int) -> int:\n \"\"\"Sums to numbers.\n\n :param a: First number\n :param b: Second number\n \"\"\"\n return a + b\n```\n\n### Stateful tools\n\nTools that need to access some state, must also be registered with the `@register_function` decorator. Each tool can be defined as a function that takes an arbitrary number of arguments.\n\nArguments that need to be injected from the environment are called _dependencies_ and work similarly to [FastAPI dependencies](https://fastapi.tiangolo.com/tutorial/dependencies/). Assuming a dependency as type `ExampleType`, and the dependency is inside of benchmark environments with the name `\"example_name\"`, then the argument needs to be type hinted using the type `Annotated[ExampleType, Depends(\"example_name\")]`.\n\n> [!NOTE]\n> The tool is responsible for updating the state (and need to make sure that the update happens in-place!)\n\nAn example implementation of stateful tool is\n\n```python\nfrom typing import Annotated\nfrom agentgym import Depends, register_function\n\nfrom pydantic import BaseModel\n\n\nclass Environment(BaseModel):\n counter: Counter\n\n\nclass Counter():\n def __init__(self):\n self.counter = 0\n\n\n@register_function\ndef add_to_counter(\n counter: Annotated[Counter, Depends(\"counter\")], n: int = 1,\n) -> int:\n \"\"\"Adds an arbitrary number `n` to the counter and returns the new value.\n\n :param n: The number to add to the counter. Default is 1.\n \"\"\"\n counter.counter += 1\n return counter.counter\n\n```\n\n> [!NOTE]\n> Note that the `counter` argument is a dependency and is updated in-place, and is **not** documented in the docstring. This is because the agent does not need to know about this argument as it is injected by the engine executing the function.\n\n## Development set-up\n\nWe use `rye`. You can follow the instructions on the [website](https://rye-up.com) to set it up.\n\n### Install the dev packages, lint, test, typecheck\n\n- Install all the packages with `rye sync --features transformers` (include `--features transformers` if you want to work on the PI detector)\n- Lint with `rye lint --fix`\n- Test with `rye test`\n- Run the static type checker with `rye run pyright`\n\n### What if I don't want to install `rye`?\n\nThen you can set up the dev environment via `uv` or `pip` by running `[uv] pip install -r requirements-dev.lock`.\n\nThen, you can lint with `ruff check . --fix`, test with `pytest`, and typecheck with `pyright`.\n"}, "216318e7094c857997024432a28f81510baa4836": {"t": 1785763936.9938056, "n": 9196, "h": "c36d5da9e9e91053", "c": "# AgentGym: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython scripts/benchmark.py -s workspace -ut user_task_0 -ut user_task_1 --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython scripts/benchmark.py --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\n## Tasks definition\n\nThe (benign) user tasks are defined by subclassing `BaseUserTask`, while the (malicious) injection tasks are defined by subclassing `BaseInjectionTask`. All tasks need to be registered with the task suite they belong to.\n\n### Task suite\n\nEach group of tasks is defined by a `TaskSuite` instantiation. The task suite is resposnible for defining:\n\n- The name of the suite\n- The tasks that belong to the suite\n- The tools avaliable for the suite tasks\n- The type for the environment used by the suite\n\nAn example is availabe in the [`workspace`](src/agentgym/task_suites/workspace/task_suite.py) task suite. Each suite is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks.\n\nThe task suite has also the ability to run tasks with the `run_task` method. This method takes as argument the user task to run and the injection task to check for. If no injection task is passed, then the method will only run the user task. The method returns booleans indicating if the task and/or the injection were successful. The method also takes as an optional argument the environment to use for the tasks. If no environment is passed, then the method will use the default environment for the suite. Moreover, the method takes as argument the injections to inject within the (default environment).\n\nThe environment for the task is defined in `src/agentgym/data/suites/{suite_name}/environment.yaml` and the injection vectors for the environment are defined in `src/function_calling_pi/data/suites/{suite_name}/injection_vectors.yaml`.\n\n### User tasks\n\nEach suite has a set of (benign) user tasks. Each user task is an instantiation of the [`BaseUserTask`](src/agentgym/task_suites/base_tasks.py) class. Also each user task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each user task **must** define:\n\n- A `PROMPT`: which is the prompt that will be passed to the LLM\n- A `COMMENT`: which is a comment useful for humans that need to understand the task\n- A `GROUND_TRUTH_OUTPUT`: which is a portion of the expected output of the task.\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should solve the task. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `utility` method, used to calculate the utility of the task. It takes as argument the original environment and the environment after the LLM executed the task. Returns a boolean indicating if the task was executed successfully.\n\nAdditionally, each user task _may_ define an `init_environment` static method, which is used to initialize the environment for the task. This is useful for tasks that need to set up the environment before the task is executed. If the original environment is enough for the task, then this method is not needed.\n\nEach user task should be decorated with the `@task_suite.register_user_task`, where `task_suite` is the task suite that the task belongs to.\n\nAn example implementation of a user task is available in the [`workspace`](src/agentgym/task_suites/workspace/user_tasks.py) module.\n\n### Injection tasks\n\nEach suite has a set of (malicious) injection tasks. Each injection task is an instantiation of the [`BaseInjectionTask`](src/agentgym/task_suites/base_tasks.py) class. Also each injection task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each injection task **must** define:\n\n- A `GOAL`: which is the goal that needs to be achieved via the injection\n- A `COMMENT`: which is a comment useful for humans that need to understand the injection goal\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should execute the injection. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `security` method, used to calculate if the injection was successful. It takes as argument the original environment and the environment after the LLM executed the task. It returns a boolean indicating if the injection was successful.\n\n### Checking the correctness of the task suite\n\nThe correctness of the task suite can be checked with the [check_suite.py](scripts/check_suites.py) script. By default it checks all the suites, but, optionally takes as arguments the name of the selected suites to check, each after the argument `-s`, and checks if:\n\n- The `ground_truth` method can be correctly executed.\n- The `ground_truth` and `utility`/`security` methods are consistent with each other.\n- If all the user tasks are injectable, i.e., if at least an injection placeholder is part of what is return by a tool and seen by the agent.\n\nThe script can be run with the following command (e.g., for the `workspace` suite):\n\n```bash\npython scripts/check_suites.py -s workspace\n```\n\nOr for all suites:\n\n```bash\npython scripts/check_suites.py\n```\n\n## Adding Tools\n\nTools are functions that can be used by the tasks to perform some operations. Tools can be stateless or stateful.\n\nEach tool must have type hints for both the input arguments and the return value. The type hints are used to generate the documentation for the tools. Moreover, tools **must** contain documentation for all the input argument (with the exception of the state argument for statefult tools, which **must not** be documented).\n\nAll tools can be tested as regular functions.\n\n### Stateless tools\n\nTools that do not need to access any state, can be registered with the `@register_function` decorator. Each tool can be defined as a function that takes any number of arguments and returns the result of the tool.\n\nAn example implementation of stateless tool is\n\n```python\nfrom agentgym import register_function\n\n\n@register_function\ndef sum(a: int, b: int) -> int:\n \"\"\"Sums to numbers.\n\n :param a: First number\n :param b: Second number\n \"\"\"\n return a + b\n```\n\n### Stateful tools\n\nTools that need to access some state, must also be registered with the `@register_function` decorator. Each tool can be defined as a function that takes an arbitrary number of arguments.\n\nArguments that need to be injected from the environment are called _dependencies_ and work similarly to [FastAPI dependencies](https://fastapi.tiangolo.com/tutorial/dependencies/). Assuming a dependency as type `ExampleType`, and the dependency is inside of benchmark environments with the name `\"example_name\"`, then the argument needs to be type hinted using the type `Annotated[ExampleType, Depends(\"example_name\")]`.\n\n> [!NOTE]\n> The tool is responsible for updating the state (and need to make sure that the update happens in-place!)\n\nAn example implementation of stateful tool is\n\n```python\nfrom typing import Annotated\nfrom agentgym import Depends, register_stateful_function\n\nfrom pydantic import BaseModel\n\n\nclass Environment(BaseModel):\n counter: Counter\n\n\nclass Counter():\n def __init__(self):\n self.counter = 0\n\n\n@register_function\ndef add_to_counter(\n counter: Annotated[Counter, Depends(\"counter\")], n: int = 1,\n) -> int:\n \"\"\"Adds an arbitrary number `n` to the counter and returns the new value.\n\n :param n: The number to add to the counter. Default is 1.\n \"\"\"\n counter.counter += 1\n return counter.counter\n\n```\n\n> [!NOTE]\n> Note that the `counter` argument is a dependency and is updated in-place, and is **not** documented in the docstring. This is because the agent does not need to know about this argument as it is injected by the engine executing the function.\n\n## Development set-up\n\nWe use `rye`. You can follow the instructions on the [website](https://rye-up.com) to set it up.\n\n### Install the dev packages, lint, test, typecheck\n\n- Install all the packages with `rye sync --features transformers` (include `--features transformers` if you want to work on the PI detector)\n- Lint with `rye lint --fix`\n- Test with `rye test`\n- Run the static type checker with `rye run pyright`\n\n### What if I don't want to install `rye`?\n\nThen you can set up the dev environment via `uv` or `pip` by running `[uv] pip install -r requirements-dev.lock`.\n\nThen, you can lint with `ruff check . --fix`, test with `pytest`, and typecheck with `pyright`.\n"}, "144849d48f042c4936d8a525699c626ed3cab1d8": {"t": 1785763938.959529, "n": 9266, "h": "83d3b6617118414d", "c": "# TODOBench: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython scripts/benchmark.py -s workspace -ut user_task_0 -ut user_task_1 --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython scripts/benchmark.py --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\n## Tasks definition\n\nThe (benign) user tasks are defined by subclassing `BaseUserTask`, while the (malicious) injection tasks are defined by subclassing `BaseInjectionTask`. All tasks need to be registered with the task suite they belong to.\n\n### Task suite\n\nEach group of tasks is defined by a `TaskSuite` instantiation. The task suite is resposnible for defining:\n\n- The name of the suite\n- The tasks that belong to the suite\n- The tools avaliable for the suite tasks\n- The type for the environment used by the suite\n\nAn example is availabe in the [`workspace`](src/function_calling_pi/task_suites/workspace/task_suite.py) task suite. Each suite is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks.\n\nThe task suite has also the ability to run tasks with the `run_task` method. This method takes as argument the user task to run and the injection task to check for. If no injection task is passed, then the method will only run the user task. The method returns booleans indicating if the task and/or the injection were successful. The method also takes as an optional argument the environment to use for the tasks. If no environment is passed, then the method will use the default environment for the suite. Moreover, the method takes as argument the injections to inject within the (default environment).\n\nThe environment for the task is defined in `src/function_calling_pi/data/suites/{suite_name}/environment.yaml` and the injection vectors for the environment are defined in `src/function_calling_pi/data/suites/{suite_name}/injection_vectors.yaml`.\n\n### User tasks\n\nEach suite has a set of (benign) user tasks. Each user task is an instantiation of the [`BaseUserTask`](src/function_calling_pi/task_suites/base_tasks.py) class. Also each user task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each user task **must** define:\n\n- A `PROMPT`: which is the prompt that will be passed to the LLM\n- A `COMMENT`: which is a comment useful for humans that need to understand the task\n- A `GROUND_TRUTH_OUTPUT`: which is a portion of the expected output of the task.\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should solve the task. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `utility` method, used to calculate the utility of the task. It takes as argument the original environment and the environment after the LLM executed the task. Returns a boolean indicating if the task was executed successfully.\n\nAdditionally, each user task _may_ define an `init_environment` static method, which is used to initialize the environment for the task. This is useful for tasks that need to set up the environment before the task is executed. If the original environment is enough for the task, then this method is not needed.\n\nEach user task should be decorated with the `@task_suite.register_user_task`, where `task_suite` is the task suite that the task belongs to.\n\nAn example implementation of a user task is available in the [`workspace`](src/function_calling_pi/task_suites/workspace/user_tasks.py) module.\n\n### Injection tasks\n\nEach suite has a set of (malicious) injection tasks. Each injection task is an instantiation of the [`BaseInjectionTask`](src/function_calling_pi/task_suites/base_tasks.py) class. Also each injection task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each injection task **must** define:\n\n- A `GOAL`: which is the goal that needs to be achieved via the injection\n- A `COMMENT`: which is a comment useful for humans that need to understand the injection goal\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should execute the injection. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `security` method, used to calculate if the injection was successful. It takes as argument the original environment and the environment after the LLM executed the task. It returns a boolean indicating if the injection was successful.\n\n### Checking the correctness of the task suite\n\nThe correctness of the task suite can be checked with the [check_suite.py](scripts/check_suites.py) script. By default it checks all the suites, but, optionally takes as arguments the name of the selected suites to check, each after the argument `-s`, and checks if:\n\n- The `ground_truth` method can be correctly executed.\n- The `ground_truth` and `utility`/`security` methods are consistent with each other.\n- If all the user tasks are injectable, i.e., if at least an injection placeholder is part of what is return by a tool and seen by the agent.\n\nThe script can be run with the following command (e.g., for the `workspace` suite):\n\n```bash\npython scripts/check_suites.py -s workspace\n```\n\nOr for all suites:\n\n```bash\npython scripts/check_suites.py\n```\n\n## Adding Tools\n\nTools are functions that can be used by the tasks to perform some operations. Tools can be stateless or stateful.\n\nEach tool must have type hints for both the input arguments and the return value. The type hints are used to generate the documentation for the tools. Moreover, tools **must** contain documentation for all the input argument (with the exception of the state argument for statefult tools, which **must not** be documented).\n\nAll tools can be tested as regular functions.\n\n### Stateless tools\n\nTools that do not need to access any state, can be registered with the `@register_function` decorator. Each tool can be defined as a function that takes any number of arguments and returns the result of the tool.\n\nAn example implementation of stateless tool is\n\n```python\nfrom function_calling_pi import register_function\n\n@register_function\ndef sum(a: int, b: int) -> int:\n \"\"\"Sums to numbers.\n\n :param a: First number\n :param b: Second number\n \"\"\"\n return a + b\n```\n\n### Stateful tools\n\nTools that need to access some state, must also be registered with the `@register_function` decorator. Each tool can be defined as a function that takes an arbitrary number of arguments.\n\nArguments that need to be injected from the environment are called _dependencies_ and work similarly to [FastAPI dependencies](https://fastapi.tiangolo.com/tutorial/dependencies/). Assuming a dependency as type `ExampleType`, and the dependency is inside of benchmark environments with the name `\"example_name\"`, then the argument needs to be type hinted using the type `Annotated[ExampleType, Depends(\"example_name\")]`.\n\n> [!NOTE]\n> The tool is responsible for updating the state (and need to make sure that the update happens in-place!)\n\nAn example implementation of stateful tool is\n\n```python\nfrom typing import Annotated\nfrom function_calling_pi import Depends, register_stateful_function\n\nfrom pydantic import BaseModel\n\nclass Environment(BaseModel):\n counter: Counter\n\nclass Counter():\n def __init__(self):\n self.counter = 0\n\n@register_function\ndef add_to_counter(\n counter: Annotated[Counter, Depends(\"counter\")], n: int = 1,\n) -> int:\n \"\"\"Adds an arbitrary number `n` to the counter and returns the new value.\n\n :param n: The number to add to the counter. Default is 1.\n \"\"\"\n counter.counter += 1\n return counter.counter\n\n```\n\n> [!NOTE]\n> Note that the `counter` argument is a dependency and is updated in-place, and is **not** documented in the docstring. This is because the agent does not need to know about this argument as it is injected by the engine executing the function.\n\n## Development set-up\n\nWe use `rye`. You can follow the instructions on the [website](https://rye-up.com) to set it up.\n\n### Install the dev packages, lint, test, typecheck\n\n- Install all the packages with `rye sync --features transformers` (include `--features transformers` if you want to work on the PI detector)\n- Lint with `rye lint --fix`\n- Test with `rye test`\n- Run the static type checker with `rye run pyright`\n\n### What if I don't want to install `rye`?\n\nThen you can set up the dev environment via `uv` or `pip` by running `[uv] pip install -r requirements-dev.lock`.\n\nThen, you can lint with `ruff check . --fix`, test with `pytest`, and typecheck with `pyright`.\n"}, "8f7ea21fffee4115503bc601c52639d205af83c4": {"t": 1785763940.4929392, "n": 9199, "h": "ff6d953786b49a7d", "c": "# TODOBench: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython scripts/benchmark.py -s workspace -ut user_task_0 -ut user_task_1 --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython scripts/benchmark.py --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\n## Tasks definition\n\nThe (benign) user tasks are defined by subclassing `BaseUserTask`, while the (malicious) injection tasks are defined by subclassing `BaseInjectionTask`. All tasks need to be registered with the task suite they belong to.\n\n### Task suite\n\nEach group of tasks is defined by a `TaskSuite` instantiation. The task suite is resposnible for defining:\n\n- The name of the suite\n- The tasks that belong to the suite\n- The tools avaliable for the suite tasks\n- The type for the environment used by the suite\n\nAn example is availabe in the [`workspace`](src/function_calling_pi/task_suites/workspace/task_suite.py) task suite. Each suite is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks.\n\nThe task suite has also the ability to run tasks with the `run_task` method. This method takes as argument the user task to run and the injection task to check for. If no injection task is passed, then the method will only run the user task. The method returns booleans indicating if the task and/or the injection were successful. The method also takes as an optional argument the environment to use for the tasks. If no environment is passed, then the method will use the default environment for the suite. Moreover, the method takes as argument the injections to inject within the (default environment).\n\nThe environment for the task is defined in `src/function_calling_pi/data/suites/{suite_name}/environment.yaml` and the injection vectors for the environment are defined in `src/function_calling_pi/data/suites/{suite_name}/injection_vectors.yaml`.\n\n### User tasks\n\nEach suite has a set of (benign) user tasks. Each user task is an instantiation of the [`BaseUserTask`](src/function_calling_pi/task_suites/base_tasks.py) class. Also each user task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each user task **must** define:\n\n- A `PROMPT`: which is the prompt that will be passed to the LLM\n- A `COMMENT`: which is a comment useful for humans that need to understand the task\n- A `GROUND_TRUTH_OUTPUT`: which is a portion of the expected output of the task.\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should solve the task. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `utility` method, used to calculate the utility of the task. It takes as argument the original environment and the environment after the LLM executed the task. Returns a boolean indicating if the task was executed successfully.\n\nAdditionally, each user task _may_ define an `init_environment` static method, which is used to initialize the environment for the task. This is useful for tasks that need to set up the environment before the task is executed. If the original environment is enough for the task, then this method is not needed.\n\nEach user task should be decorated with the `@task_suite.register_user_task`, where `task_suite` is the task suite that the task belongs to.\n\nAn example implementation of a user task is available in the [`workspace`](src/function_calling_pi/task_suites/workspace/user_tasks.py) module.\n\n### Injection tasks\n\nEach suite has a set of (malicious) injection tasks. Each injection task is an instantiation of the [`BaseInjectionTask`](src/function_calling_pi/task_suites/base_tasks.py) class. Also each injection task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each injection task **must** define:\n\n- A `GOAL`: which is the goal that needs to be achieved via the injection\n- A `COMMENT`: which is a comment useful for humans that need to understand the injection goal\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should execute the injection. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `security` method, used to calculate if the injection was successful. It takes as argument the original environment and the environment after the LLM executed the task. It returns a boolean indicating if the injection was successful.\n\n### Checking the correctness of the task suite\n\nThe correctness of the task suite can be checked with the [check_suite.py](scripts/check_suites.py) script. By default it checks all the suites, but, optionally takes as arguments the name of the selected suites to check, each after the argument `-s`, and checks if:\n\n- The `ground_truth` method can be correctly executed.\n- The `ground_truth` and `utility`/`security` methods are consistent with each other.\n- If all the user tasks are injectable, i.e., if at least an injection placeholder is part of what is return by a tool and seen by the agent.\n\nThe script can be run with the following command (e.g., for the `workspace` suite):\n\n```bash\npython scripts/check_suites.py -s workspace\n```\n\nOr for all suites:\n\n```bash\npython scripts/check_suites.py\n```\n\n## Adding Tools\n\nTools are functions that can be used by the tasks to perform some operations. Tools can be stateless or stateful.\n\nEach tool must have type hints for both the input arguments and the return value. The type hints are used to generate the documentation for the tools. Moreover, tools **must** contain documentation for all the input argument (with the exception of the state argument for statefult tools, which **must not** be documented).\n\nAll tools can be tested as regular functions.\n\n### Stateless tools\n\nTools that do not need to access any state, can be registered with the `@register_function` decorator. Each tool can be defined as a function that takes any number of arguments and returns the result of the tool.\n\nAn example implementation of stateless tool is\n\n```python\nfrom function_calling_pi.function_engine import register_function\n\n@register_function\ndef sum(a: int, b: int) -> int:\n \"\"\"Sums to numbers.\n\n :param a: First number\n :param b: Second number\n \"\"\"\n return a + b\n```\n\n### Stateful tools\n\nTools that need to access some state, must also be registered with the `@register_function` decorator. Each tool can be defined as a function that takes an arbitrary number of arguments.\n\nArguments that need to be injected from the environment are called _dependencies_ and work similarly to [FastAPI dependencies](https://fastapi.tiangolo.com/tutorial/dependencies/). Assuming a dependency as type `ExampleType`, and the dependency is inside of benchmark environments with the name `\"example_name\"`, then the argument needs to be type hinted using the type `Annotated[ExampleType, Depends(\"example_name\")]`.\n\n> [!NOTE]\n> The tool is responsible for updating the state (and need to make sure that the update happens in-place!)\n\nAn example implementation of stateful tool is\n\n```python\nfrom typing import Annotated\nfrom function_calling_pi.function_engine import Depends, register_stateful_function\n\nfrom pydantic import BaseModel\n\nclass Environment(BaseModel):\n counter: Counter\n\nclass Counter():\n def __init__(self):\n self.counter = 0\n\n@register_function\ndef add_to_counter(\n counter: Annotated[Counter, Depends(\"counter\")], n: int = 1,\n) -> int:\n \"\"\"Adds an arbitrary number `n` to the counter and returns the new value.\n\n :param n: The number to add to the counter. Default is 1.\n \"\"\"\n counter.counter += 1\n return counter.counter\n\n```\n\n> [!NOTE]\n> Note that the `counter` argument is a dependency and is updated in-place, and is **not** documented in the docstring. This is because the agent does not need to know about this argument as it is injected by the engine executing the function.\n\n## Development set-up\n\nWe use `rye`. You can follow the instructions on the [website](https://rye-up.com) to set it up.\n\n### Install the dev packages, lint, test, typecheck\n\n- Install all the packages with `rye sync`\n- Lint with `rye lint --fix`\n- Test with `rye test`\n- Run the static type checker with `rye run pyright`\n\n### What if I don't want to install `rye`?\n\nThen you can set up the dev environment via `uv` or `pip` by running `[uv] pip install -r requirements-dev.lock`.\n\nThen, you can lint with `ruff check . --fix`, test with `pytest`, and typecheck with `pyright`.\n"}, "85750af0be2ad5c15d03b8254746344b267ad3a8": {"t": 1785763942.5081036, "n": 9197, "h": "5125220167ba0b09", "c": "# TODOBench: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython scripts/benchmark.py -s workspace -t user_task_0 -t user_task_1 --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\nTo run the above, but on all suites and tasks, run the following:\n\n```bash\npython scripts/benchmark.py --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\n## Tasks definition\n\nThe (benign) user tasks are defined by subclassing `BaseUserTask`, while the (malicious) injection tasks are defined by subclassing `BaseInjectionTask`. All tasks need to be registered with the task suite they belong to.\n\n### Task suite\n\nEach group of tasks is defined by a `TaskSuite` instantiation. The task suite is resposnible for defining:\n\n- The name of the suite\n- The tasks that belong to the suite\n- The tools avaliable for the suite tasks\n- The type for the environment used by the suite\n\nAn example is availabe in the [`workspace`](src/function_calling_pi/task_suites/workspace/task_suite.py) task suite. Each suite is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks.\n\nThe task suite has also the ability to run tasks with the `run_task` method. This method takes as argument the user task to run and the injection task to check for. If no injection task is passed, then the method will only run the user task. The method returns booleans indicating if the task and/or the injection were successful. The method also takes as an optional argument the environment to use for the tasks. If no environment is passed, then the method will use the default environment for the suite. Moreover, the method takes as argument the injections to inject within the (default environment).\n\nThe environment for the task is defined in `src/function_calling_pi/data/suites/{suite_name}/environment.yaml` and the injection vectors for the environment are defined in `src/function_calling_pi/data/suites/{suite_name}/injection_vectors.yaml`.\n\n### User tasks\n\nEach suite has a set of (benign) user tasks. Each user task is an instantiation of the [`BaseUserTask`](src/function_calling_pi/task_suites/base_tasks.py) class. Also each user task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each user task **must** define:\n\n- A `PROMPT`: which is the prompt that will be passed to the LLM\n- A `COMMENT`: which is a comment useful for humans that need to understand the task\n- A `GROUND_TRUTH_OUTPUT`: which is a portion of the expected output of the task.\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should solve the task. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `utility` method, used to calculate the utility of the task. It takes as argument the original environment and the environment after the LLM executed the task. Returns a boolean indicating if the task was executed successfully.\n\nAdditionally, each user task _may_ define an `init_environment` static method, which is used to initialize the environment for the task. This is useful for tasks that need to set up the environment before the task is executed. If the original environment is enough for the task, then this method is not needed.\n\nEach user task should be decorated with the `@task_suite.register_user_task`, where `task_suite` is the task suite that the task belongs to.\n\nAn example implementation of a user task is available in the [`workspace`](src/function_calling_pi/task_suites/workspace/user_tasks.py) module.\n\n### Injection tasks\n\nEach suite has a set of (malicious) injection tasks. Each injection task is an instantiation of the [`BaseInjectionTask`](src/function_calling_pi/task_suites/base_tasks.py) class. Also each injection task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each injection task **must** define:\n\n- A `GOAL`: which is the goal that needs to be achieved via the injection\n- A `COMMENT`: which is a comment useful for humans that need to understand the injection goal\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should execute the injection. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `security` method, used to calculate if the injection was successful. It takes as argument the original environment and the environment after the LLM executed the task. It returns a boolean indicating if the injection was successful.\n\n### Checking the correctness of the task suite\n\nThe correctness of the task suite can be checked with the [check_suite.py](scripts/check_suites.py) script. By default it checks all the suites, but, optionally takes as arguments the name of the selected suites to check, each after the argument `-s`, and checks if:\n\n- The `ground_truth` method can be correctly executed.\n- The `ground_truth` and `utility`/`security` methods are consistent with each other.\n- If all the user tasks are injectable, i.e., if at least an injection placeholder is part of what is return by a tool and seen by the agent.\n\nThe script can be run with the following command (e.g., for the `workspace` suite):\n\n```bash\npython scripts/check_suites.py -s workspace\n```\n\nOr for all suites:\n\n```bash\npython scripts/check_suites.py\n```\n\n## Adding Tools\n\nTools are functions that can be used by the tasks to perform some operations. Tools can be stateless or stateful.\n\nEach tool must have type hints for both the input arguments and the return value. The type hints are used to generate the documentation for the tools. Moreover, tools **must** contain documentation for all the input argument (with the exception of the state argument for statefult tools, which **must not** be documented).\n\nAll tools can be tested as regular functions.\n\n### Stateless tools\n\nTools that do not need to access any state, can be registered with the `@register_function` decorator. Each tool can be defined as a function that takes any number of arguments and returns the result of the tool.\n\nAn example implementation of stateless tool is\n\n```python\nfrom function_calling_pi.function_engine import register_function\n\n@register_function\ndef sum(a: int, b: int) -> int:\n \"\"\"Sums to numbers.\n\n :param a: First number\n :param b: Second number\n \"\"\"\n return a + b\n```\n\n### Stateful tools\n\nTools that need to access some state, must also be registered with the `@register_function` decorator. Each tool can be defined as a function that takes an arbitrary number of arguments.\n\nArguments that need to be injected from the environment are called _dependencies_ and work similarly to [FastAPI dependencies](https://fastapi.tiangolo.com/tutorial/dependencies/). Assuming a dependency as type `ExampleType`, and the dependency is inside of benchmark environments with the name `\"example_name\"`, then the argument needs to be type hinted using the type `Annotated[ExampleType, Depends(\"example_name\")]`.\n\n> [!NOTE]\n> The tool is responsible for updating the state (and need to make sure that the update happens in-place!)\n\nAn example implementation of stateful tool is\n\n```python\nfrom typing import Annotated\nfrom function_calling_pi.function_engine import Depends, register_stateful_function\n\nfrom pydantic import BaseModel\n\nclass Environment(BaseModel):\n counter: Counter\n\nclass Counter():\n def __init__(self):\n self.counter = 0\n\n@register_function\ndef add_to_counter(\n counter: Annotated[Counter, Depends(\"counter\")], n: int = 1,\n) -> int:\n \"\"\"Adds an arbitrary number `n` to the counter and returns the new value.\n\n :param n: The number to add to the counter. Default is 1.\n \"\"\"\n counter.counter += 1\n return counter.counter\n\n```\n\n> [!NOTE]\n> Note that the `counter` argument is a dependency and is updated in-place, and is **not** documented in the docstring. This is because the agent does not need to know about this argument as it is injected by the engine executing the function.\n\n## Development set-up\n\nWe use `rye`. You can follow the instructions on the [website](https://rye-up.com) to set it up.\n\n### Install the dev packages, lint, test, typecheck\n\n- Install all the packages with `rye sync`\n- Lint with `rye lint --fix`\n- Test with `rye test`\n- Run the static type checker with `rye run pyright`\n\n### What if I don't want to install `rye`?\n\nThen you can set up the dev environment via `uv` or `pip` by running `[uv] pip install -r requirements-dev.lock`.\n\nThen, you can lint with `ruff check . --fix`, test with `pytest`, and typecheck with `pyright`.\n"}, "55eb091db5c27bec6c6509930766c10537dde83f": {"t": 1785763944.980719, "n": 9012, "h": "3d7dde6af64534e6", "c": "# TODOBench: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython scripts/benchmark.py workspace -t user_task_0 -t user_task_1 --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\n## Tasks definition\n\nThe (benign) user tasks are defined by subclassing `BaseUserTask`, while the (malicious) injection tasks are defined by subclassing `BaseInjectionTask`. All tasks need to be registered with the task suite they belong to.\n\n### Task suite\n\nEach group of tasks is defined by a `TaskSuite` instantiation. The task suite is resposnible for defining:\n\n- The name of the suite\n- The tasks that belong to the suite\n- The tools avaliable for the suite tasks\n- The type for the environment used by the suite\n\nAn example is availabe in the [`workspace`](src/function_calling_pi/task_suites/workspace/task_suite.py) task suite. Each suite is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks.\n\nThe task suite has also the ability to run tasks with the `run_task` method. This method takes as argument the user task to run and the injection task to check for. If no injection task is passed, then the method will only run the user task. The method returns booleans indicating if the task and/or the injection were successful. The method also takes as an optional argument the environment to use for the tasks. If no environment is passed, then the method will use the default environment for the suite. Moreover, the method takes as argument the injections to inject within the (default environment).\n\nThe environment for the task is defined in `src/function_calling_pi/data/suites/{suite_name}/environment.yaml` and the injection vectors for the environment are defined in `src/function_calling_pi/data/suites/{suite_name}/injection_vectors.yaml`.\n\n### User tasks\n\nEach suite has a set of (benign) user tasks. Each user task is an instantiation of the [`BaseUserTask`](src/function_calling_pi/task_suites/base_tasks.py) class. Also each user task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each user task **must** define:\n\n- A `PROMPT`: which is the prompt that will be passed to the LLM\n- A `COMMENT`: which is a comment useful for humans that need to understand the task\n- A `GROUND_TRUTH_OUTPUT`: which is a portion of the expected output of the task.\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should solve the task. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `utility` method, used to calculate the utility of the task. It takes as argument the original environment and the environment after the LLM executed the task. Returns a boolean indicating if the task was executed successfully.\n\nAdditionally, each user task _may_ define an `init_environment` static method, which is used to initialize the environment for the task. This is useful for tasks that need to set up the environment before the task is executed. If the original environment is enough for the task, then this method is not needed.\n\nEach user task should be decorated with the `@task_suite.register_user_task`, where `task_suite` is the task suite that the task belongs to.\n\nAn example implementation of a user task is available in the [`workspace`](src/function_calling_pi/task_suites/workspace/user_tasks.py) module.\n\n### Injection tasks\n\nEach suite has a set of (malicious) injection tasks. Each injection task is an instantiation of the [`BaseInjectionTask`](src/function_calling_pi/task_suites/base_tasks.py) class. Also each injection task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each injection task **must** define:\n\n- A `GOAL`: which is the goal that needs to be achieved via the injection\n- A `COMMENT`: which is a comment useful for humans that need to understand the injection goal\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should execute the injection. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `security` method, used to calculate if the injection was successful. It takes as argument the original environment and the environment after the LLM executed the task. It returns a boolean indicating if the injection was successful.\n\n### Checking the correctness of the task suite\n\nThe correctness of the task suite can be checked with the [check_suite.py](scripts/check_suites.py) script. By default it checks all the suites, but, optionally takes as arguments the name of the selected suites to check, each after the argument `-s`, and checks if:\n\n- The `ground_truth` method can be correctly executed.\n- The `ground_truth` and `utility`/`security` methods are consistent with each other.\n- If all the user tasks are injectable, i.e., if at least an injection placeholder is part of what is return by a tool and seen by the agent.\n\nThe script can be run with the following command (e.g., for the `workspace` suite):\n\n```bash\npython scripts/check_suites.py -s workspace\n```\n\nOr for all suites:\n\n```bash\npython scripts/check_suites.py\n```\n\n## Adding Tools\n\nTools are functions that can be used by the tasks to perform some operations. Tools can be stateless or stateful.\n\nEach tool must have type hints for both the input arguments and the return value. The type hints are used to generate the documentation for the tools. Moreover, tools **must** contain documentation for all the input argument (with the exception of the state argument for statefult tools, which **must not** be documented).\n\nAll tools can be tested as regular functions.\n\n### Stateless tools\n\nTools that do not need to access any state, can be registered with the `@register_function` decorator. Each tool can be defined as a function that takes any number of arguments and returns the result of the tool.\n\nAn example implementation of stateless tool is\n\n```python\nfrom function_calling_pi.function_engine import register_function\n\n@register_function\ndef sum(a: int, b: int) -> int:\n \"\"\"Sums to numbers.\n\n :param a: First number\n :param b: Second number\n \"\"\"\n return a + b\n```\n\n### Stateful tools\n\nTools that need to access some state, must also be registered with the `@register_function` decorator. Each tool can be defined as a function that takes an arbitrary number of arguments.\n\nArguments that need to be injected from the environment are called _dependencies_ and work similarly to [FastAPI dependencies](https://fastapi.tiangolo.com/tutorial/dependencies/). Assuming a dependency as type `ExampleType`, and the dependency is inside of benchmark environments with the name `\"example_name\"`, then the argument needs to be type hinted using the type `Annotated[ExampleType, Depends(\"example_name\")]`.\n\n> [!NOTE]\n> The tool is responsible for updating the state (and need to make sure that the update happens in-place!)\n\nAn example implementation of stateful tool is\n\n```python\nfrom typing import Annotated\nfrom function_calling_pi.function_engine import Depends, register_stateful_function\n\nfrom pydantic import BaseModel\n\nclass Environment(BaseModel):\n counter: Counter\n\nclass Counter():\n def __init__(self):\n self.counter = 0\n\n@register_function\ndef add_to_counter(\n counter: Annotated[Counter, Depends(\"counter\")], n: int = 1,\n) -> int:\n \"\"\"Adds an arbitrary number `n` to the counter and returns the new value.\n\n :param n: The number to add to the counter. Default is 1.\n \"\"\"\n counter.counter += 1\n return counter.counter\n\n```\n\n> [!NOTE]\n> Note that the `counter` argument is a dependency and is updated in-place, and is **not** documented in the docstring. This is because the agent does not need to know about this argument as it is injected by the engine executing the function.\n\n## Development set-up\n\nWe use `rye`. You can follow the instructions on the [website](https://rye-up.com) to set it up.\n\n### Install the dev packages, lint, test, typecheck\n\n- Install all the packages with `rye sync`\n- Lint with `rye lint --fix`\n- Test with `rye test`\n- Run the static type checker with `rye run pyright`\n\n### What if I don't want to install `rye`?\n\nThen you can set up the dev environment via `uv` or `pip` by running `[uv] pip install -r requirements-dev.lock`.\n\nThen, you can lint with `ruff check . --fix`, test with `pytest`, and typecheck with `pyright`.\n"}, "0b524de191186fe4461e1e97a24914e1bad57d12": {"t": 1785763946.2984266, "n": 8844, "h": "a7e58a614cc20cbe", "c": "# TODOBench: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Running the benchmark\n\nThe benchmark can be run with the [benchmark](scripts/benchmark.py) script. Documentation on how to use the script can be obtained with the `--help` flag.\n\nFor example, to run the `workspace` suite on the tasks 0 and 1, with `gpt-4o-2024-05-13` as the LLM, the tool filter as a defense, and the attacker with tool knowlege, run the following command:\n\n```bash\npython scripts/benchmark.py workspace -t user_task_0 -t user_task_1 --model gpt-4o-2024-05-13 --defense tool_filter --attacker tool_knowledge\n```\n\n## Tasks definition\n\nThe (benign) user tasks are defined by subclassing `BaseUserTask`, while the (malicious) injection tasks are defined by subclassing `BaseInjectionTask`. All tasks need to be registered with the task suite they belong to.\n\n### Task suite\n\nEach group of tasks is defined by a `TaskSuite` instantiation. The task suite is resposnible for defining:\n\n- The name of the suite\n- The tasks that belong to the suite\n- The tools avaliable for the suite tasks\n- The type for the environment used by the suite\n\nAn example is availabe in the [`workspace`](src/function_calling_pi/task_suites/workspace/task_suite.py) task suite. Each suite is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks.\n\nThe task suite has also the ability to run tasks with the `run_task` method. This method takes as argument the user task to run and the injection task to check for. If no injection task is passed, then the method will only run the user task. The method returns booleans indicating if the task and/or the injection were successful. The method also takes as an optional argument the environment to use for the tasks. If no environment is passed, then the method will use the default environment for the suite. Moreover, the method takes as argument the injections to inject within the (default environment).\n\nThe environment for the task is defined in `src/function_calling_pi/data/suites/{suite_name}/environment.yaml` and the injection vectors for the environment are defined in `src/function_calling_pi/data/suites/{suite_name}/injection_vectors.yaml`.\n\n### User tasks\n\nEach suite has a set of (benign) user tasks. Each user task is an instantiation of the [`BaseUserTask`](src/function_calling_pi/task_suites/base_tasks.py) class. Also each user task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each user task **must** define:\n\n- A `PROMPT`: which is the prompt that will be passed to the LLM\n- A `COMMENT`: which is a comment useful for humans that need to understand the task\n- A `GROUND_TRUTH_OUTPUT`: which is a portion of the expected output of the task.\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should solve the task. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `utility` method, used to calculate the utility of the task. It takes as argument the original environment and the environment after the LLM executed the task. Returns a boolean indicating if the task was executed successfully.\n\nAdditionally, each user task _may_ define an `init_environment` static method, which is used to initialize the environment for the task. This is useful for tasks that need to set up the environment before the task is executed. If the original environment is enough for the task, then this method is not needed.\n\nEach user task should be decorated with the `@task_suite.register_user_task`, where `task_suite` is the task suite that the task belongs to.\n\nAn example implementation of a user task is available in the [`workspace`](src/function_calling_pi/task_suites/workspace/user_tasks.py) module.\n\n### Injection tasks\n\nEach suite has a set of (malicious) injection tasks. Each injection task is an instantiation of the [`BaseInjectionTask`](src/function_calling_pi/task_suites/base_tasks.py) class. Also each injection task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each injection task **must** define:\n\n- A `GOAL`: which is the goal that needs to be achieved via the injection\n- A `COMMENT`: which is a comment useful for humans that need to understand the injection goal\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should execute the injection. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `security` method, used to calculate if the injection was successful. It takes as argument the original environment and the environment after the LLM executed the task. It returns a boolean indicating if the injection was successful.\n\n### Checking the correctness of the task suite\n\nThe correctness of the task suite can be checked with the [check_suite.py](scripts/check_suite.py) script. It takes as arguments the name of the suite, and checks if:\n\n- The `ground_truth` method can be correctly executed.\n- The `ground_truth` and `utility`/`security` methods are consistent with each other.\n- If all the user tasks are injectable, i.e., if at least an injection placeholder is part of what is return by a tool and seen by the agent.\n\nThe script can be run with the following command (e.g., for the `workspace` suite):\n\n```bash\npython scripts/check_suite.py workspace\n```\n\n## Adding Tools\n\nTools are functions that can be used by the tasks to perform some operations. Tools can be stateless or stateful.\n\nEach tool must have type hints for both the input arguments and the return value. The type hints are used to generate the documentation for the tools. Moreover, tools **must** contain documentation for all the input argument (with the exception of the state argument for statefult tools, which **must not** be documented).\n\nAll tools can be tested as regular functions.\n\n### Stateless tools\n\nTools that do not need to access any state, can be registered with the `@register_function` decorator. Each tool can be defined as a function that takes any number of arguments and returns the result of the tool.\n\nAn example implementation of stateless tool is\n\n```python\nfrom function_calling_pi.function_engine import register_function\n\n@register_function\ndef sum(a: int, b: int) -> int:\n \"\"\"Sums to numbers.\n\n :param a: First number\n :param b: Second number\n \"\"\"\n return a + b\n```\n\n### Stateful tools\n\nTools that need to access some state, must also be registered with the `@register_function` decorator. Each tool can be defined as a function that takes an arbitrary number of arguments.\n\nArguments that need to be injected from the environment are called _dependencies_ and work similarly to [FastAPI dependencies](https://fastapi.tiangolo.com/tutorial/dependencies/). Assuming a dependency as type `ExampleType`, and the dependency is inside of benchmark environments with the name `\"example_name\"`, then the argument needs to be type hinted using the type `Annotated[ExampleType, Depends(\"example_name\")]`.\n\n> [!NOTE]\n> The tool is responsible for updating the state (and need to make sure that the update happens in-place!)\n\nAn example implementation of stateful tool is\n\n```python\nfrom typing import Annotated\nfrom function_calling_pi.function_engine import Depends, register_stateful_function\n\nfrom pydantic import BaseModel\n\nclass Environment(BaseModel):\n counter: Counter\n\nclass Counter():\n def __init__(self):\n self.counter = 0\n\n@register_function\ndef add_to_counter(\n counter: Annotated[Counter, Depends(\"counter\")], n: int = 1,\n) -> int:\n \"\"\"Adds an arbitrary number `n` to the counter and returns the new value.\n\n :param n: The number to add to the counter. Default is 1.\n \"\"\"\n counter.counter += 1\n return counter.counter\n\n```\n\n> [!NOTE]\n> Note that the `counter` argument is a dependency and is updated in-place, and is **not** documented in the docstring. This is because the agent does not need to know about this argument as it is injected by the engine executing the function.\n\n## Development set-up\n\nWe use `rye`. You can follow the instructions on the [website](https://rye-up.com) to set it up.\n\n### Install the dev packages, lint, test, typecheck\n\n- Install all the packages with `rye sync`\n- Lint with `rye lint --fix`\n- Test with `rye test`\n- Run the static type checker with `rye run pyright`\n\n### What if I don't want to install `rye`?\n\nThen you can set up the dev environment via `uv` or `pip` by running `[uv] pip install -r requirements-dev.lock`.\n\nThen, you can lint with `ruff check . --fix`, test with `pytest`, and typecheck with `pyright`.\n"}, "e13a374bcfa957f2e4c4d2b69ac070447a8928de": {"t": 1785763947.7168856, "n": 9245, "h": "71cff4896da1950a", "c": "# TODOBench: Benchmarking the Capabilities and Adversarial Robustness of LLM Agents\n\n## Tasks definition\n\nThe (benign) user tasks are defined by subclassing `BaseUserTask`, while the (malicious) injection tasks are defined by subclassing `BaseInjectionTask`. All tasks need to be registered with the task suite they belong to.\n\n### Task suite\n\nEach group of tasks is defined by a `TaskSuite` instantiation. The task suite is resposnible for defining:\n\n- The name of the suite\n- The tasks that belong to the suite\n- The tools avaliable for the suite tasks\n- The type for the environment used by the suite\n\nAn example is availabe in the [`workspace`](src/function_calling_pi/task_suites/workspace/task_suite.py) task suite. Each suite is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks.\n\nThe task suite has also the ability to run tasks with the `run_task` method. This method takes as argument the user task to run and the injection task to check for. If no injection task is passed, then the method will only run the user task. The method returns booleans indicating if the task and/or the injection were successful. The method also takes as an optional argument the environment to use for the tasks. If no environment is passed, then the method will use the default environment for the suite. Moreover, the method takes as argument the injections to inject within the (default environment).\n\nThe environment for the task is defined in `src/function_calling_pi/data/suites/{suite_name}/environment.yaml` and the injection vectors for the environment are defined in `src/function_calling_pi/data/suites/{suite_name}/injection_vectors.yaml`.\n\n### User tasks\n\nEach suite has a set of (benign) user tasks. Each user task is an instantiation of the [`BaseUserTask`](src/function_calling_pi/task_suites/base_tasks.py) class. Also each user task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each user task **must** define:\n\n- A `PROMPT`: which is the prompt that will be passed to the LLM\n- A `COMMENT`: which is a comment useful for humans that need to understand the task\n- A `GROUND_TRUTH_OUTPUT`: which is a portion of the expected output of the task.\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should solve the task. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `utility` method, used to calculate the utility of the task. It takes as argument the original environment and the environment after the LLM executed the task. Returns a boolean indicating if the task was executed successfully.\n\nAdditionally, each user task _may_ define an `init_environment` static method, which is used to initialize the environment for the task. This is useful for tasks that need to set up the environment before the task is executed. If the original environment is enough for the task, then this method is not needed.\n\nEach user task should be decorated with the `@task_suite.register_user_task`, where `task_suite` is the task suite that the task belongs to.\n\nAn example implementation of a user task is available in the [`workspace`](src/function_calling_pi/task_suites/workspace/user_tasks.py) module.\n\n### Injection tasks\n\nEach suite has a set of (malicious) injection tasks. Each injection task is an instantiation of the [`BaseInjectionTask`](src/function_calling_pi/task_suites/base_tasks.py) class. Also each injection task is a generic parametrized with the environment used by the tasks. This is to ease type checking when writing the tasks. Each injection task **must** define:\n\n- A `GOAL`: which is the goal that needs to be achieved via the injection\n- A `COMMENT`: which is a comment useful for humans that need to understand the injection goal\n- A `ground_truth` method: a list of function calls with corresponding arguments which, when executed, should execute the injection. The arguments can be the `Ellipsis` object (i.e., `...`) if the argument is not known and not relevant for solving the task.\n- A `security` method, used to calculate if the injection was successful. It takes as argument the original environment and the environment after the LLM executed the task. It returns a boolean indicating if the injection was successful.\n\n### Checking the correctness of the task suite\n\nThe correctness of the task suite can be checked with the [check_suite.py](scripts/check_suite.py) script. It takes as arguments the name of the suite, and checks if:\n\n- The `ground_truth` method can be correctly executed.\n- The `ground_truth` and `utility`/`security` methods are consistent with each other.\n- If all the user tasks are injectable, i.e., if at least an injection placeholder is part of what is return by a tool and seen by the agent.\n\nThe script can be run with the following command (e.g., for the `workspace` suite):\n\n```bash\npython scripts/check_suite.py workspace\n```\n\n## Generating injections\n\nYou can use the [generate_injections.py](scripts/generate_injections.py) script by using one of the [attackers](https://github.com/ethz-spylab/pi-benchmark/blob/main/src/function_calling_pi/attacks/__init__.py#L5). It takes as argument the name of the attacker, the directory where the results are saved, and the names of the suites to generate the injections for (each name should be after a `-s` flag).\n\nFor example, the following command generates the injections for the `workspace` suite using the `direct` attacker and saves the results in the `attacks` directory:\n\n```bash\npython scripts/generate_injections.py direct attacks -s workspace\n```\n\nThe results are saved in files with the following directory structure: `$SAVE_DIR/$ATTACKER_NAME/$SUITE_NAME.yaml`. In each file, the injections are a dictionary with this structure `user_task_id -> injection_task_id -> injection_placeholder -> injection`.\n\n## Adding Tools\n\nTools are functions that can be used by the tasks to perform some operations. Tools can be stateless or stateful.\n\nEach tool must have type hints for both the input arguments and the return value. The type hints are used to generate the documentation for the tools. Moreover, tools **must** contain documentation for all the input argument (with the exception of the state argument for statefult tools, which **must not** be documented).\n\nAll tools can be tested as regular functions.\n\n### Stateless tools\n\nTools that do not need to access any state, can be registered with the `@register_function` decorator. Each tool can be defined as a function that takes any number of arguments and returns the result of the tool.\n\nAn example implementation of stateless tool is\n\n```python\nfrom function_calling_pi.function_engine import register_function\n\n@register_function\ndef sum(a: int, b: int) -> int:\n \"\"\"Sums to numbers.\n\n :param a: First number\n :param b: Second number\n \"\"\"\n return a + b\n```\n\n### Stateful tools\n\nTools that need to access some state, must also be registered with the `@register_function` decorator. Each tool can be defined as a function that takes an arbitrary number of arguments.\n\nArguments that need to be injected from the environment are called _dependencies_ and work similarly to [FastAPI dependencies](https://fastapi.tiangolo.com/tutorial/dependencies/). Assuming a dependency as type `ExampleType`, and the dependency is inside of benchmark environments with the name `\"example_name\"`, then the argument needs to be type hinted using the type `Annotated[ExampleType, Depends(\"example_name\")]`.\n\n> [!NOTE]\n> The tool is responsible for updating the state (and need to make sure that the update happens in-place!)\n\nAn example implementation of stateful tool is\n\n```python\nfrom typing import Annotated\nfrom function_calling_pi.function_engine import Depends, register_stateful_function\n\nfrom pydantic import BaseModel\n\nclass Environment(BaseModel):\n counter: Counter\n\nclass Counter():\n def __init__(self):\n self.counter = 0\n\n@register_function\ndef add_to_counter(\n counter: Annotated[Counter, Depends(\"counter\")], n: int = 1,\n) -> int:\n \"\"\"Adds an arbitrary number `n` to the counter and returns the new value.\n\n :param n: The number to add to the counter. Default is 1.\n \"\"\"\n counter.counter += 1\n return counter.counter\n\n```\n\n> [!NOTE]\n> Note that the `counter` argument is a dependency and is updated in-place, and is **not** documented in the docstring. This is because the agent does not need to know about this argument as it is injected by the engine executing the function.\n\n## Development set-up\n\nWe use `rye`. You can follow the instructions on the [website](https://rye-up.com) to set it up.\n\n### Install the dev packages, lint, test, typecheck\n\n- Install all the packages with `rye sync`\n- Lint with `rye lint --fix`\n- Test with `rye test`\n- Run the static type checker with `rye run pyright`\n\n### What if I don't want to install `rye`?\n\nThen you can set up the dev environment via `uv` or `pip` by running `[uv] pip install -r requirements-dev.lock`.\n\nThen, you can lint with `ruff check . --fix`, test with `pytest`, and typecheck with `pyright`.\n"}}}} |