🤖 Awesome Code Agents
Towards AI-Powered Software 3.0

A curated list of products, benchmarks, and research papers on autonomous code agents.
Beyond coding — they're redefining how software changes the world.

Website • X/Twitter • LinkedIn • Discord • Reddit • GitHub

Awesome Code Agents

*Photo Credit: [Gemini-Nano-Banana-Pro🍌](https://deepmind.google/models/gemini-image/pro/)*.

🔥 We are actively tracking the frontier research of code agents.
📚 Currently collected: 459 papers and products — (Last update: 2025-11-29)

🚀 Products & Tools¶

Leading agentic systems, frameworks, and platforms for automated software development.

Claude Code. Anthropic. 2025.
Cursor. Cursor. 2025.
Codex. OpenAI. 2025.
Lovable. Lovable. 2024.
Devin. Cognition AI. 2024.
Replit. Replit. 2025.
Bolt.new. StackBlitz. 2025.
GitHub Copilot. GitHub. 2025.
Grok Studio. xAI. 2025.
Stitch. Google. 2025.
v0. Vercel. 2025.
Windsurf. Windsurf (product now acquired by Cognition AI). 2025.
Spec Kit. GitHub. 2025.
Rork. Rork. 2025.
Clark. Superblocks. 2025.
Augment Code. Augment. 2025.
Mocha. Mocha. 2025.
Vitara. Vitara.AI. 2025.
a0.dev. a0.dev. 2025.
Canva AI Code Generator. Canva. 2025.
OpenHands. All Hands AI. 2024.
Aider. Aider AI. 2025.
Prometheus. EuniAI. 2025.
SWE-agent. Princeton University. 2024.
Open Lovable. Firecrawl. 2025.
PR-Agent & Qodo Merge. Qodo. 2024.
Serena. Oraios AI. 2025.
Zen MCP. Beehive Innovations. 2025.
Context7 MCP. Upstash. 2025.
Trae AI. ByteDance. 2025.
Cline. Cline. 2025.
Continue. Continue Dev. 2025.
Databutton. Databutton. 2025.
Base44. Base44. 2025.
Lingma SWE-GPT. Tongyi / Alibaba. 2025.
Agentless. University of Illinois Urbana-Champaign. 2024.
AutoCodeRover. National University of Singapore (product now acquired by Sonar). 2024.
OpenCode. SST. 2025.
Jules. Google. 2025.
Droids. Factory. 2025.
KAT-Coder. Kwaipilot / Kuaishou. 2025.
Kiro. AWS. 2025.
Essential. Nothing. 2025.
Anything. Anything. 2025.

📚 Papers¶

Explore foundational, recent, and influential works advancing the code agent research landscape.

🌍 Foundation Models¶

Large Language Models designed or extended for advanced software engineering capabilities.

CWM: An Open-Weights LLM for Research on Code Generation with World Models. FAIR CodeGen team, Jade Copet, Quentin Carbonneaux, Gal Cohen, Jonas Gehring, Jacob Kahn, Jannik Kossen, Felix Kreuk, Emily McMilin, Michel Meyer, et al. arXiv 2025/09.
Devstral: Fine-tuning Language Models for Coding Agent Applications. Abhinav Rastogi, Adam Yang, Albert Q. Jiang, Alexander H. Liu, Alexandre Sablayrolles, Amélie Héliou, Amélie Martin, Anmol Agarwal, Andy Ehrenberg, Andy Lo, et al. arXiv 2025/09.
Qwen3-Coder: Agentic Coding in the World. QwenTeam. 2025/07.
Kimi K2: Open Agentic Intelligence. Kimi Team: Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, Zhuofu Chen, et al. arXiv 2025/07.

🔧 Software General Engineering Agents¶

🛠 Issue Resolution¶

Automated bug fixing, patch generation, repair techniques.

Prometheus: Unified Knowledge Graphs for Issue Resolution in Multilingual Codebases. Zimin Chen, Yue Pan, Siyu Lu, Jiayi Xu, Claire Le Goues, Martin Monperrus, He Ye. arXiv 2025.
Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly? Chunqiu Steven Xia, Zhe Wang, Yan Yang, Yuxiang Wei, Lingming Zhang. arXiv 2025/11.
The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agents. Xingyao Wang, Simon Rosenberg, Juan Michelini, Calvin Smith, Hoang Tran, Engel Nyst, Rohit Malhotra, Xuhui Zhou, Valerie Chen, Robert Brennan, et al. arXiv 2025/11.
SWE-Compass: Towards Unified Evaluation of Agentic Coding Abilities for Large Language Models. Jingxuan Xu, Ken Deng, Weihao Li, Songwei Yu, Huaixi Tang, Haoyang Huang, Zhiyi Lai, Zizheng Zhan, Yanan Wu, Chenchen Zhang, et al. arXiv 2025/11.
SWE-Sharp-Bench: A Reproducible Benchmark for C# Software Engineering Tasks. Sanket Mhatre, Yasharth Bajpai, Sumit Gulwani, Emerson Murphy-Hill, Gustavo Soares. arXiv 2025/11.
U2F: Encouraging SWE-Agent to Seize Novelty without Losing Feasibility. Wencheng Ye, Yan Liu. arXiv 2025/11.
When “Correct” Is Not Safe: Can We Trust Functionally Correct Patches Generated by Code Agents? Yibo Peng, James Song, Lei Li, Xinyu Yang, Mihai Christodorescu, Ravi Mangal, Corina Pasareanu, Haizhong Zheng, Beidi Chen. arXiv 2025/10.
Enhancing repository-level software repair via repository-aware knowledge graphs. Boyang Yang, Jiadong Ren, Shunfu Jin, Yang Liu, Feng Liu, Bach Le, Haoye Tian. arXiv 2025/10.
SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks? Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Nitin Pasari, Chetan Rane, et al. arXiv 2025.
Is Your Automated Software Engineer Trustworthy? Noble Saji Mathews, Meiyappan Nagappan. arXiv 2025/06.
SWE-PolyBench: A multi-language benchmark for repository level evaluation of coding agents. Muhammad Shihab Rashid, Christian Bock, Yuan Zhuang, Alexander Buchholz, Tim Esler, Simon Valentin, Luca Franceschi, Martin Wistuba, Prabhu Teja Sivaprasad, Woo Jung Kim, et al. arXiv 2025.
Multi-SWE-bench: A Multilingual Benchmark for Issue Resolving. Daoguang Zan, Zhirong Huang, Wei Liu, Hanwu Chen, Linhao Zhang, Shulin Xin, Lu Chen, Qi Liu, Xiaojian Zhong, Aoyan Li, et al. arXiv 2025.
SWE-rebench: An Automated Pipeline for Task Collection and Decontaminated Evaluation of Software Engineering Agents. Ibragim Badertdinov, Alexander Golubev, Maksim Nekrashevich, Anton Shevtsov, Simon Karasik, Andrei Andriushchenko, Maria Trofimova, Daria Litvintseva, Boris Yangel. arXiv 2025.
Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scaling. Trae Research Team: Pengfei Gao, Zhao Tian, Xiangxin Meng, Xinchen Wang, Ruida Hu, Yuanan Xiao, Yizhou Liu, Zhao Zhang, Junjie Chen, Cuiyun Gao, et al. arXiv 2025.
ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. Siru Ouyang, Jun Yan, I-Hung Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long T. Le, Samira Daruki, Xiangru Tang, et al. arXiv 2025.
EXPEREPAIR: Dual-Memory Enhanced LLM-based Repository-Level Program Repair. Fangwen Mu, Junjie Wang, Lin Shi, Song Wang, Shoubin Li, Qing Wang. arXiv 2025.
SWE-Exp: Experience-Driven Software Issue Resolution. Silin Chen, Shaoxin Lin, Xiaodong Gu, Yuling Shi, Heng Lian, Longfei Yun, Dong Chen, Weiguo Sun, Lin Cao, Qianxiang Wang. arXiv 2025.
Thinking Longer, Not Larger: Enhancing Software Engineering Agents via Scaling Test-Time Compute. Yingwei Ma, Yongbin Li, Yihong Dong, Xue Jiang, Rongyu Cao, Jue Chen, Fei Huang, Binhua Li. arXiv 2025.
Satori-SWE: Evolutionary Test-Time Scaling for Sample-Efficient Software Engineering. Guangtao Zeng, Maohao Shen, Delin Chen, Zhenting Qi, Subhro Das, Dan Gutfreund, David Cox, Gregory Wornell, Wei Lu, Zhang-Wei Hong, et al. arXiv 2025/05.
DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversal. Vaibhav Aggarwal, Ojasv Kamal, Abhinav Japesh, Zhijing Jin, Bernhard Schölkopf. arXiv 2025/03.
debug-gym: A Text-Based Environment for Interactive Debugging. Xingdi Yuan, Morgane M Moss, Charbel El Feghali, Chinmay Singh, Darya Moldavskaya, Drew MacPhee, Lucas Caccia, Matheus Pereira, Minseon Kim, Alessandro Sordoni, et al. arXiv 2025.
R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents. Naman Jain, Jaskirat Singh, Manish Shetty, Liang Zheng, Koushik Sen, Ion Stoica. arXiv 2025.
HAFixAgent: History-Aware Automated Program Repair Agent. Yu Shi, Hao Li, Bram Adams, Ahmed E. Hassan. arXiv 2025.
SWE-Debate: Competitive Multi-Agent Debate for Software Issue Resolution. Han Li, Yuling Shi, Shaoxin Lin, Xiaodong Gu, Heng Lian, Xin Wang, Yantao Jia, Tao Huang, Qianxiang Wang. arXiv 2025.
SWE-Factory: Your Automated Factory for Issue Resolution Training Data and Evaluation Benchmarks. Lianghong Guo, Yanlin Wang, Caihua Li, Pengyu Yang, Jiachi Chen, Wei Tao, Yingtian Zou, Duyu Tang, Zibin Zheng. arXiv 2025.
Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents. Zonghan Yang, Shengjie Wang, Kelin Fu, Wenyang He, Weimin Xiong, Yibo Liu, Yibo Miao, Bofei Gao, Yejie Wang, Yingwei Ma, et al. arXiv 2025.
SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement. Antonis Antoniades, Albert Örwall, Kexun Zhang, Yuxi Xie, Anirudh Goyal, William Wang. arXiv 2025.
SEAlign: Alignment Training for Software Engineering Agent. Kechi Zhang, Huangzhao Zhang, Ge Li, Jinliang You, Jia Li, Yunfei Zhao, Zhi Jin. arXiv 2025.
Lingxi: Repository-Level Issue Resolution Framework Enhanced by Procedural Knowledge Guided Scaling. Xu Yang, Jiayuan Zhou, Michael Pacheco, Wenhan Zhu, Pengfei He, Shaowei Wang, Kui Liu, Ruiqi Pan. arXiv 2025.
Code Graph Model (CGM): A Graph-Integrated Large Language Model for Repository-Level Software Engineering Tasks. Hongyuan Tao, Ying Zhang, Zhenhao Tang, Hongen Peng, Xukun Zhu, Bingchang Liu, Yingguang Yang, Ziyin Zhang, Zhaogui Xu, Haipeng Zhang, et al. arXiv 2025.
SWE-Synth: Synthesizing Verifiable Bug-Fix Data to Enable Large Language Models in Resolving Real-World Bugs. Minh V.T. Pham, Huy N. Phan, Hoang N. Phan, Cuong Le Chi, Tien N. Nguyen, Nghi D. Q. Bui. arXiv 2025.
ComBench: Compilation Error Repair Benchmark Platform. Anonymous. 2025.
SWE-Bench-CL: Continual Learning for Coding Agents. Thomas Joshi, Shayan Chowdhury, Fatih Uysal. arXiv 2025.
A Self-Improving Coding Agent. Maxime Robeyns, Martin Szummer, Laurence Aitchison. arXiv 2025.
Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards. Jeff Da, Clinton Wang, Xiang Deng, Yuntao Ma, Nikhil Barhate, Sean Hendryx. arXiv 2025.
Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learning. Alexander Golubev, Maria Trofimova, Sergei Polezhaev, Ibragim Badertdinov, Maksim Nekrashevich, Anton Shevtsov, Simon Karasik, Sergey Abramov, Andrei Andriushchenko, Filipp Fisin, et al. arXiv 2025.
SWE-Mirror: Scaling Issue-Resolving Datasets by Mirroring Issues Across Repositories. Junhao Wang, Daoguang Zan, Shulin Xin, Siyao Liu, Yurong Wu, Kai Shen. arXiv 2025.
SWE-Effi: Re-Evaluating Software AI Agent System Effectiveness Under Resource Constraints. Zhiyu Fan, Kirill Vasilevski, Dayi Lin, Boyuan Chen, Yihao Chen, Zhiqing Zhong, Jie M. Zhang, Pinjia He, Ahmed E. Hassan. arXiv 2025.
Learn-by-interact: A Data-Centric Framework for Self-Adaptive Agents in Realistic Environments. Hongjin Su, Ruoxi Sun, Jinsung Yoon, Pengcheng Yin, Tao Yu, Sercan Ö. Arık. arXiv 2025.
Enhancing repository-level software repair via repository-aware knowledge graphs. Boyang Yang, Jiadong Ren, Shunfu Jin, Yang Liu, Feng Liu, Bach Le, Haoye Tian. arXiv 2025.
SemAgent: A Semantics Aware Program Repair Agent. Anvith Pabba, Alex Mathai, Anindya Chakraborty, Baishakhi Ray. arXiv 2025.
HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scale. Huy Nhat Phan, Tien N. Nguyen, Phong X. Nguyen, Nghi D. Q. Bui. arXiv 2025.
Saving SWE-Bench: A Benchmark Mutation Approach for Realistic Agent Evaluation. Spandan Garg, Ben Steenhoek, Yufan Huang. arXiv 2025.
RepoForge: Training a SOTA Fast-thinking SWE Agent with an End-to-End Data Curation Pipeline Synergizing SFT and RL at Scale. Zhilong Chen, Chengzong Zhao, Boyuan Chen, Dayi Lin, Yihao Chen, Arthur Leung, Gopi Krishnan Rajbahadur, Gustavo Oliva, Haoxiang Zhang, Aadi Bhatia, et al. arXiv 2025.
MCTS-Refined CoT: High-Quality Fine-Tuning Data for LLM-Based Repository Issue Resolution. Yibo Wang, Zhihao Peng, Ying Wang, Zhao Wei, Hai Yu, Zhiliang Zhu. arXiv 2025.
SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Tasks. Pavel Adamenko, Mikhail Ivanov, Aidar Valeev, Rodion Levichev, Pavel Zadorozhny, Ivan Lopatin, Dmitry Babayev, Alena Fenogenova, Valentin Malykh. arXiv 2025.
Auto-SWE-Bench: A Framework for the Scalable Generation of Software Engineering Benchmark from Open-Source Repositories. Anonymous Authors. 2025.
Can Agents Fix Agent Issues? Alfin Wijaya Rahardja, Junwei Liu, Weitong Chen, Zhenpeng Chen, Yiling Lou. NeurIPS 2025.
Co-PatcheR: Collaborative Software Patching with Component(s)-specific Small Reasoning Models. Yuheng Tang, Hongwei Li, Kaijie Zhu, Michael Yang, Yangruibo Ding, Wenbo Guo. NeurIPS 2025.
RepoMaster: Autonomous Exploration and Understanding of GitHub Repositories for Complex Task Solving. Huacan Wang, Ziyi Ni, Shuo Zhang, Shuo Lu, Sen Hu, Ziyang He, Chen Hu, Jiaye Lin, Yifu Guo, Ronghao Chen, et al. NeurIPS 2025.
SE-Agent: Self-Evolution Trajectory Optimization in Multi-Step Reasoning with LLM-Based Agents. Jiaye Lin, Yifu Guo, Yuzhen Han, Sen Hu, Ziyi Ni, Licheng Wang, Mingguang Chen, Hongzhang Liu, Ronghao Chen, Yangfan He, et al. NeurIPS 2025.
Co-Evolving LLM Coder and Unit Tester via Reinforcement Learning. Yinjie Wang, Ling Yang, Ye Tian, Ke Shen, Mengdi Wang. NeurIPS 2025.
SWE-RL: Advancing LLM Reasoning via Reinforcement Learning on Open Software Evolution. Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, Sida I. Wang. NeurIPS 2025.
SWE-smith: Scaling Data for Software Engineering Agents. John Yang, Kilian Lieret, Carlos E. Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, Diyi Yang. NeurIPS 2025 Datasets & Benchmarks Track.
SWE-bench Goes Live! Linghao Zhang, Shilin He, Chaoyun Zhang, Yu Kang, Bowen Li, Chengxing Xie, Junhao Wang, Maoquan Wang, Yufan Huang, Shengyu Fu, et al. NeurIPS 2025 Datasets & Benchmarks Track.
Training Software Engineering Agents and Verifiers with SWE-Gym. Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, Yizhe Zhang. ICML 2025.
SWE-Flow: Synthesizing Software Engineering Data in a Test-Driven Manner. Lei Zhang, Jiaxi Yang, Min Yang, Jian Yang, Mouxiang Chen, Jiajun Zhang, Zeyu Cui, Binyuan Hui, Junyang Lin. ICML 2025.
Guided Search Strategies in Non-Serializable Environments with Applications to Software Engineering Agents. Karina Zainullina, Alexander Golubev, Maria Trofimova, Sergei Polezhaev, Ibragim Badertdinov, Daria Litvintseva, Simon Karasik, Filipp Fisin, Sergei Skvortsov, Maksim Nekrashevich, et al. ICML 2025.
SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering? Samuel Miserendino, Michele Wang, Tejal Patwardhan, Johannes Heidecke. ICML 2025.
Automated Benchmark Generation for Repository-Level Coding Tasks. Konstantinos Vergopoulos, Mark Niklas Müller, Martin Vechev. ICML 2025.
OpenHands: An Open Platform for AI Software Developers as Generalist Agents. Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, et al. ICLR 2025.
Diversity Empowers Intelligence: Integrating Expertise of Software Engineering Agents. Kexun Zhang, Weiran Yao, Zuxin Liu, Yihao Feng, Zhiwei Liu, Rithesh R N, Tian Lan, Lei Li, Renze Lou, Jiacheng Xu, et al. ICLR 2025.
SWE-GPT: A Process-Centric Language Model for Automated Software Improvement. Yingwei Ma, Rongyu Cao, Yongchang Cao, Yue Zhang, Jue Chen, Yibo Liu, Yuchen Liu, Binhua Li, Fei Huang, Yongbin Li. ISSTA 2025.
SPICE: An Automated SWE-Bench Labeling Pipeline for Issue Clarity, Test Coverage, and Effort Estimation. Gustavo A. Oliva, Gopi Krishnan Rajbahadur, Aaditya Bhatia, Haoxiang Zhang, Yihao Chen, Zhilong Chen, Arthur Leung, Dayi Lin, Boyuan Chen, Ahmed E. Hassan. ASE 2025.
DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversal. Vaibhav Aggarwal, Ojasv Kamal, Abhinav Japesh, Zhijing Jin, Bernhard Schölkopf. ACL 2025.
CompileAgent: Automated Real-World Repo-Level Compilation with Tool-Integrated LLM-based Agent System. Li Hu, Guoqiang Chen, Xiuwei Shang, Shaoyin Cheng, Benlong Wu, LiGangyang LiGangyang, Xu Zhu, Weiming Zhang, Nenghai Yu. ACL 2025.
SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning. Zexiong Ma, Chao Peng, Pengfei Gao, Xiangxin Meng, Yanzhen Zou, Bing Xie. ACL 2025.
SWE-Fixer: Training Open-Source LLMs for Effective and Efficient GitHub Issue Resolution. Chengxing Xie, Bowen Li, Chang Gao, He Du, Wai Lam, Difan Zou, Kai Chen. ACL 2025 Findings.
Agentless: Demystifying LLM-based Software Engineering Agents. Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, Lingming Zhang. FSE 2025.
OmniGIRL: A Multilingual and Multimodal Benchmark for GitHub Issue Resolution. Lianghong Guo, Wei Tao, Runhan Jiang, Yanlin Wang, Jiachi Chen, Xilin Liu, Yuchi Ma, Mingzhi Mao, Hongyu Zhang, Zibin Zheng. ISSTA 2025.
Boosting Open-Source LLMs for Program Repair via Reasoning Transfer and LLM-Guided Reinforcement Learning. Xunzhu Tang, Jacques Klein, Tegawendé F. Bissyandé. TOSEM 2025.
AutoCodeRover: Autonomous Program Improvement. Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, Abhik Roychoudhury. ISSTA 2024.
SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik R. Narasimhan, Ofir Press. NeurIPS 2024.
MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution. Wei Tao, Yucheng Zhou, Yanlin Wang, Wenqiang Zhang, Hongyu Zhang, Yu Cheng. NeurIPS 2024.
MASAI: Modular Architecture for Software-engineering AI Agents. Nalin Wadhwa, Atharv Sonwane, Daman Arora, Abhav Mehrotra, Saiteja Utpala, Ramakrishna B. Bairi, Aditya Kanade, Nagarajan Natarajan. NeurIPS 2024 Workshop.
SWE-bench: Can Language Models Resolve Real-World GitHub Issues? Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, Karthik Narasimhan. ICLR 2024.
CodeR: Issue Resolving with Multi-Agent and Task Graphs. Dong Chen, Shaoxin Lin, Muhan Zeng, Daoguang Zan, Jian-Gang Wang, Anton Cheshkov, Jun Sun, Hao Yu, Guoliang Dong, Artem Aliev, et al. arXiv 2024.
MarsCode Agent: AI-native Automated Bug Fixing. Yizhou Liu, Pengfei Gao, Xinchen Wang, Jie Liu, Yexuan Shi, Zhao Zhang, Chao Peng. arXiv 2025.

🖥️ Terminal Operating¶

AI agents that operate within terminal environments, executing shell commands, managing system operations, and automating command-line workflows through natural language interfaces and autonomous task execution.

Terminus: A research-preview agent for consistently evaluating the abilities of language models to power autonomous agents in the terminal. Mike Merrill, Alex Shaw. 2025.
Terminal-Bench: A Benchmark for AI Agents in Terminal Environments. The Terminal-Bench Team. 2025.

🧑‍💻 Code Generation¶

AI agents that autonomously generate, scaffold, and synthesize code at the repository level, leveraging external tools and APIs to create new modules, build complete projects, and construct large-scale codebases.

Lost in Code Generation: Reimagining the Role of Software Models in AI-driven Software Engineering. Jürgen Cito, Dominik Bork. arXiv 2025/11.
Towards Realistic Project-Level Code Generation via Multi-Agent Collaboration and Semantic Architecture Modeling. Qianhui Zhao, Li Zhang, Fang Liu, Junhang Cheng, Chengru Wu, Junchen Ai, Qiaoyuanhe Meng, Lichen Zhang, Xiaoli Lian, Shubin Song, et al. arXiv 2025/11.
RPG: A Repository Planning Graph for Unified and Scalable Codebase Generation. Jane Luo, Xin Zhang, Steven Liu, Jie Wu, Jianfeng Liu, Yiming Huang, Yangyu Huang, Chengyu Yin, Ying Xin, Yuefeng Zhan, et al. arXiv 2025.
SimdBench: Benchmarking Large Language Models for SIMD-Intrinsic Code Generation. Yibo He, Shuoran Zhao, Jiaming Huang, Yingjie Fu, Hao Yu, Cunjian Huang, Tao Xie. arXiv 2025.
MutaGReP: Execution-Free Repository-Grounded Plan Search for Code-Use. Zaid Khan, Ali Farhadi, Ranjay Krishna, Luca Weihs, Mohit Bansal, Tanmay Gupta. arXiv 2025.
Improving Cursor Tab with online RL. Jacob Jackson, Phillip Kravtsov, Shomil Jain. 2025.
EvoAgentX: An Automated Framework for Evolving Agentic Workflows. Yingxu Wang, Siwei Liu, Jinyuan Fang, Zaiqiao Meng. arXiv 2025.
SEW: Self-Evolving Agentic Workflows for Automated Code Generation. Siwei Liu, Jinyuan Fang, Han Zhou, Yingxu Wang, Zaiqiao Meng. arXiv 2025.
Co-Saving: Resource Aware Multi-Agent Collaboration for Software Development. Rennai Qiu, Chen Qian, Ran Li, Yufan Dang, Weize Chen, Cheng Yang, Yingli Zhang, Ye Tian, Xuantang Xiong, Lei Han, et al. arXiv 2025.
Think Like an Engineer: A Neuro-Symbolic Collaboration Agent for Generative Software Requirements Elicitation and Self-Review. Sai Zhang, Zhenchang Xing, Jieshan Chen, Dehai Zhao, Zizhong Zhu, Xiaowang Zhang, Zhiyong Feng, Xiaohong Li. arXiv 2025.
HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scale. Huy Nhat Phan, Tien N. Nguyen, Phong X. Nguyen, Nghi D. Q. Bui. arXiv 2025.
Verbal Process Supervision Elicits Better Coding Agents. Hao-Yuan Chen, Cheng-Pong Huang, Jui-Ming Yao. arXiv 2025.
KernelBench: Can LLMs Write Efficient GPU Kernels? Anne Ouyang, Simon Guo, Simran Arora, Alex L. Zhang, William Hu, Christopher Ré, Azalia Mirhoseini. ICML 2025.
On the Impacts of Contexts on Repository-Level Code Generation. Nam Le Hai, Dung Manh Nguyen, Nghi D. Q. Bui. NAACL 2025 Findings.
CodeSIM: Multi-Agent Code Generation and Problem Solving through Simulation-Driven Planning and Debugging. Md. Ashraful Islam, Mohammed Eunus Ali, Md Rizwan Parvez. NAACL 2025 Findings.
ProjectEval: A Benchmark for Programming Agents Automated Evaluation on Project-Level Code Generation. Kaiyuan Liu, Youcheng Pan, Yang Xiang, Daojing He, Jing Li, Yexing Du, Tianrun Gao. ACL 2025 Findings.
AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology. Minh Huynh Nguyen, Thang Chau Phan, Phong X. Nguyen, Nghi D. Q. Bui. FORGE 2025.
CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generation. Xinchen Wang, Pengfei Gao, Chao Peng, Ruida Hu, Cuiyun Gao. ASE 2025.
Multi-Agent Collaboration via Evolving Orchestration. Yufan Dang, Chen Qian, Xueheng Luo, Jingru Fan, Zihao Xie, Ruijie Shi, Weize Chen, Cheng Yang, Xiaoyin Che, Ye Tian, et al. NeurIPS 2025.
Multi-Agent Collaboration via Cross-Team Orchestration. Zhuoyun Du, Chen Qian, Wei Liu, Zihao Xie, YiFei Wang, Rennai Qiu, Yufan Dang, Weize Chen, Cheng Yang, Ye Tian, et al. ACL 2025 Findings.
Scaling Large Language Model-based Multi-Agent Collaboration. Chen Qian, Zihao Xie, YiFei Wang, Wei Liu, Kunlun Zhu, Hanchen Xia, Yufan Dang, Zhuoyun Du, Weize Chen, Cheng Yang, et al. ICLR 2025.
Commit0: Library Generation from Scratch. Wenting Zhao, Nan Jiang, Celine Lee, Justin T Chiu, Claire Cardie, Matthias Gallé, Alexander M Rush. ICLR 2025.
AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors. Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, et al. ICLR 2024.
RepoBench: Benchmarking Repository-Level Code Auto-Completion Systems. Tianyang Liu, Canwen Xu, Julian McAuley. ICLR 2024.
RepoAgent: An LLM-Powered Open-Source Framework for Repository-level Code Documentation Generation. Qinyu Luo, Yining Ye, Shihao Liang, Zhong Zhang, Yujia Qin, Yaxi Lu, Yesai Wu, Xin Cong, Yankai Lin, Yingli Zhang, et al. EMNLP 2024 Demo.
Experiential Co-Learning of Software-Developing Agents. Chen Qian, Yufan Dang, Jiahao Li, Wei Liu, Zihao Xie, YiFei Wang, Weize Chen, Cheng Yang, Xin Cong, Xiaoyin Che, et al. ACL 2024.
ChatDev: Communicative Agents for Software Development. Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. ACL 2024.
MapCoder: Multi-Agent Code Generation for Competitive Problem Solving. Md. Ashraful Islam, Mohammed Eunus Ali, Md Rizwan Parvez. ACL 2024.
A Pair Programming Framework for Code Generation via Multi-Plan Exploration and Feedback-Driven Refinement. Huan Zhang, Wei Cheng, Yuhan Wu, Wei Hu. ASE 2024.
Iterative Experience Refinement of Software-Developing Agents. Chen Qian, Jiahao Li, Yufan Dang, Wei Liu, YiFei Wang, Zihao Xie, Weize Chen, Cheng Yang, Yingli Zhang, Zhiyuan Liu, et al. arXiv 2024.
CodeTree: Agent‐guided Tree Search for Code Generation with Large Language Models. Jierui Li, Hung Le, Yingbo Zhou, Caiming Xiong, Silvio Savarese, Doyen Sahoo. arXiv 2024.
AgentCoder: Multi-Agent-based Code Generation with Iterative Testing and Optimisation. Dong Huang, Jie M.Zhang, Michael Luck, Qingwen Bu, Yuhao Qing, Heming Cui. arXiv 2024.
Self-Organized Agents: A LLM Multi-Agent Framework toward Ultra Large-Scale Code Generation and Optimization. Yoichi Ishibashi, Yoshimasa Nishimura. arXiv 2024.

🏗 Environment Building¶

Papers describing new environments, IDE sandboxes, benchmarks, or agent playgrounds.

DI-BENCH: Benchmarking Large Language Models on Dependency Inference with Testable Repositories at Scale. Linghao Zhang, Junhao Wang, Shilin He, Chaoyun Zhang, Yu Kang, Bowen Li, Jiaheng Wen, Chengxing Xie, Maoquan Wang, Yufan Huang, et al. arXiv 2025.
AutoDev: Automated AI-Driven Development. Michele Tufano, Anisha Agarwal, Jinu Jang, Roshanak Zilouchian Moghaddam, Neel Sundaresan. arXiv 2025.
R2E-Gym: Procedural Environments and Hybrid Verifiers for Scaling Open-Weights SWE Agents. Naman Jain, Jaskirat Singh, Manish Shetty, Liang Zheng, Koushik Sen, Ion Stoica. arXiv 2025.
Can Language Models Go Beyond Coding? Assessing the Capability of Language Models to Build Real-World Systems. Chenyu Zhao, Shenglin Zhang, Zeshun Huang, Weilin Jin, Yongqian Sun, Dan Pei, Chaoyun Zhang, Qingwei Lin, Chetan Bansal, Saravan Rajmohan, et al. arXiv 2025.
Repo2Run: Automated Building Executable Environment for Code Repository at Scale. Ruida Hu, Chao Peng, Xinchen Wang, Junjielong Xu, Cuiyun Gao. NeurIPS 2025.
RepoST: Scalable Repository-Level Coding Environment Construction with Sandbox Testing. Yiqing Xie, Alex Xie, Divyanshu Sheth, Pengfei Liu, Daniel Fried, Carolyn Rose. COLM 2025.
Treefix: Enabling Execution with a Tree of Prefixes. Beatriz Souza, Michael Pradel. ICSE 2025.
You Name It, I Run It: An LLM Agent to Execute Tests of Arbitrary Projects. Islem Bouzenia, Michael Pradel. ISSTA 2025.
CXXCrafter: An LLM-Based Agent for Automated C/C++ Open Source Software Building. Zhengmin Yu, Yuan Zhang, Ming Wen, Yinan Nie, Wenhui Zhang, Min Yang. FSE 2025.
CompileAgent: Automated Real-World Repo-Level Compilation with Tool-Integrated LLM-based Agent System. Li Hu, Guoqiang Chen, Xiuwei Shang, Shaoyin Cheng, Benlong Wu, LiGangyang LiGangyang, Xu Zhu, Weiming Zhang, Nenghai Yu. ACL 2025.
CSR-Bench: Benchmarking LLM Agents in Deployment of Computer Science Research Repositories. Yijia Xiao, Runhui Wang, Luyang Kong, Davor Golac, Wei Wang. NAACL 2025.
EnvBench: A Benchmark for Automated Environment Setup. Aleksandra Eliseeva, Alexander Kovrigin, Ilia Kholkin, Egor Bogomolov, Yaroslav Zharov. ICLR 2025 Workshop.
Beyond pip install: Evaluating LLM Agents for the Automated Installation of Python Projects. Louis Milliken, Sungmin Kang, Shin Yoo. SANER 2025.
R2E: Turning any Github Repository into a Programming Agent Environment. Naman Jain, Manish Shetty, Tianjun Zhang, King Han, Koushik Sen, Ion Stoica. ICML 2024.
Automatically Generating Dockerfiles via Deep Learning: Challenges and Promises. Giovanni Rosa, Antonio Mastropaolo, Simone Scalabrino, Gabriele Bavota, Rocco Oliveto. ICSSP 2023.

🔁 Issue Reproduction¶

Research on reproducing software bugs deterministically.

Issue2Test: Generating Reproducing Test Cases from Issue Reports. Noor Nashid, Islem Bouzenia, Michael Pradel, Ali Mesbah. arXiv 2025/10.
Execution-Feedback Driven Test Generation from SWE Issues. Toufique Ahmed, Jatin Ganhotra, Avraham Shinnar, Martin Hirzel. arXiv 2025/08.
Benchmarking LLMs for Unit Test Generation from Real-World Functions. Dong Huang, Jie M. Zhang, Mark Harman, Qianru Zhang, Mingzhe Du, See-Kiong Ng. arXiv 2025/08.
AssertFlip: Reproducing Bugs via Inversion of LLM-Generated Passing Tests. Lara Khatib, Noble Saji Mathews, Meiyappan Nagappan. arXiv 2025/07.
Agentic Bug Reproduction for Effective Automated Program Repair at Google. Runxiang Cheng, Michele Tufano, Jürgen Cito, José Cambronero, Pat Rondon, Renyao Wei, Aaron Sun, Satish Chandra. arXiv 2025/05.
Can LLM Generate Regression Tests for Software Commits? Jing Liu, Seongmin Lee, Eleonora Losiouk, Marcel Böhme. arXiv 2025/01.
Otter: Generating Tests from Issues to Validate SWE Patches. Toufique Ahmed, Jatin Ganhotra, Rangeet Pan, Avraham Shinnar, Saurabh Sinha, Martin Hirzel. ICML 2025.
Agents in the Sandbox: End-to-End Crash Bug Reproduction for Minecraft. Eray Yapağcı, Yavuz Alp Sencer Öztürk, Eray Tüzün. ASE 2025.
Automated Generation of Issue-Reproducing Tests by Combining LLMs and Search-Based Testing. Konstantinos Kitsios, Marco Castelluccio, Alberto Bacchelli. ASE 2025.
AEGIS: An Agent-based Framework for General Bug Reproduction from Issue Descriptions. Xinchen Wang, Pengfei Gao, Xiangxin Meng, Chao Peng, Ruida Hu, Yun Lin, Cuiyun Gao. FSE 2025 Industry Papers.
An Empirical Study on Leveraging Images in Automated Bug Report Reproduction. Dingbang Wang, Zhaoxu Zhang, Sidong Feng, William G. J. Halfond, Tingting Yu. MSR 2025.
LLMs as Continuous Learners: Improving the Reproduction of Defective Code in Software Issues. Yalan Lin, Yingwei Ma, Rongyu Cao, Binhua Li, Fei Huang, Xiaodong Gu, Yongbin Li. arXiv 2024.
TDD-Bench Verified: Can LLMs Generate Tests for Issues Before They Get Resolved? Toufique Ahmed, Martin Hirzel, Rangeet Pan, Avraham Shinnar, Saurabh Sinha. arXiv 2024/12.
Evaluating Diverse Large Language Models for Automatic and General Bug Reproduction. Sungmin Kang, Juyeon Yoon, Nargiz Askarbekkyzy, Shin Yoo. IEEE Transactions on Software Engineering 2024.
SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents. Niels Mündler, Mark Niklas Müller, Jingxuan He, Martin Vechev. NeurIPS 2024.
Large Language Models are Few-shot Testers: Exploring LLM-based General Bug Reproduction. Sungmin Kang, Juyeon Yoon, Shin Yoo. ICSE 2023.

🎯 Issue Localization¶

Code search, fault localization, vulnerability detection.

Improving Code Localization with Repository Memory. Boshi Wang, Weijian Xu, Yunsheng Li, Mei Gao, Yujia Xie, Huan Sun, Dongdong Chen. arXiv 2025.
Leveraging Large Language Model for Information Retrieval-based Bug Localization. Moumita Asad, Rafed Muhammad Yasir, Sam Malek. arXiv 2025/10.
Tool-integrated Reinforcement Learning for Repo Deep Search. Zexiong Ma, Chao Peng, Qunhong Zeng, Pengfei Gao, Yanzhen Zou, Bing Xie. arXiv 2025.
Bridging Bug Localization and Issue Fixing: A Hierarchical Localization Framework Leveraging Large Language Models. Jianming Chang, Xin Zhou, Lulu Wang, David Lo, Bixin Li. arXiv 2025/02.
LocAgent: Graph-Guided LLM Agents for Code Localization. Zhaoling Chen, Robert Tang, Gangda Deng, Fang Wu, Jialong Wu, Zhiwei Jiang, Viktor Prasanna, Arman Cohan, Xingyao Wang. ACL 2025.
Issue Localization via LLM-Driven Iterative Code Graph Searching. Zhonghao Jiang, Xiaoxue Ren, Meng Yan, Wei Jiang, Yong Li, Zhongxin Liu. ASE 2025.

❓ Question Answering¶

Code understanding, documentation, and retrieval-based Q&A.

SWE-QA: Can Language Models Answer Repository-level Code Questions? Weihan Peng, Yuling Shi, Yuhang Wang, Xinyun Zhang, Beijun Shen, Xiaodong Gu. arXiv 2025.
Benchmarking Long-Context Language Models on Long Code Understanding. Jia Li, Xuyuan Guo, Lei Li, Kechi Zhang, Ge Li, Jia Li, Zhengwei Tao, Fang Liu, Chongyang Tao, Yuqi Zhu, et al. ACL 2025.
On Improving Repository-Level Code QA for Large Language Models. Jan Strich, Florian Schneider, Irina Nikishina, Chris Biemann. ACL 2024 Workshop.

🔍 Pull Request Review¶

Automated pull request creation, review assistance, linting, refactoring.

Benchmarking and Studying the LLM-based Code Review. Zhengran Zeng, Ruikai Shi, Keke Han, Yixin Li, Kaicheng Sun, Yidong Wang, Zhuohao Yu, Rui Xie, Wei Ye, Shikun Zhang. arXiv 2025.
On the Use of Agentic Coding: An Empirical Study of Pull Requests on GitHub. Miku Watanabe, Hao Li, Yutaro Kashiwa, Brittany Reid, Hajimu Iida, Ahmed E. Hassan. arXiv 2025/09.
The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering. Hao Li, Haoxiang Zhang, Ahmed E. Hassan. arXiv 2025.
Issue-Oriented Agent-Based Framework for Automated Review Comment Generation. Shuochuan Li, Dong Wang, Patanamon Thongtanunam, Zan Wang, Jiuqiao Yu, Junjie Chen. arXiv 2025.
Armchair. Armchair. 2025.
PReview: A Benchmark Dataset for Pull Request Outcomes and Quality Analysis. Anonymous Authors. 2025.
PR-Agent: An AI-Powered Tool for Automated Pull Request Analysis, Feedback, Suggestions and More! Qodo. 2024.
CodeAgent: Autonomous Communicative Agents for Code Review. Xunzhu Tang, Kisub Kim, Yewei Song, Cedric Lothritz, Bei Li, Saad Ezzini, Haoye Tian, Jacques Klein, Tegawendé F. Bissyandé. EMNLP 2024.
Automating Code Review Activities by Large-Scale Pre-training. Zhiyu Li, Shuai Lu, Daya Guo, Nan Duan, Shailesh Jannu, Grant Jenks, Deep Majumder, Jared Green, Alexey Svyatkovskiy, Shengyu Fu, et al. FSE 2022.
Can We Benchmark Code Review Studies? A Systematic Mapping Study of Methodology, Dataset, and Metric. Dong Wang, Yuki Ueda, Raula Gaikovina Kula, Takashi Ishio, Kenichi Matsumoto. Journal of Systems and Software, 2021, Elsevier.

✨ Feature Development¶

Studies on agent-driven feature extension, repo-level edits.

EvoDev: An Iterative Feature-Driven Framework for End-to-End Software Development with LLM-based Agents. Junwei Liu, Chen Xu, Chong Wang, Tong Bai, Weitong Chen, Kaseng Wong, Yiling Lou, Xin Peng. arXiv 2025/11.
NoCode-bench: A Benchmark for Evaluating Natural Language-Driven Feature Addition. Le Deng, Zhonghao Jiang, Jialun Cao, Michael Pradel, Zhongxin Liu. arXiv 2025.
FEA-Bench: A Benchmark for Evaluating Repository-Level Code Generation for Feature Implementation. Wei Li, Xin Zhang, Zhongxin Guo, Shaoguang Mao, Wen Luo, Guangyue Peng, Yangyu Huang, Houfeng Wang, Scarlett Li. ACL 2025.
SWE-Dev: Building Software Engineering Agents with Training and Inference Scaling. Haoran Wang, Zhenyu Hou, Yao Wei, Jie Tang, Yuxiao Dong. ACL 2025 Findings.

🔄 Git Management¶

Agents for git workflows (branching, rebasing, conflict resolution).

GitGoodBench: A Novel Benchmark For Evaluating Agentic Performance On Git. Tobias Lindenbauer, Egor Bogomolov, Yaroslav Zharov. REALM 2025.

⚡ Performance Optimization¶

Code profiling, optimization, memory & latency improvements.

SWE-Perf: Can Language Models Optimize Code Performance on Real-World Repositories? Xinyi He, Qian Liu, Mingzhe Du, Lin Yan, Zhijie Fan, Yiming Huang, Zejian Yuan, Zejun Ma. arXiv 2025.

🧪 Test Generation¶

Autonomous agents for automated software testing through intelligent fuzzing techniques.

Intention-Driven Generation of Project-Specific Test Cases. Binhang Qi, Yun Lin, Xinyi Weng, Yuhuan Huang, Chenyan Liu, Hailong Sun, Zhi Jin, Jin Song Dong. arXiv 2025/09.
Locus: Agentic Predicate Synthesis for Directed Fuzzing. Jie Zhu, Chihao Shen, Ziyang Li, Jiahao Yu, Yizheng Chen, Kexin Pei. arXiv 2025.

🚚 Code Migration¶

Agent-based migration of codebases and files between programming languages.

MigrationBench: Repository-Level Code Migration Benchmark from Java 8. Linbo Liu, Xinle Liu, Qiang Zhou, Lin Chen, Yihan Liu, Hoan Nguyen, Behrooz Omidvar-Tehrani, Xi Shen, Jun Huan, Omer Tripp, et al. arXiv 2025/05.
MatchFixAgent: Language-Agnostic Autonomous Repository-Level Code Translation Validation and Repair. Ali Reza Ibrahimzada, Brandon Paulsen, Reyhaneh Jabbarvand, Joey Dodds, Daniel Kroening. arXiv 2025.
Exploring and Unleashing the Power of Large Language Models in CI/CD Configuration Translation. Chong Wang, Chen Zhang, Jiajun Wu, Wunan Guo, Jianfeng Qu, Yewen Tian, Yang Liu. arXiv 2025.
What a diff makes: automating code migration with large language models. Katherine A. Rosenfeld, Cliff C. Kerr, Jessica Lundin. arXiv 2025.

🧹 Code Refactoring¶

Autonomous agents that analyze and transform source code to improve structure, maintainability, or performance.

RefAgent: A Multi-agent LLM-based Framework for Automatic Software Refactoring. Khouloud Oueslati, Maxime Lamothe, Foutse Khomh. arXiv 2025.

🔒 Software Security Engineering Agents¶

Studies on agentic vulnerability detection, patching, and secure coding.

SecRepoBench: Benchmarking Code Agents for Secure Code Completion in Real-World Repositories. Chihao Shen, Connor Dilgren, Purva Chiniya, Luke Griffith, Yu Ding, Yizheng Chen. arXiv 2025/11.
SEC-bench: Automated Benchmarking of LLM Agents on Real-World Software Security Tasks. Hwiwon Lee, Ziqi Zhang, Hanxiao Lu, Lingming Zhang. arXiv 2025.
SecureAgentBench: Benchmarking Secure Code Generation under Realistic Vulnerability Scenarios. Junkai Chen, Huihui Huang, Yunbo Lyu, Junwen An, Jieke Shi, Chengran Yang, Ting Zhang, Haoye Tian, Yikun Li, Zhenhao Li, et al. arXiv 2025.
Towards Exception Safety Code Generation with Intermediate Representation Agents Framework. Xuanming Zhang, Yuxuan Chen, Yuan Yuan, Minlie Huang. arXiv 2025.
CVE-Bench: A Benchmark for AI Agents’ Ability to Exploit Real-World Web Application Vulnerabilities. Yuxuan Zhu, Antony Kellermann, Dylan Bowman, Philip Li, Akul Gupta, Adarsh Danda, Richard Fang, Conner Jensen, Eric Ihli, Jason Benn, et al. ICML 2025.
CVE-Bench: Benchmarking LLM-based Software Engineering Agent’s Ability to Repair Real-World CVE Vulnerabilities. Peiran Wang, Xiaogeng Liu, Chaowei Xiao. NAACL 2025.

🖥️ System Engineering Agents¶

AI agents for operating system development, low-level systems programming, compiler/toolchain engineering, and large-scale systems infrastructure. This includes OS kernel code, runtime systems, device drivers, and system-level code generation.

🗃️ Database Engineering Agents¶

Autonomous agents for solving SQL challenges in real-world database systems (e.g., query generation and optimization, issue resolution).

SWE-SQL: Illuminating LLM Pathways to Solve User SQL Issues in Real-World Applications. Jinyang Li, Xiaolong Li, Ge Qu, Per Jacobsson, Bowen Qin, Binyuan Hui, Shuzheng Si, Nan Huo, Xiaohan Xu, Yue Zhang, et al. arXiv 2025.
GateLens: A Reasoning-Enhanced LLM Agent for Automotive Software Release Analytics. Arsham Gholamzadeh Khoee, Shuai Wang, Yinan Yu, Robert Feldt, Dhasarathy Parthasarathy. arXiv 2025.

⚙️ Hardware Engineering Agents¶

AI agents for hardware design, HDL/HLS code generation, FPGA/ASIC toolchains, RTL/HLS synthesis, accelerator co-design, and hardware-aware optimization. This category covers Verilog/VHDL/RTL, FPGA kernels, and hardware–software co-design pipelines.

Focus: Better Verilog Generation from Large Language Model via Focused Reasoning. Zhuorui Zhao, Bing Li, Grace Li Zhang, Ulf Schlichtmann. SOCC 2025.

🌐 Website Engineering Agents¶

🌐 Front-End UI Generation¶

Code agents for the automated creation and maintenance of web interfaces and front-end components.

JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence. Qiushi Sun, Jingyang Gong, Yang Liu, Qiaosheng Chen, Lei Li, Kai Chen, Qipeng Guo, Ben Kao, Fei Yuan. arXiv 2025/10.
InteractScience: Programmatic and Visually-Grounded Evaluation of Interactive Scientific Demonstration Code Generation. Qiaosheng Chen, Yang Liu, Lei Li, Kai Chen, Qipeng Guo, Gong Cheng, Fei Yuan. arXiv 2025/10.
WebGen-Agent: Enhancing Interactive Website Generation with Multi-Level Feedback and Step-Level Reinforcement Learning. Zimu Lu, Houxing Ren, Yunqiao Yang, Ke Wang, Zhuofan Zong, Junting Pan, Mingjie Zhan, Hongsheng Li. arXiv 2025.
WebGen-Bench: Evaluating LLMs on Generating Interactive and Functional Websites from Scratch. Zimu Lu, Yunqiao Yang, Houxing Ren, Haotian Hou, Han Xiao, Ke Wang, Weikang Shi, Aojun Zhou, Mingjie Zhan, Hongsheng Li. arXiv 2025.
ScreenCoder: Advancing Visual-to-Code Generation for Front-End Automation via Modular Multimodal Agents. Yilei Jiang, Yaozhi Zheng, Yuxuan Wan, Jiaming Han, Qunzhong Wang, Michael R. Lyu, Xiangyu Yue. arXiv 2025.
Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototyping. Jingyu Xiao, Yuxuan Wan, Yintong Huo, Zixin Wang, Xinyi Xu, Wenxuan Wang, Zhiyao Xu, Yuhang Wang, Michael R. Lyu. arXiv 2025.
DesignBench: A Comprehensive Benchmark for MLLM-based Front-end Code Generation. Jingyu Xiao, Ming Wang, Man Ho Lam, Yuxuan Wan, Junliang Liu, Yintong Huo, Michael R. Lyu. arXiv 2025.
DesignCoder: Hierarchy-Aware and Self-Correcting UI Code Generation with Large Language Models. Yunnong Chen, Shixian Ding, YingYing Zhang, Wenkai Chen, Jinzhou Du, Lingyun Sun, Liuqing Chen. arXiv 2025.
MLLM-Based UI2Code Automation Guided by UI Layout Information. Fan Wu, Cuiyun Gao, Shuqing Li, Xin-Cheng Wen, Qing Liao. ISSTA 2025.
Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineering. Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, Diyi Yang. NAACL 2025.
WebMMU: A Benchmark for Multimodal Multilingual Website Understanding and Code Generation. Rabiul Awal, Mahsa Massoud, Aarash Feizi, Zichao Li, Suyuchen Wang, Christopher Pal, Aishwarya Agrawal, David Vazquez, Siva Reddy, Juan A. Rodriguez, et al. EMNLP 2025.
UXAgent: An LLM Agent-Based Usability Testing Framework for Web Design. Yuxuan Lu, Bingsheng Yao, Hansu Gu, Jing Huang, Jessie Wang, Yang Li, Jiri Gesi, Qi He, Toby Jia-Jun Li, Dakuo Wang. CHI EA 2025.

🖥️ Backend Service Generation¶

Code agents for automated creation, modification, and optimization of backend services, APIs, and server-side logic.

BaxBench: Can LLMs Generate Correct and Secure Backends? Mark Vero, Niels Mündler, Victor Chibotaru, Veselin Raychev, Maximilian Baader, Nikola Jovanović, Jingxuan He, Martin Vechev. ICML 2025.

🌐 Code-Executing Web Agents¶

Autonomous agents that generate and execute code to interact with web applications and browsers.

OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use Agents. Hongrui Jia, Jitong Liao, Xi Zhang, Haiyang Xu, Tianbao Xie, Chaoya Jiang, Ming Yan, Si Liu, Wei Ye, Fei Huang. arXiv 2025.
Browser Use: The AI browser agent. Browser Use. 2025.
SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills. Boyuan Zheng, Michael Y. Fatemi, Xiaolong Jin, Zora Zhiruo Wang, Apurva Gandhi, Yueqi Song, Yu Gu, Jayanth Srinivasa, Gaowen Liu, Graham Neubig, et al. arXiv 2025.
WebDS: An End-to-End Benchmark for Web-based Data Science. Ethan Hsu, Hong Meng Yam, Ines Bouissou, Aaron Murali John, Raj Thota, Josh Koe, Vivek Sarath Putta, G K Dharesan, Alexander Spangher, Shikhar Murty, et al. arXiv 2025.
Inducing Programmatic Skills for Agentic Tasks. Zora Zhiruo Wang, Apurva Gandhi, Graham Neubig, Daniel Fried. COLM 2025.
Tree-of-Code: A Self-Growing Tree Framework for End-to-End Code Generation and Execution in Complex Tasks. Ziyi Ni, Yifan Li, Ning Yang, Dou Shen, Pin Lyu, Daxiang Dong. ACL 2025 Findings.
Executable Code Actions Elicit Better LLM Agents. Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, Heng Ji. ICML 2024.
AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents. Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, Niranjan Balasubramanian. ACL 2024.
APIGen: Automated PIpeline for Generating Verifiable and Diverse Function-Calling Datasets. Zuxin Liu, Thai Hoang, Jianguo Zhang, Ming Zhu, Tian Lan, Shirley Kokane, Juntao Tan, Weiran Yao, Zhiwei Liu, Yihao Feng, et al. NeurIPS 2024 Datasets and Benchmarks Track.
If LLM Is the Wizard, Then Code Is the Wand: A Survey on How Code Empowers Large Language Models to Serve as Intelligent Agents. Ke Yang, Jiateng Liu, John Wu, Chaoqi Yang, Yi R. Fung, Sha Li, Zixuan Huang, Xu Cao, Xingyao Wang, Yiquan Wang, et al. ICLR 2024 Workshop.

🔬 Research Engineering Agents¶

👩‍💻 Machine Learning Engineering¶

Autonomous agents across end-to-end ML workflows.

DeepCode: Open Agentic Coding. Data Intelligence Lab@HKU. 2025.
Reinforcement Learning for Machine Learning Engineering Agents. Sherry Yang, Joy He-Yueya, Percy Liang. arXiv 2025.
MLE-Smith: Scaling MLE Tasks with Automated Multi-Agent Pipeline. Rushi Qiang, Yuchen Zhuang, Anikait Singh, Percy Liang, Chao Zhang, Sherry Yang, Bo Dai. arXiv 2025.
PerfDojo: Automated ML Library Generation for Heterogeneous Architectures. Andrei Ivanov, Siyuan Shen, Gioele Gottardo, Marcin Chrapek, Afif Boudaoud, Timo Schneider, Luca Benini, Torsten Hoefler. arXiv 2025.
AIDE: AI-Driven Exploration in the Space of Code. Zhengyao Jiang, Dominik Schmidt, Dhruv Srikanth, Dixing Xu, Ian Kaplan, Deniss Jacenko, Yuxiang Wu. arXiv 2025.
Towards Community-Driven Agents for Machine Learning Engineering. Sijie Li, Weiwei Sun, Shanda Li, Ameet Talwalkar, Yiming Yang. arXiv 2025.
ML-Agent: Reinforcing LLM Agents for Autonomous Machine Learning Engineering. Zexi Liu, Jingyi Chai, Xinyu Zhu, Shuo Tang, Rui Ye, Bo Zhang, Lei Bai, Siheng Chen. arXiv 2025.
MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering. Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, et al. arXiv 2025.
MLE-STAR: Machine Learning Engineering Agent via Search and Targeted Refinement. Jaehyun Nam, Jinsung Yoon, Jiefeng Chen, Jinwoo Shin, Sercan Ö. Arık, Tomas Pfister. arXiv 2025.
ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologies. Shubham Gandhi, Dhruv Shah, Manasi Patwardhan, Lovekesh Vig, Gautam Shroff. arXiv 2025.
MLR-Bench: Evaluating AI Agents on Open-Ended Machine Learning Research. Hui Chen, Miao Xiong, Yujie Lu, Wei Han, Ailin Deng, Yufei He, Jiaying Wu, Yibo Li, Yue Liu, Bryan Hooi. NeurIPS 2025 Datasets & Benchmarks Track.
RepoMaster: Autonomous Exploration and Understanding of GitHub Repositories for Complex Task Solving. Huacan Wang, Ziyi Ni, Shuo Zhang, Shuo Lu, Sen Hu, Ziyang He, Chen Hu, Jiaye Lin, Yifu Guo, Ronghao Chen, et al. NeurIPS 2025.
MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering. Jun Shern Chan, Neil Chowdhury, Oliver Jaffe, James Aung, Dane Sherburn, Evan Mays, Giulio Starace, Kevin Liu, Leon Maksin, Tejal Patwardhan, et al. ICLR 2025.
ML-Bench: Evaluating Large Language Models and Agents for Machine Learning Tasks on Repository-Level Code. Xiangru Tang, Yuliang Liu, Zefan Cai, Daniel Shao, Junjie Lu, Yichi Zhang, Zexuan Deng, Helan Hu, Kaikai An, Ruijun Huang, et al. ICLR 2025 Workshop.
MLE-Agent: Your Intelligent Companion for Seamless AI Engineering and Research. Huaizheng Zhang, Yizheng Huang, Lei Zhang. 2024.

🤖 Automated Data Science¶

Agents programmatically orchestrate data workflows—exploring datasets, preprocessing, engineering features, and automating end-to-end analyses via executable code pipelines.

DeepAnalyze: Agentic Large Language Models for Autonomous Data Science. Shaolei Zhang, Ju Fan, Meihao Fan, Guoliang Li, Xiaoyong Du. arXiv 2025.
WebDS: An End-to-End Benchmark for Web-based Data Science. Ethan Hsu, Hong Meng Yam, Ines Bouissou, Aaron Murali John, Raj Thota, Josh Koe, Vivek Sarath Putta, G K Dharesan, Alexander Spangher, Shikhar Murty, et al. arXiv 2025.
AutoMind: Adaptive Knowledgeable Agent for Automated Data Science. Yixin Ou, Yujie Luo, Jingsheng Zheng, Lanning Wei, Zhuoyun Yu, Shuofei Qiao, Jintian Zhang, Da Zheng, Yuren Mao, Yunjun Gao, et al. arXiv 2025.
DA-Code: Agent Data Science Code Generation Benchmark for Large Language Models. Yiming Huang, Jianwen Luo, Yan Yu, Yitong Zhang, Fangyu Lei, Yifan Wei, Shizhu He, Lifu Huang, Xiao Liu, Jun Zhao, et al. EMNLP 2024.

📊 Agentic Visualization¶

Agents dedicated to automated plotting, data visualization, and graphical insight generation.

JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence. Qiushi Sun, Jingyang Gong, Yang Liu, Qiaosheng Chen, Lei Li, Kai Chen, Qipeng Guo, Ben Kao, Fei Yuan. arXiv 2025/10.
Plot2Code: A Comprehensive Benchmark for Evaluating Multi-modal Large Language Models in Code Generation from Scientific Plots. Chengyue Wu, Yixiao Ge, Qiushan Guo, Jiahao Wang, Zhixuan Liang, Zeyu Lu, Ying Shan, Ping Luo. arXiv 2025.
OpusAnimation: Code-Based Dynamic Chart Generation. Bozheng Li, Miao Yang, Zhenhan Chen, Jiawang Cao, Mushui Liu, Yi Lu, Yongliang Wu, Bin Zhang, Yangguang Ji, Licheng Tang, et al. arXiv 2025.
From Charts to Code: A Hierarchical Benchmark for Multimodal Models. Jiahao Tang, Henry Hengyuan Zhao, Lijian Wu, Yifei Tao, Dongxing Mao, Yang Wan, Jingru Tan, Min Zeng, Min Li, Alex Jinpeng Wang. arXiv 2025.
Chart-CoCa: Self-Improving Chart Understanding of Vision LMs via Code-Driven Synthesis and Candidate-Conditioned Answering. Gongyao Jiang, Qiong Luo. CIKM 2025.

🧬 Scientific Software Workflows¶

Agents designed to autonomously utilize specialized scientific software—such as simulation engines, data analysis suites, and visualization platforms—to automate and enhance domain-specific scientific workflows.

ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows. Qiushi Sun, Zhoumianze Liu, Chang Ma, Zichen Ding, Fangzhi Xu, Zhangyue Yin, Haiteng Zhao, Zhenyu Wu, Kanzhi Cheng, Zhaoyang Liu, et al. arXiv 2025/06.

🎨 Visual Engineering Agents¶

🌀 Animation Generation¶

Agents that automate the creation or manipulation of animations, including video, GIFs, or interactive visual content, using code-based workflows.

JanusCoder: Towards a Foundational Visual-Programmatic Interface for Code Intelligence. Qiushi Sun, Jingyang Gong, Yang Liu, Qiaosheng Chen, Lei Li, Kai Chen, Qipeng Guo, Ben Kao, Fei Yuan. arXiv 2025/10.
LogoMotion: Visually-Grounded Code Synthesis for Creating and Editing Animation. Vivian Liu, Rubaiat Habib Kazi, Li-Yi Wei, Matthew Fisher, Timothy Langlois, Seth Walker, Lydia Chilton. CHI 2025.

🖼️ SVG Generation¶

Agents that programmatically generate SVG or vector graphics using script-driven automation.

Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models. Ronghuan Wu, Wanchao Su, Jing Liao. CVPR 2025.

🎮 Game Engineering Agents¶

🎮 Game Generation¶

Agents designed to autonomously create, modify, or enhance games by generating code, assets, or logic.

Game-RL: Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoning. Jingqi Tong, Jixin Tang, Hangcheng Li, Yurong Mou, Ming Zhang, Jun Zhao, Yanbo Wen, Fan Song, Jiahao Zhan, Yuyang Lu, et al. arXiv 2025.
Agents in the Sandbox: End-to-End Crash Bug Reproduction for Minecraft. Eray Yapağcı, Yavuz Alp Sencer Öztürk, Eray Tüzün. ASE 2025.

🕹️ Code-Executing Game Agents¶

Autonomous agents that generate and execute code to interact with game environments, enabling tasks like gameplay, content creation, and environment manipulation through code.

Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agents. Zihao Wang, Xujing Li, Yining Ye, Junjie Fang, Haoming Wang, Longxiang Liu, Shihao Liang, Junting Lu, Zhiyong Wu, Jiazhan Feng, et al. arXiv 2025.
One Life to Learn: Inferring Symbolic World Models for Stochastic Environments from Unguided Exploration. Zaid Khan, Archiki Prasad, Elias Stengel-Eskin, Jaemin Cho, Mohit Bansal. arXiv 2025.
PoE-World: Compositional World Modeling with Products of Programmatic Experts. Wasu Top Piriyakulkij, Yichao Liang, Hao Tang, Adrian Weller, Marta Kryven, Kevin Ellis. arXiv 2025.
WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment. Hao Tang, Darren Yan Key, Kevin Ellis. NeurIPS 2024.

🧊 3D Engineering Agents¶

🧊 3D Object Design¶

Agents that generate and render 3D models or scenes through code automation, enabling creative design workflows.

From Idea to CAD: A Language Model-Driven Multi-Agent System for Collaborative Design. Felix Ocker, Stefan Menzel, Ahmed Sadik, Thiago Rios. arXiv 2025.
CADDesigner: Conceptual Design of CAD Models Based on General-Purpose Agent. Jingzhe Ni, Xiaolong Yin, Xingyu Lu, Xintong Li, Ji Wei, Ruofeng Tong, Min Tang, Peng Du. arXiv 2025.
Agentic Design of Compositional Machines. Wenqian Zhang, Weiyang Liu, Zhen Liu. arXiv 2025.
Imperative vs. Declarative Programming Paradigms for Open-Universe Scene Generation. Maxim Gumin, Do Heon Han, Seung Jean Yoo, Aditya Ganeshan, R. Kenny Jones, Rio Aguina-Kang, Stewart Morris, Daniel Ritchie. arXiv 2025.
Learning Object Placement Programs for Indoor Scene Synthesis with Iterative Self Training. Adrian Chang, Kai Wang, Yuanbo Li, Manolis Savva, Angel X. Chang, Daniel Ritchie. arXiv 2025.
HSM: Hierarchical Scene Motifs for Multi-Scale Indoor Scene Generation. Hou In Derek Pun, Hou In Ivan Tam, Austin T. Wang, Xiaoliang Huo, Angel X. Chang, Manolis Savva. arXiv 2025.
MeshCoder: LLM-Powered Structured Mesh Code Generation from Point Clouds. Bingquan Dai, Li Ray Luo, Qihong Tang, Jie Wang, Xinyu Lian, Hao Xu, Minghan Qin, Xudong Xu, Bo Dai, Haoqian Wang, et al. arXiv 2025.
MetaGen: A DSL, Database, and Benchmark for VLM-Assisted Metamaterial Generation. Liane Makatura, Benjamin Jones, Siyuan Bian, Wojciech Matusik. arXiv 2025.
ShapeLib: Designing a Library of Programmatic 3D Shape Abstractions with Large Language Models. R. Kenny Jones, Paul Guerrero, Niloy J. Mitra, Daniel Ritchie. arXiv 2025.
VLMaterial: Procedural Material Generation with Large Vision-Language Models. Beichen Li, Rundi Wu, Armando Solar-Lezama, Changxi Zheng, Liang Shi, Bernd Bickel, Wojciech Matusik. ICLR 2025.
Generating CAD Code with Vision-Language Models for 3D Designs. Kamel Alrashedy, Pradyumna Tambwekar, Zulfiqar Haider Zaidi, Megan Langwasser, Wei Xu, Matthew Gombolay. ICLR 2025.
CAD-Llama: Leveraging Large Language Models for Computer-Aided Design Parametric 3D Model Generation. Jiahao Li, Weijian Ma, Xueyang Li, Yunzhong Lou, Guichun Zhou, Xiangdong Zhou. CVPR 2025.
The Scene Language: Representing Scenes with Programs, Words, and Embeddings. Yunzhi Zhang, Zizhang Li, Matt Zhou, Shangzhe Wu, Jiajun Wu. CVPR 2025.
BlenderGym: Benchmarking Foundational Model Systems for Graphics Editing. Yunqi Gu, Ian Huang, Jihyeon Je, Guandao Yang, Leonidas Guiba. CVPR 2025.
CAD-Editor: A Locate-then-Infill Framework with Automated Training Data Synthesis for Text-Based CAD Editing. Yu Yuan, Shizhao Sun, Qi Liu, Jiang Bian. ICML 2025.
IR3D-Bench: Evaluating Vision-Language Model Scene Understanding as Agentic Inverse Rendering. Parker Liu, Chenxin Li, Zhengxin Li, Yipeng Wu, Wuyang Li, Zhiqin Yang, Zhenyuan Zhang, Yunlong Lin, Sirui Han, Brandon Y. Feng. NeurIPS 2025 Datasets and Benchmarks Track.
SceneMotifCoder: Example-driven Visual Program Learning for Generating 3D Object Arrangements. Hou In Ivan Tam, Hou In Derek Pun, Austin T. Wang, Angel X. Chang, Manolis Savva. 3DV 2025.
3D-GPT: Procedural 3D Modeling with Large Language Models. Chunyi Sun, Junlin Han, Weijian Deng, Xinlong Wang, Zishan Qin, Stephen Gould. 3DV 2025.
SceneGenAgent: Precise Industrial Scene Generation with Coding Agent. Xiao Xia, Dan Zhang, Zibo Liao, Zhenyu Hou, Tianrui Sun, Jing Li, Ling Fu, Yuxiao Dong. ACL 2025.
CADTalk: An Algorithm and Benchmark for Semantic Commenting of CAD Programs. Haocheng Yuan, Jing Xu, Hao Pan, Adrien Bousseau, Niloy J. Mitra, Changjian Li. CVPR 2024.
SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code. Ziniu Hu, Ahmet Iscen, Aashi Jain, Thomas Kipf, Yisong Yue, David A. Ross, Cordelia Schmid, Alireza Fathi. ICML 2024.
Open-Universe Indoor Scene Generation using LLM Program Synthesis and Uncurated Object Databases. Rio Aguina-Kang, Maxim Gumin, Do Heon Han, Stewart Morris, Seung Jean Yoo, Aditya Ganeshan, R. Kenny Jones, Qiuhong Anna Wei, Kailiang Fu, Daniel Ritchie. arXiv 2024.
How Can Large Language Models Help Humans in Design and Manufacturing? Liane Makatura, Michael Foshey, Bohan Wang, Felix HähnLein, Pingchuan Ma, Bolei Deng, Megan Tjandrasuwita, Andrew Spielberg, Crystal Elaine Owens, Peter Yichen Chen, et al. arXiv 2023.

🤖 Embodied Engineering Agents¶

🤖 Code-Executing Embodied Agents¶

Agents that interact with physical or simulated environments by executing code for embodied tasks including reasoning, navigation, and manipulation. These agents shift the representation of plans from action sequences to code and embed task queries, robot actions, solution samples, and fallback behaviors as programs.

MORSE-500: A Programmatically Controllable Video Benchmark to Stress-Test Multimodal Reasoning. Zikui Cai, Andrew Wang, Anirudh Satheesh, Ankit Nakhawa, Hyunwoo Jae, Keenan Powell, Minghui Liu, Neel Jay, Sungbin Oh, Xiyao Wang, et al. arXiv 2025.
Chain-of-Modality: Learning Manipulation Programs from Multimodal Human Videos with Vision-Language-Models. Chen Wang, Fei Xia, Wenhao Yu, Tingnan Zhang, Ruohan Zhang, C. Karen Liu, Li Fei-Fei, Jie Tan, Jacky Liang. ICRA 2025.
Visual Agentic AI for Spatial Reasoning with a Dynamic API. Damiano Marsili, Rohun Agrawal, Yisong Yue, Georgia Gkioxari. CVPR 2025.
Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection. Enshen Zhou, Qi Su, Cheng Chi, Zhizheng Zhang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, He Wang. CVPR 2025.
GRS: Generating Robotic Simulation Tasks from Real-World Images. Alex Zook, Fan-Yun Sun, Josef Spjut, Valts Blukis, Stan Birchfield, Jonathan Tremblay. CVPR 2025.
Can Large Language Models Understand Symbolic Graphics Programs? Zeju Qiu, Weiyang Liu, Haiwen Feng, Zhen Liu, Tim Z. Xiao, Katherine M. Collins, Joshua B. Tenenbaum, Adrian Weller, Michael J. Black, Bernhard Schölkopf. ICLR 2025.
Endowing Visual Reprogramming with Adversarial Robustness. Shengjie Zhou, Xin Cheng, Haiyang Xu, Ming Yan, Tao Xiang, Feng Liu, Lei Feng. ICLR 2025.
IR3D-Bench: Evaluating Vision-Language Model Scene Understanding as Agentic Inverse Rendering. Parker Liu, Chenxin Li, Zhengxin Li, Yipeng Wu, Wuyang Li, Zhiqin Yang, Zhenyuan Zhang, Yunlong Lin, Sirui Han, Brandon Y. Feng. NeurIPS 2025 Datasets and Benchmarks Track.
Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generation. Yue Yang, Ajay Patel, Matt Deitke, Tanmay Gupta, Luca Weihs, Andrew Head, Mark Yatskar, Chris Callison-Burch, Ranjay Krishna, Aniruddha Kembhavi, et al. ACL 2025.
RoboScript: Code Generation for Free-Form Manipulation Tasks across Real and Simulation. Junting Chen, Yao Mu, Qiaojun Yu, Tianming Wei, Silang Wu, Zhecheng Yuan, Zhixuan Liang, Chao Yang, Kaipeng Zhang, Wenqi Shao, et al. arXiv 2024.
RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis. Yao Mu, Junting Chen, Qinglong Zhang, Shoufa Chen, Qiaojun Yu, Chongjian Ge, Runjian Chen, Zhixuan Liang, Mengkang Hu, Chaofan Tao, et al. ICML 2024.
Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models. Yushi Hu, Otilia Stretcu, Chun-Ta Lu, Krishnamurthy Viswanathan, Kenji Hata, Enming Luo, Ranjay Krishna, Ariel Fuxman. CVPR 2024.
Recursive Visual Programming. Jiaxin Ge, Sanjay Subramanian, Baifeng Shi, Roei Herzig, Trevor Darrell. ECCV 2024.
Video Question Answering with Procedural Programs. Rohan Choudhury, Koichiro Niinuma, Kris Kitani, Laszlo Jeni. ECCV 2024.
ChatGPT for Robotics: Design Principles and Model Abilities. Sai Vemprala, Rogerio Bonatti, Arthur Bucker, Ashish Kapoor. arXiv 2023.
ProgPrompt: Generating Situated Robot Task Plans using Large Language Models. Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, Animesh Garg. ICRA 2023.
Code as Policies: Language Model Programs for Embodied Control. Jacky Liang, Wenlong Huang, Fei Xia, Peng Xu, Karol Hausman, Brian Ichter, Pete Florence, Andy Zeng. ICRA 2023.
Visual Programming: Compositional visual reasoning without training. Tanmay Gupta, Aniruddha Kembhavi. CVPR 2023.
ViperGPT: Visual Inference via Python Execution for Reasoning. Dídac Surís, Sachit Menon, Carl Vondrick. ICCV 2023.
ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representation. Yangyi Chen, Xingyao Wang, Manling Li, Derek Hoiem, Heng Ji. EMNLP 2023.
Modular Visual Question Answering via Code Generation. Sanjay Subramanian, Medhini Narasimhan, Kushal Khangaonkar, Kevin Yang, Arsha Nagrani, Cordelia Schmid, Andy Zeng, Trevor Darrell, Dan Klein. ACL 2023 Short Papers.
Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Model. Siyuan Huang, Zhengkai Jiang, Hao Dong, Yu Qiao, Peng Gao, Hongsheng Li. arXiv 2023.

��️ Research Landscape¶

We are a young team passionate about the future of code agents, and we look forward to discussing exciting ideas with the community. This field sits at the intersection of software engineering, artificial intelligence (especially LLMs and agentic reasoning), and automated code development, experiencing extremely rapid evolution since 2023.

🌟 Vision¶

Advancing toward general-purpose agents capable of understanding, modifying, and creating complex codebases, collaborating with humans, and autonomously driving end-to-end software engineering processes—from requirements, to implementation, to testing, deployment, and maintenance.

🧩 Open Problems¶

Long-horizon planning: Enabling agents to reason and act coherently over many steps in large, realistic codebases.
Robust evaluation: Designing benchmarks and metrics that reflect real-world complexity, generalizability, and value beyond short snippets.
Interpretability & safety: Ensuring agent actions are understandable, controllable, and safe for deployment on critical systems.
Collaboration: Seamlessly integrating multiple agents and human-in-the-loop workflows.
Repository-level grounding: Equipping agents with persistent context over evolving, multi-file software.
Resource efficiency: Addressing compute/memory requirements for large-scale agentic work.

Conferences and Workshops¶

ICSE — International Conference on Software Engineering [SE]
FSE (ESEC/FSE) — Foundations of Software Engineering [SE]
ASE — Automated Software Engineering [SE]
ISSTA — International Symposium on Software Testing and Analysis [SE/Testing]
ICLR — International Conference on Learning Representations [ML]
ICML — International Conference on Machine Learning [ML]
NeurIPS — Conference on Neural Information Processing Systems [ML]
ACL — Annual Meeting of the Association for Computational Linguistics [NLP]
EMNLP — Empirical Methods in Natural Language Processing [NLP]
NAACL — North American Chapter of the ACL [NLP]
TheWebConf (WWW) — The Web Conference (formerly WWW) [Web]

🧪 Frontier Labs and Teams¶

OpenAI: Work on MLE-bench, large-scale evaluations, and agent architecture.
Google DeepMind: Pioneering code-centric models and embodied agent applications.
Microsoft Research: Advances in multi-agent collaboration, feature-benchmarks, and tool-assisted agents.
THUDM (Tsinghua): SWE-Dev, general SE agent architecture research.
Scale AI: SWE-Bench, SWE-Bench Pro, real-world repo agent benchmarking.
Amazon AWS AI Lab: SWE-PolyBench and multilingual repo agent research.
Meta AI Research: Studies on agent robustness and failure analysis.
QuantaAlpha: GitTaskBench and RepoMaster for sophisticated repo understanding.
Stanford Human-Centered AI: Software agents for ML engineering, pipeline automation.

(See main list for additional innovative contributors. Please suggest more leading labs!)

🤝 Contributing¶

We welcome contributions! Please: 1. Use the entry template. 2. Place items in the right category & order by reverse-chronology. 3. Include badges for GitHub stars, arXiv, website if available.

We're grateful to all our amazing contributors who have made this project what it is today!

If you have any questions or encounter issues, please feel free to reach out. For quick queries, you can also check our Issues page for common questions and solutions.