🔍 Overview | 🛠️ Installation | 🚀 Quick Start | 📝 Citation

• 🎉 [May 2025] PatchPilot accepted at ICML 2025!
• 🚀 [May 2025] PatchPilot code are now open-sourced!
• 🚀 [February 2025] PatchPilot achieves superior performance on bench while maintaining low cost (< $1 per instance)!
• 📄 [February 2025] PatchPilot paper is available on arXiv!

PatchPilot is an innovative rule-based planning patching tool that strikes the excellent balance between patching efficacy, stability, and cost-efficiency.

Key Innovations:

• 🎯 Five-Component Workflow: Reproduction, Localization, Generation, Validation, and Refinement
• 💰 Cost-Efficient: Less than $1 per instance while maintaining high performance
• 🔒 High Stability: More stable than agent-based planning methods
• ⚡ Superior Performance: Outperforms existing open-source methods on SWE-bench

PatchPilot's workflow consists of five specialized components:

1. 🔄 Reproduction: Reproduce the reported bug to understand the issue
2. 🔍 Localization: Identify problematic code locations with multi-level analysis
3. ⚡ Generation: Generate high-quality patch candidates
4. 🛡️ Validation: Validate patches through comprehensive testing
5. ✨ Refinement: Unique refinement step to improve patch quality

1. Pull the Docker image:

docker pull 3rdn4/patchpilot_verified:v1

2. Run the container with Docker-in-Docker support:

docker run --privileged -v /var/run/docker.sock:/var/run/docker.sock -it 3rdn4/patchpilot_verified:v1

Note: --privileged -v /var/run/docker.sock:/var/run/docker.sock is required for Docker-in-Docker functionality used by SWE-bench.

3. Set up the environment inside the container:

cd /opt
git clone git@github.com:ucsb-mlsec/PatchPilot.git
cd PatchPilot
conda activate patchpilot
export PYTHONPATH=$PYTHONPATH:$(pwd)

4. Configure API keys:

# For Anthropic Claude
export ANTHROPIC_API_KEY=your_anthropic_key_here

# OR for OpenAI
export OPENAI_API_KEY=your_openai_key_here

First, reproduce the bugs to understand the issues:

python patchpilot/reproduce/reproduce.py \
    --reproduce_folder results/reproduce \
    --num_threads 50 \
    --setup_map setup_result/verified_setup_map.json \
    --tasks_map setup_result/verified_tasks_map.json \
    --task_list_file swe_verify_tasks.txt

python patchpilot/fl/localize.py \
    --file_level \
    --direct_line_level \
    --output_folder results/localization \
    --top_n 5 \
    --compress \
    --context_window=20 \
    --temperature 0.7 \
    --match_partial_paths \
    --reproduce_folder results/reproduce \
    --task_list_file swe_verify_tasks.txt \
    --num_samples 4 \
    --num_threads 16 \
    --benchmark verified

python patchpilot/fl/localize.py \
    --merge \
    --output_folder results/localization/merged \
    --start_file results/localization/loc_outputs.jsonl \
    --num_samples 4

Generate patches with integrated validation:

python patchpilot/repair/repair.py \
    --loc_file results/localization/merged/loc_all_merged_outputs.jsonl \
    --output_folder results/repair \
    --loc_interval \
    --top_n=5 \
    --context_window=20 \
    --max_samples 12 \
    --batch_size 4 \
    --benchmark verified \
    --reproduce_folder results/reproduce \
    --verify_folder results/verify \
    --setup_map setup_result/verified_setup_map.json \
    --tasks_map setup_result/verified_tasks_map.json \
    --num_threads 16 \
    --task_list_file swe_verify_tasks.txt \
    --refine_mod \
    --benchmark verified

Note: Functionality tests are retrieved through useful_scripts/generate_functest.py and do not use the pass_to_pass approach.

Run SWE-bench evaluation on the generated patches:

cd /opt/orig_swebench/SWE-bench
conda activate swe_bench

python -m swebench.harness.run_evaluation \
    --predictions_path [path_to_best_patches_round_2.jsonl] \
    --max_workers 16 \
    --run_id [experiment_name]

If an experiment is interrupted, simply rerun the same command - PatchPilot will resume from where it left off. For different experiments, clean the folders or use different output directories.

If you find PatchPilot useful in your research, please cite our paper:

@article{li2025patchpilot,
  title={PatchPilot: A Stable and Cost-Efficient Agentic Patching Framework},
  author={Li, Hongwei and Tang, Yuheng and Wang, Shiqi and Guo, Wenbo},
  journal={arXiv preprint arXiv:2502.02747},
  year={2025}
}

Made with ❤️ by the UCSB ML Security Team