一句话：提出 Porting Benchmark 与统一评测框架，系统衡量五类安全补丁回移工具并揭示其泛化短板。
自动化安全补丁向后移植对缓解 N-day 漏洞至关重要，但现有工具的高成功率多来自同仓库或特定版本环境，泛化性不明。本文构建 Porting Benchmark，包含 1,234 个跨版本、跨分支与跨仓库的安全补丁回移案例，并配套统一评测框架；在一致设置下评估传统程序分析、LLM prompting 与 LLM agent 五款先进工具。结果显示，对齐评测改变性能格局：PortGPT 与 TSBPort 在 Replication 数据集上仍较强，FixMorph 与 Mystique 明显退化；补丁越复杂，最佳 commit-level 成功率从 Type-I 的 85.2% 降至 Type-IV 的 24.0%。作者归纳四类根因，并在 45 个带测试与 POC 的动态验证子集上发现静态参考分数不能完全反映真实修复，执行反馈精修仅带来有限但可测的恢复。
Automated security patch backporting is critical for mitigating N-day vulnerabilities. Recent tools have reported high success rates, often exceeding 80% on their respective datasets. However, these evaluations are typically confined to homogeneous environments, such as within the same repository or specific project versions. Consequently, it remains unclear how well these tools generalize beyond their originally targeted scenarios. We present Porting Benchmark, a curated dataset of 1,234 security patch backporting cases spanning cross-version, cross-branch, and cross-repository scenarios, paired with a common evaluation framework. Using this benchmark, we evaluate five state-of-the-art tools spanning traditional program analysis, LLM-based prompting, and LLM agents under aligned evaluation settings. Our results show that aligned evaluation changes the apparent performance landscape: PortGPT and TSBPort remain comparatively strong on the Replication Dataset, while FixMorph and Mystique degrade substantially under the common protocol. Performance degrades sharply on structurally complex patches: the best commit-level success rate falls from 85.2% on Type-I patches to 24.0% on Type-IV. We identify four root-cause categories (missing target API awareness, cross-version semantic mismatch, non-local dependency propagation failure, and patch construction or localization failure) and derive concrete directions for next-generation tool design. On a 45-case dynamically validated subset with verified test cases and constructed POCs, we further observe that reference-based benchmark scores do not fully capture real-world remediation: exact match sharply under-credits harder target adaptations, while executable validation reveals residual integration failures in the target that static reference agreement misses. Executable-feedback refinement provides limited but measurable recovery on the hardest executable cases.
@inproceedings{paperbot12496,
title = {Benchmarking Automated Security Patch Backporting: How Far Are We?},
author = {Jincheng Yang and Yulong Fu and Chengwei Liu and Lyuye Zhang and Fangyuan Zhang and Bingyang Ren and Yang Liu and Hui Li},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3837542}
}