一句话：实证研究 LLM 能否为三种转换规则 DSL 合成正确、可泛化、可复用的代码转换规则，并揭示其能力边界。
论文关注 LLM 的黑箱性、不确定性以及在大规模代码库上重复使用带来的高成本问题，指出代码转换规则（code transformation rules）可缓解这些挑战。为此作者针对三种转换规则领域特定语言——Comby、GritQL 与 Ast-Grep，开展了一项新的实证研究：在覆盖四类软件演化任务（API 误用纠正、程序修复、API 迁移、语言版本迁移）的六个数据集上评估三个 LLM（GPT-5.4、GPT-oss-120B、Llama3.1-8B）。结果显示，规则合成在前沿模型上已超越概念验证：GPT-5.4 在多数基准上规则可应用率稳定偏高，生成的变换最接近基准真值；较小的开源权重模型对简单、局部改动仍有效，但在复杂迁移场景中表现不佳。论文还观察到通过元变量获得的不可忽略的泛化性，以及多数数据集第一四分位上的高复用分。与 anti-unification 算法相比，LLM 在正确性上更优，但规则可应用性较差。总体而言，LLM 在生成合理、正确、可泛化且可复用的规则方面展现出巨大潜力。
Due to their black-box nature, LLMs suffer from limited explain- ability and a lack of determinism. Their usage cost can also rise, particularly with repetitive tasks on large codebases. Code transfor- mation rules mitigate those challenges. This paper fills this gap, with a novel empirical study targeting three domain-specific languages for transformation rules, namely Comby, GritQL, and Ast-Grep We evaluate three LLMs (GPT-5.4, GPT-oss-120B, and Llama3.1-8B) on six diverse datasets covering four software-evolution tasks: API misuse correction, program repair, API migration, and language ver- sion migration. Our results provide evidence that transformation rule synthesis moves beyond proof-of-concept with strong frontier models. GPT-5.4 achieves consistently high rule applicability rates and produces transformations closest to the ground truth across most benchmarks. Smaller and open-weight GPT-oss-120B and Llama3.1-8B models remain effective for simpler, localized changes but struggle with complex migration scenarios. We also observe non-negligible generalizability through the usage of meta-variables and through a high reuse score in the first quartile of many datasets. Finally, when compared to the anti-unification algorithm, LLMs outperform it in correctness, but underperform in rule applicabil- ity. Overall, our results show great potential for LLMs to generate sound, correct, generalizable, and reusable rules.