一句话：提出用不确定性估计改进 LLM 在低资源编程语言上的代码生成，并构建新的评估基准。
Code LLM 在高资源编程语言上表现优异，但在低资源语言（LRPL）上表现欠佳。已有工作通过继续训练或分词器适配来提升 LRPL 性能，但依赖昂贵数据且易导致灾难性遗忘。本文提出利用不确定性估计（UE）来改善 LLM 在 LRPL 上的代码生成。研究预计贡献三点：一是覆盖 Python、Java、Racket 与 Elixir 的新代码生成基准，除功能正确性外还评估可读性、效率与惯用风格；二是用于评估代码生成不确定性估计的新基准；三是基于 UE 的提升方法，包括按低不确定性过滤合成训练数据、UE 驱动的课程学习策略、不确定性感知解码，以及将不确定性作为对齐阶段的强化学习奖励。预期通过全面评估证明 UE 能提升 LRPL 代码生成，并开源基准与模型。
Large language models for source code (Code LLMs) demonstrate great performance on high-resource programming languages (HRPLs) but struggle with low-resource ones (LRPLs). Previous studies have improved LLM performance on LRPLs by continued training or tokenizer adaptation. However, they require costly data and can cause catastrophic forgetting. This paper proposes to address the poor performance of LLMs on LRPLs using uncertainty estimation (UE). UE methods have advanced LLM performance on natural language tasks, but are underexplored in source code settings. The research may provide three contributions: (1) a new code generation benchmark evaluating not only functional correctness but also readability, efficiency, and idiomatic style across Python, Java, Racket, and Elixir; (2) a new benchmark for evaluating uncertainty estimation when generating code; and (3) methods to improve LRPL code generation by leveraging UE. The methods utilizing UE include filtering synthetic training data by low uncertainty, an UE-driven curriculum learning strategy, uncertainty-aware decoding, and using uncertainty as an RL reward in alignment. The research may provide a comprehensive evaluation of uncertainty in code models, demonstrate that UE can improve LRPL generation, and open-source release of benchmarks and models as outcomes.
DOI 原文 ·
@inproceedings{paperbot9317,
title = {Improving Quality of LLM Code Generation in Low-Resource Programming Languages via Uncertainty Estimation},
author = {Georgii Andriushchenko},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00399}
}