一句话：提出类型约束解码：用前缀自动机与可居留类型搜索保证 LLM 生成代码良类型，编译错误减少过半。
LLM 虽在代码生成上表现突出，但其逐 token 推理不建模代码的形式性质，常生成无法编译、尤其带类型错误的代码；已有约束解码仅覆盖 DSL 或语法特征，难以约束类型。本文提出类型约束解码：用类型系统引导生成，设计了新的前缀自动机与可居留类型（inhabitable types）搜索，形成可靠的良类型性约束，并在基础简单类型语言上形式化、扩展到 TypeScript 以验证实用性。在 HumanEval 与 MBPP 上，跨多个模型规模与家族的代码合成、翻译与修复任务中，编译错误减少一半以上，功能正确性显著提升。
Large language models (LLMs) have achieved notable success in code generation. However, they still frequently produce uncompilable output because their next-token inference procedure does not model formal aspects of code. Although constrained decoding is a promising approach to alleviate this issue, it has only been applied to handle either domain-specific languages or syntactic features of general-purpose programming languages. However, LLMs frequently generate code with typing errors, which are beyond the domain of syntax and generally hard to adequately constrain. To address this challenge, we introduce a type-constrained decoding approach that leverages type systems to guide code generation. For this purpose, we develop novel prefix automata and a search over inhabitable types, forming a sound approach to enforce well-typedness on LLM-generated code. We formalize our approach on a foundational simply-typed language and extend it to TypeScript to demonstrate practicality. Our evaluation on the HumanEval and MBPP datasets shows that our approach reduces compilation errors by more than half and significantly increases functional correctness in code synthesis, translation, and repair tasks across LLMs of various sizes and model families, including state-of-the-art open-weight models with more than 30B parameters. The results demonstrate the generality and effectiveness of our approach in constraining LLM code generation with formal rules of type systems.
@article{paperbot3123,
title = {Type-Constrained Code Generation with Language Models},
author = {Niels Mündler and Jingxuan He and Hao Wang and Koushik Sen and Dawn Song and Martin Vechev},
journal = {Proceedings of the ACM on Programming Languages},
volume = {9},
number = {PLDI},
year = {2025},
doi = {10.1145/3729274}
}