一句话：提出 PtTrust，借助内部状态预训练的两阶段风险评估框架，实现代码行级风险检测并跨任务、跨语言泛化。
预训练范式推动了大语言模型的成功，具备编码能力的代码 LLM 正逐渐融入开发者日常工作，但其可能生成错误、不安全或不可靠的代码，可信性仍是严峻挑战。已有探索表明，分析 LLM 的内部状态有望像人脑无意识察觉错误那样识别风险输出，但现有方法多局限于 LLM 行为的狭窄子领域，难以达到工业级可扩展性与实用性。本文提出 PtTrust，一个基于内部状态预训练的两阶段代码 LLM 风险评估框架，可与软件公司现有基础设施无缝集成。其核心思想是让风险评估框架本身也经历类似 LLM 的预训练：先在大规模无标注源码上进行无监督预训练，学习 LLM 状态的通用表示；再用小规模标注数据训练风险预测器。作者通过细粒度的代码行级风险评估验证了有效性，并表明其可跨任务、跨编程语言泛化；进一步实验显示 PtTrust 提供高度直观、可解释的特征，有助于增强用户信任。该工作向可扩展、可信赖的代码 LLM 保障迈出一步。
The pre-training paradigm plays a key role in the success of Large Language Models (LLMs), which have been recognized as one of the most significant advancements of AI recently. Building on these breakthroughs, code LLMs with advanced coding capabilities bring huge impacts on software engineering, showing the tendency to become an essential part of developers' daily routines. However, the current code LLMs still face serious challenges related to trustworthiness, as they can generate incorrect, insecure, or unreliable code. Recent exploratory studies find that it can be promising to detect such risky outputs by analyzing LLMs' internal states, akin to how the human brain unconsciously recognizes its own mistakes. Yet, most of these approaches are limited to narrow sub-domains of LLM operations and fall short of achieving industry-level scalability and practicability. To address these challenges, in this paper, we propose PtTrust, a two-stage risk assessment framework for code LLM based on internal state pre-training, designed to integrate seamlessly with the existing infrastructure of software companies. The core idea is that the risk assessment framework could also undergo a pre-training process similar to LLMs. Specifically, PtTrust first performs unsupervised pre-training on large-scale unlabeled source code to learn general representations of LLM states. Then, it uses a small, labeled dataset to train a risk predictor. We demonstrate the effectiveness of PtTrust through fine-grained, code line-level risk assessment and demonstrate that it generalizes across tasks and different programming languages. Further experiments also reveal that PtTrust provides highly intuitive and interpretable features, fostering greater user trust. We believe PtTrust makes a promising step toward scalable and trustworthy assurance for code LLMs.