一句话：提出 CodeEraser，用梯度上升式机器遗忘免重训地擦除代码语言模型中的敏感记忆，并保持代码结构与功能正确性。
代码语言模型在代码生成、摘要等任务上表现优异，但实证研究揭示其会无意记忆训练数据中的敏感信息，在特定提示下逐字复现机密内容。已有的训练数据去重与差分隐私增强等方法需要全量重训练，对已部署模型而言计算代价过高。本文首次系统研究能否通过机器遗忘——一种无需全量重训的事后修改手段——有效且高效地擦除代码语言模型中的敏感记忆。作者先量化训练数据中敏感数据的记忆风险，构建包含 5 万条高风险敏感记忆样本的遗忘目标数据集，随后考察两种基于梯度上升的遗忘方法（原始版与约束版），并提出 CodeEraser：在代码中选择性地遗忘敏感记忆片段，同时保持周边代码的结构完整性与功能正确性。在 CodeParrot、CodeGen-Mono 与 Qwen2.5-Coder 三个模型族上的大量实验验证了 CodeEraser 在擦除目标敏感记忆方面的有效性与高效性，并维持了模型效用。
While Code Language Models (CLMs) have demonstrated superior performance in software engineering tasks such as code generation and summarization, recent empirical studies reveal a critical privacy vulnerability: these models exhibit unintended memorization of sensitive training data, enabling verbatim reproduction of confidential information when specifically prompted. To address this issue, several approaches, including training data de-duplication and differential privacy augmentation, have been proposed. However, these methods require full-model retraining for deployed CLMs, which incurs substantial computational costs. In this paper, we aim to answer the following research question: Can sensitive information memorized by CLMs be erased effectively and efficiently? We conduct a pioneering investigation into erasing sensitive memorization in CLMs through machine unlearning—a post-hoc modification method that removes specific information from trained models without requiring full retraining. Specifically, we first quantify the memorization risks of sensitive data within CLM training datasets and curate a high-risk dataset of 50,000 sensitive memorized samples as unlearning targets. We study two widely used gradient ascent-based unlearning approaches: the vanilla and constraint-based methods, and introduce CodeEraser, an advanced variant that selectively unlearns sensitive memorized segments in code while preserving the structural integrity and functional correctness of the surrounding code. Extensive experiments on three families of CLMs, i.e., CodeParrot, CodeGen-Mono, and Qwen2.5-Coder, validate the effectiveness and efficiency of CodeEraser in erasing targeted sensitive memorization while maintaining model utility.
@inproceedings{abs-2509-13755,
title = {Scrub It Out! Erasing Sensitive Memorization in Code Language Models via Machine Unlearning},
author = {Zhaoyang Chu and Yao Wan and Zhikun Zhang and Di Wang and Zhou Yang and Hongyu Zhang and Pan Zhou and Xuanhua Shi and Hai Jin and David Lo},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3764573}
}