一句话：提出静态分析检测数据科学 Notebook 中训练/测试数据泄漏，并在十万级公开 Notebook 上验证其普遍性。
数据科学流水线在训练与评估机器学习模型时可能像普通代码一样引入缺陷，其中训练集与测试集之间的数据泄漏会导致离线评估高估模型精度，进而可能在生产中部署低质量模型。此类泄漏常因疏忽或不良实践产生，人工检测繁琐。作者提出一种静态分析方法，用于识别数据科学代码中常见的数据泄漏形式。评估表明该方法能准确检测泄漏，且在超过 100,000 个公开 Notebook 中发现泄漏普遍存在。文章还讨论了该分析如何帮助实践者与教育者，以及如何将防泄漏设计融入开发流程。
Data science pipelines to train and evaluate models with machine learning may contain bugs just like any other code. Leakage between training and test data can lead to overestimating the model’s accuracy during offline evaluations, possibly leading to deployment of low-quality models in production. Such leakage can happen easily by mistake or by following poor practices, but may be tedious and challenging to detect manually. We develop a static analysis approach to detect common forms of data leakage in data science code. Our evaluation shows that our analysis accurately detects data leakage and that such leakage is pervasive among over 100,000 analyzed public notebooks. We discuss how our static analysis approach can help both practitioners and educators, and how leakage prevention can be designed into the development process.