一句话：提出基于 SIR 的静态检查技术，并实现首个统计学 linter stat-lint，用于发现数据科学程序中的统计最佳实践违规。
数据科学程序常误用统计推断方法，而这类误用通常不会引发崩溃或明显异常，可能让分析者得出错误结论。本文提出一种快速静态检查技术，依据统计最佳实践规则检查数据科学程序，覆盖统计方法的各项假设以及多重比较问题（data dredging）的校正。该技术以新的统计中间表示 SIR 为基础，编码对统计而言最关键的信息；作者实现首个统计学 linter 工具 stat-lint。对 90 个 Python 数据科学 notebook 的评估显示，仅 14 个完全检查了所有义务，仅 2 个应用了多重比较校正，没有一个验证模型残差，且超过三分之二的义务未被检查。
Data analysts need to be careful when they apply statistical inference techniques to data, as misuse of statistical inference methods can lead an analyst to draw the wrong conclusions. They need to be careful because, in the general case, misuse of statistics does not result in obvious problems; the numbers returned often look reasonable, and programs with misuses of statistics do not crash. In this work, we propose a technique to quickly and statically check data science programs for compliance with statistics best practice rules, including checking all assumptions made by statistical methods, as well as correcting for the multiple comparison problem, or "data dredging". This technique is predicated on a novel statistics intermediate representation, called SIR, that encodes the details most salient to statistics. We implement this technique in a tool called stat-lint, the first statistics linter, and evaluate stat-lint on 90 Python data science notebooks, finding that only 14 fully check all obligations, only two apply any correction for multiple comparisons, none validate model residuals, and over two thirds of obligations go unchecked.
DOI 原文 ·
@inproceedings{paperbot9249,
title = {The Fault in our Stats},
author = {Alexi Turcotte and Neev Nirav Mehta},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00205}
}