一句话：提出 TorchQL，用查询语言表达并检查 ML 模型与数据集上的完整性约束，提升错误发现效率。
机器学习应用的错误排查需要系统探索模型与数据行为，但现有实践常为临时脚本且缺乏可扩展抽象。TorchQL 是一个编程框架，用户可编写查询来规约并检查机器学习模型和数据集上的完整性约束；它将关系代数与函数式编程无缝集成，仅用八个算子即可表达复杂查询。作者在自动驾驶视频帧检测的时序不一致、医疗时序记录插补错误、真实图像标注错误以及语言模型偏见与输出约束等用例上评估，结果显示查询执行比 Pandas 和 MongoDB 快至 13 倍，查询比原生 Python 短至 40%，用户研究也表明熟悉 Python 的开发者能用其自然表达复杂完整性约束。
Finding errors in machine learning applications requires a thorough exploration of their behavior over data. Existing approaches used by practitioners are often ad-hoc and lack the abstractions needed to scale this process. We present TorchQL, a programming framework to evaluate and improve the correctness of machine learning applications. TorchQL allows users to write queries to specify and check integrity constraints over machine learning models and datasets. It seamlessly integrates relational algebra with functional programming to allow for highly expressive queries using only eight intuitive operators. We evaluate TorchQL on diverse use-cases including finding critical temporal inconsistencies in objects detected across video frames in autonomous driving, finding data imputation errors in time-series medical records, finding data labeling errors in real-world images, and evaluating biases and constraining outputs of language models. Our experiments show that TorchQL enables up to 13x faster query executions than baselines like Pandas and MongoDB, and up to 40% shorter queries than native Python. We also conduct a user study and find that TorchQL is natural enough for developers familiar with Python to specify complex integrity constraints.
@article{paperbot2530,
title = {TorchQL: A Programming Framework for Integrity Constraints in Machine Learning},
author = {Aaditya Naik and Adam Stein and Yinjun Wu and Mayur Naik and Eric Wong},
journal = {Proceedings of the ACM on Programming Languages},
volume = {8},
number = {OOPSLA1},
year = {2024},
doi = {10.1145/3649841}
}