一句话：提出 DL Contract，为深度学习库引入契约式设计，支持指定模型架构、数据与训练过程属性，在开发阶段检测缺陷。
深度学习技术被广泛用于关键系统，但其软件同样存在缺陷。现有工作多聚焦于缺陷刻画、修复模式与检测定位，本文提出一种预防性手段：为深度学习库引入契约式设计（DL Contract）。由于训练过程的属性无法在函数接口中体现，作者提供机制让开发者描述模型架构、数据和训练过程属性，并在 Python 深度学习库上实现，用以刻画 Keras 的契约。评估中，作者针对训练问题和结构缺陷编写 15 个示例契约，并在四个既有基准上测试：DL Contract 在 272 个真实缺陷程序中正确检测出 259 个缺陷，运行时开销较小；二十名参与者的可用性调查也表明其对调试有帮助。
Deep Learning (DL) techniques are increasingly being incorporated in critical software systems today. DL software is buggy too. Recent work in SE has characterized these bugs, studied fix patterns, and proposed detection and localization strategies. In this work, we introduce a preventative measure. We propose design by contract for DL libraries, DL Contract for short, to document the properties of DL libraries and provide developers with a mechanism to identify bugs during development. While DL Contract builds on the traditional design by contract techniques, we need to address unique challenges. In particular, we need to document properties of the training process that are not visible at the functional interface of the DL libraries. To solve these problems, we have introduced mechanisms that allow developers to specify properties of the model architecture, data, and training process. We have designed and implemented DL Contract for Python-based DL libraries and used it to document the properties of Keras, a well-known DL library. We evaluate DL Contract in terms of effectiveness, runtime overhead, and usability. To evaluate the utility of DL Contract, we have developed 15 sample contracts specifically for training problems and structural bugs. We have adopted four well-vetted benchmarks from prior works on DL bug detection and repair. For the effectiveness, DL Contract correctly detects 259 bugs in 272 real-world buggy programs, from well-vetted benchmarks provided in prior work on DL bug detection and repair. We found that the DL Contract overhead is fairly minimal for the used benchmarks. Lastly, to evaluate the usability, we conducted a survey of twenty participants who have used DL Contract to find and fix bugs. The results reveal that DL Contract can be very helpful to DL application developers when debugging their code.
@inproceedings{AhmedISCR23,
title = {Design by Contract for Deep Learning APIs},
author = {Shibbir Ahmed and Sayem Mohammad Imtiaz and Samantha Syeda Khairunnesa and Breno Dantas Cruz and Hridesh Rajan},
booktitle = {FSE},
year = {2023},
doi = {10.1145/3611643.3616247}
}