一句话：提出DLJSFuzzer，通过面向缓存复用与推理加速的变异规则对JavaScript深度学习框架进行模糊测试，发现大量崩溃和数值缺陷。
随着JavaScript环境中深度学习应用日益流行，JavaScript深度学习框架的质量保障变得至关重要。然而，受限于计算资源，这类框架通常采用缓存复用、推理加速等优化机制，而现有测试方法忽视了这些机制，导致许多缺陷被遗漏。本文提出基于变异的JavaScript深度学习框架测试方法DLJSFuzzer，针对缓存复用机制设计13条张量变异规则以生成测试输入张量，并针对推理加速机制设计8条模型变异规则以生成测试输入模型。作者在广泛使用的TensorFlow.js上进行实验，结果表明DLJSFuzzer在有效性和效率上均优于现有方法：成功检测到21个独特崩溃和126个独特NaN与不一致缺陷，其中12个崩溃已被开发者确认；相比所有基线，模型生成效率提升超过47%，缺陷检测效率提升超过91%。
In recent years, Deep Learning (DL) applications in JavaScript environment have become increasingly popular. As the infrastructure for DL applications, JavaScript DL frameworks play a crucial role in the development and deployment. It is essential to ensure the quality of JavaScript DL frameworks. However, the bottleneck of limited computational resources in the JavaScript environment brings new challenges to framework testing. Specifically, JavaScript DL frameworks are equipped with various optimization mechanisms (e.g., cache reuse, inference acceleration) to overcome the bottleneck of limited computational resources. These optimization mechanisms are overlooked by existing methods, resulting in many bugs in JavaScript DL frameworks being missed. To address the above challenges, we propose a mutation-based JavaScript DL framework testing method named DLJSFuzzer. DLJSFuzzer designs 13 tensor mutation rules targeting the cache reuse mechanism to generate test input tensors. Besides, DLJSFuzzer designs eight model mutation rules targeting the inference acceleration mechanism to generate test input models. To evaluate the effectiveness of DLJS-Fuzzer, we conduct experiments on the most widely-used JavaScript DL framework, TensorFlow.js. The experimental results show that DLJSFuzzer outperforms state-of-the-art methods in both effectiveness and efficiency. DLJSFuzzer successfully detects 21 unique crashes and 126 unique NaN & Inconsistency bugs. All detected crashes have been reported to the open-source community, with 12 of them already confirmed by developers. Additionally, DLJSFuzzer has improved by over 47% in model generation efficiency and over 91% in bug detection efficiency compared to all baselines.
@inproceedings{ZouZFLZ024,
title = {Mutation-Based Deep Learning Framework Testing Method in JavaScript Environment},
author = {Yinglong Zou and Juan Zhai and Chunrong Fang and Jiawei Liu and Tao Zheng and Zhenyu Chen},
booktitle = {ASE},
year = {2024},
doi = {10.1145/3691620.3695478}
}