paperbot · PL 论文追踪

全部论文 待读 收藏

P2 CrossASR++: a modular differential testing framework for automatic speech recognition

CrossASR++：面向自动语音识别的模块化差分测试框架
FSE Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering2021匹配度 3/10 · 论文关注 ASR 系统的差分测试与失败用例发现，属于机器学习系统测试/调试方向，与用户画像中 PL→AI 的静态分析、Agentic 程序、浮点程序分析或贝叶斯程序分析等核心方向均无直接交集；仅在“机器学习系统质量保障”上有边缘关联，且不涉及 LLM 或形式化验证。引用 1 inbox
Muhammad Hilmi Asyrofi, Zhou Yang, David Lo
测试、调试与修复机器学习与程序语言
crossref: 2026-09-12

AI 速览

一句话：提出模块化可扩展的 ASR 差分测试框架，通过动态分块与多种失败估计器更高效发现失败用例。

ASR 系统测试中人工收集用例成本高。CrossASR++ 是一个模块化差分测试工具：用 TTS 从文本生成音频，送入多个 ASR 交叉比对以发现失败用例，并支持灵活接入不同 TTS、ASR 和失败估计器。工具新增动态文本分块，使估计器更有效。评估显示，相同 TTS/ASR 下，它对 4 个 ASR 发现的失败用例比原工具多 26.2%；每增加一个交叉比对 ASR，4 个 ASR 的失败用例分别增加 25.07%、39.63%、20.95%、8.17%；相同时间内最佳估计器比最差多发现 10.41%。

问题
ASR 系统质量保障依赖充分测试，但人工构造或收集测试用例耗时费力，且需要高效发现会导致识别失败的输入。
方法
提出模块化差分测试框架 CrossASR++：借助 TTS 从文本生成音频，跨多个 ASR 系统比对，并动态分块文本、集成可插拔失败估计器以优先发现失败用例。
评估
在 4 个 ASR 系统上与原工具及不同失败估计器比较；结果显示失败用例发现量提升 26.2%，增加交叉比对 ASR 和采用更好估计器均能进一步增加发现数。
原文摘要（Abstract）

Developers need to perform adequate testing to ensure the quality of Automatic Speech Recognition (ASR) systems. However, manually collecting required test cases is tedious and time-consuming. Our recent work proposes CrossASR, a differential testing method for ASR systems. This method first utilizes Text-to-Speech (TTS) to generate audios from texts automatically and then feed these audios into different ASR systems for cross-referencing to uncover failed test cases. It also leverages a failure estimator to find failing test cases more efficiently. Such a method is inherently self-improvable: the performance can increase by leveraging more advanced TTS and ASR systems. So, in this accompanying tool demo paper, we further engineer CrossASR and propose CrossASR++, an easy-to-use ASR testing tool that can be conveniently extended to incorporate different TTS and ASR systems, and failure estimators. We also make CrossASR++ chunk texts from a given corpus dynamically and enable the estimator to work in a more effective and flexible way. We demonstrate that the new features can help CrossASR++ discover more failed test cases. Using the same TTS and ASR systems, CrossASR++ can uncover 26.2% more failed test cases for 4 ASRs than the original tool. Moreover, by simply adding one more ASR for cross-referencing, we can increase the number of failed test cases uncovered for each of the 4 ASR systems by 25.07%, 39.63%, 20.95% and 8.17% respectively. We also extend CrossASR++ with 5 additional failure estimators. Compared to worst estimator, the best one can discover 10.41% more failed test cases within the same amount of time. The demo video for CrossASR++ can be viewed at https://youtu.be/ddRk-f0QV-g and the source code can be found at https://github.com/soarsmu/CrossASRplus.

链接与引用

DOI 原文 · PDF（开放获取） ·

BibTeX 
@inproceedings{paperbot6329,
  title = {CrossASR++: a modular differential testing framework for automatic speech recognition},
  author = {Muhammad Hilmi Asyrofi and Zhou Yang and David Lo},
  booktitle = {FSE},
  year = {2021},
  doi = {10.1145/3468264.3473124}
}