一句话：提出 BigTest，将 UDF 内部语义与数据流/关系算子等价类结合，实现大数据分析应用的白盒测试与测试数据缩减。
数据密集型可扩展计算（DISC）系统（如 MapReduce、Hadoop、Spark）常用于云端海量数据处理。此类应用由数据流算子和复杂用户自定义函数（UDF）组成，不同于 SQL 查询，难以穷尽、自动测试。本文提出白盒测试方法 BigTest，将 UDF 内部语义与每个数据流/关系算子产生的等价类联合推理，以覆盖联合数据流与 UDF（JDU）路径并缩减测试输入。评估显示，真实 DISC 应用常存在倾斜与覆盖不足，34% 的 JDU 路径未被测试；BigTest 可将本地测试数据量缩小 10^5–10^8 倍，发现的人工注入故障比先前方法多 2 倍，仅需数十条记录即可达到全量生产数据的 JDU 覆盖，平均节省 194 倍 CPU 时间。
Data-intensive scalable computing (DISC) systems such as Google’s MapReduce, Apache Hadoop, and Apache Spark are being leveraged to process massive quantities of data in the cloud. Modern DISC applications pose new challenges in exhaustive, automatic testing because they consist of dataflow operators, and complex user-defined functions (UDF) are prevalent unlike SQL queries. We design a new white-box testing approach, called BigTest to reason about the internal semantics of UDFs in tandem with the equivalence classes created by each dataflow and relational operator. Our evaluation shows that, despite ultra-large scale input data size, real world DISC applications are often significantly skewed and inadequate in terms of test coverage, leaving 34% of Joint Dataflow and UDF (JDU) paths untested. BigTest shows the potential to minimize data size for local testing by 10^5 to 10^8 orders of magnitude while revealing 2X more manually-injected faults than the previous approach. Our experiment shows that only few of the data records (order of tens) are actually required to achieve the same JDU coverage as the entire production data. The reduction in test data also provides CPU time saving of 194X on average, demonstrating that interactive and fast local testing is feasible for big data analytics, obviating the need to test applications on huge production data.
@inproceedings{paperbot5038,
title = {White-box testing of big data analytics with complex user-defined functions},
author = {Muhammad Ali Gulzar and Shaghayegh Mardani and Madanlal Musuvathi and Miryung Kim},
booktitle = {FSE},
year = {2019},
doi = {10.1145/3338906.3338953}
}