一句话：提出块测试（block tests），将内联测试从单语句推广到任意代码片段，并为 Java 实现 BDK 工具包。
内联测试可验证单条程序语句，能发现单元测试遗漏的单语句缺陷，但其粒度局限使其无法验证跨多条语句的数据流或控制流。本文提出块测试，将内联测试推广到任意代码片段，并给出指定输入、预期输出与作用域的语法和语义，以及用于 Java 的块测试开发套件 BDK。作者在 146 个开源项目的 346 个代码片段上编写了 1,012 个块测试；开发者单元测试未覆盖其中 58.7% 的片段，自动单元测试生成在 30.8 CPU 天后仍无法到达 46% 的片段。平均每个块测试编写耗时 2.2 分钟、运行 0.9 秒。变异测试表明，块测试杀死了单元测试存活变异体中的 4,418/9,554 个。结果初步但有力地证明了块测试的可行性与实用性。
Inline tests validate single program statements and were shown to find single-statement bugs or kill mutants that unit tests miss. Inline tests complement unit tests by enabling testing at a finer program granularity level than methods. So, inline tests can more easily find faults in target statements that unit tests do not reach, or where errors do not propagate to unit tests’ oracles. But, the limitation to single statements means inline tests cannot validate data or control flow across code fragments—sequences of multiple statements in a method. We motivate the need for testing arbitrary fragments and propose block tests, which generalize inline tests and validate code fragments. To motivate, we discuss six software testing needs (e.g., due to increasing usage of lambdas in imperative code) for which unit tests are too coarse grained and inline tests are too fine grained. To bridge this gap, we propose syntax and semantics for specifying inputs, expected outputs, and scope of block tests. We also implement a block-test development kit (BDK) for writing and running block tests in Java. We evaluate block tests and BDK in two ways. First, we write 1,012 block tests for 346 fragments in 146 open-source projects. Developer written unit tests do not cover 58.7% of these fragments, and automated unit-test generation does not reach 46% of them even after 30.8 CPU days. But, each block test takes us 2.2 minutes to write and 0.9 seconds to run on average. Second, we use mutation testing to evaluate the fault-finding effectiveness of block tests in fragments that unit tests cover. Block tests kill 4,418 of 9,554 mutants that survived unit tests. These results provide initial but strong evidence on block tests’ feasibility and utility. We outline an agenda for future research on block testing.