The automotive industry is increasingly exploring vision-language models (VLMs) for in-car scene understanding (ISU), extracting semantic information from camera-recorded scenes to detect safety-critical events such as driver distraction. However, VLMs may generate incomplete, erroneous, or misleading scene descriptions, highlighting the need for systematic testing. Collecting real in-vehicle data is costly, difficult to scale, and often infeasible, particularly in early design stages. In this paper, we present ISU-Test, an automated testing approach that combines rendering-based scene generation with search-based testing to evaluate ISU systems. By framing testing as an optimization problem and systematically modifying scene parameters, our method generates diverse in-car scenarios and explores a wide range of configurations. We evaluate ISU-Test on both an industrial prototype and open-source VLMs across two case studies: question answering and captioning, comparing against randomized scenario generation. Results show that ISU-Test significantly outperforms the baseline, achieving up to 10× higher failure rates and up to 3.6× higher coverage.

@inproceedings{abs-2607-02300,
  title = {Search-Based Testing of Vision Language Models for In-Car Scene Understanding},
  author = {Lev Sorokin and Chen Yang and Ken E. Friedl and Andrea Stocco},
  booktitle = {ASE},
  year = {2026},
  doi = {10.1145/3832783.3834506}
}