一句话：系统对比云模拟器与真实云服务的API行为，发现255个API中94个（37%）存在差异，并提出缓解技术。
云服务日益成为现代软件的重要组件，但其后端不透明且持续调用会产生费用，因此开发者常在在线测试与部署之前使用云模拟器进行离线开发与测试。本文从云软件测试的角度对云模拟器进行系统分析，目标是（1）理解云模拟在软件质量保障与部署安全方面引入的差异，（2）弥合模拟服务与真实云服务之间不可避免的鸿沟。作者对来自Azure与AWS的五个云服务、共255个API进行了仿真与真实服务的对比分析，结果令人担忧：94个（37%）API在模拟与真实服务之间存在行为差异。这些差异导致测试结果不一致，威胁部署安全，产生误报并造成可调试性问题；其根因多样，既包括偶然的实现缺陷，也包括仿真本身固有的挑战。论文讨论了潜在解决方案，并提出一种实用的缓解技术来应对云模拟器的差异问题。
Modern software projects have been increasingly using cloud services as important components. The cloud-based programming practice greatly simplifies software development by harvesting cloud benefits (e.g., high availability and elasticity). However, it imposes new challenges for software testing and analysis, due to opaqueness of cloud backends and monetary cost of invoking cloud services for continuous integration and deployment. As a result, cloud emulators are developed for offline development and testing, before online testing and deployment. This paper presents a systematic analysis of cloud emulators from the perspective of cloud-based software testing. Our goal is to (1) understand the discrepancies introduced by cloud emulation with regard to software quality assurance and deployment safety and (2) address inevitable gaps between emulated and real cloud services. The analysis results are concerning. Among 255 APIs of five cloud services from Azure and Amazon Web Services (AWS), we detected discrepant behavior between the emulated and real services in 94 (37 %) of the APIs. These discrepancies lead to inconsistent testing results, threatening deployment safety, introducing false alarms, and creating debuggability issues. The root causes are diverse, including accidental implementation defects and essential emulation challenges. We discuss potential solutions and develop a practical mitigation technique to address discrepancies of cloud emulators for software testing.
DOI 原文 ·
@inproceedings{paperbot8726,
title = {Fidelity of Cloud Emulators: The Imitation Game of Testing Cloud-Based Software},
author = {Anna Mazhar and Saad Sher Alam and William X. Zheng and Yinfang Chen and Suman Nath and Tianyin Xu},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00077}
}