一句话:提出用户态 Linux 可复现容器 DetTrace,使容器内计算成为初始文件系统状态的纯函数,并自动消除构建与工作流中的不可复现性。
软件构建与数据工作流常因执行中的非确定性而不可复现,而现有可复现方案往往需要硬件、操作系统或应用改动。本文设计并实现 DetTrace,一个在用户空间实现的 Linux 可复现容器抽象,使容器内所有计算都成为其初始文件系统状态的纯函数,从而自动获得可复现性。作者用 DetTrace 对 12,130 个 Debian 包构建(超过 8 亿行代码)、生物信息学与机器学习工作流进行验证;这些软件最初不可复现,DetTrace 无需硬件、OS 或应用改动即可恢复可复现性。性能取决于系统调用频率:IO 密集型构建平均开销为 3.49 倍,而计算密集型生物信息学工作流低于 2%。
We describe the design and implementation of DetTrace, a reproducible container abstraction for Linux implemented in user space. All computation that occurs inside a DetTrace container is a pure function of the initial filesystem state of the container. Reproducible containers can be used for a variety of purposes, including replication for fault-tolerance, reproducible software builds and reproducible data analytics. We use DetTrace to achieve, in an automatic fashion, reproducibility for 12,130 Debian package builds, containing over 800 million lines of code, as well as bioinformatics and machine learning workflows. We show that, while software in each of these domains is initially irreproducible, DetTrace brings reproducibility without requiring any hardware, OS or application changes. DetTrace's performance is dictated by the frequency of system calls: IO-intensive software builds have an average overhead of 3.49x, while a compute-bound bioinformatics workflow is under 2%.
@inproceedings{paperbot5427,
title = {Reproducible Containers},
author = {Omar S. Navarro Leija and Kelly Shiptoski and Ryan G. Scott and Baojun Wang and Nicholas Renner and Ryan R. Newton and Joseph Devietti},
booktitle = {ASPLOS},
year = {2020},
doi = {10.1145/3373376.3378519}
}