一句话:提出并落地 SCARF,自动识别并安全删除 Meta 大规模代码库中的死代码与废弃数据。
大型代码库在功能生命周期中会不断积累不再使用的代码和数据,移除它们既有利于用户隐私,也能提升工程效率,但工业界此前缺乏大规模代码废弃与死代码移除的实证。本文提出 SCARF(Systematic Code and Asset Removal Framework),一个面向大型代码库的产品废弃系统,能够识别未使用的代码与数据资产并安全移除,且可全自动执行提交代码、删除数据库表等操作;在无法自动处理时收集开发者输入以推动后续移除。SCARF 在 Meta 已产生重要影响:仅过去一年就跨 1280 万个不同资产移除了数 PB 数据,并删除了超过 1.04 亿行代码。
Software constantly evolves in response to user needs: new features are built, deployed, mature and grow old, and eventually their usage drops enough to merit switching them off. In any large codebase, this feature lifecycle can naturally lead to retaining unnecessary code and data. Removing these respects users’ privacy expectations, as well as helping engineers to work efficiently. In prior software engineering research, we have found little evidence of code deprecation or dead-code removal at industrial scale. We describe Systematic Code and Asset Removal Framework (SCARF), a product deprecation system to assist engineers working in large codebases. SCARF identifies unused code and data assets and safely removes them. It operates fully automatically, including committing code and dropping database tables. It also gathers developer input where it cannot take automated actions, leading to further removals. Dead code removal increases the quality and consistency of large codebases, aids with knowledge management and improves reliability. SCARF has had an important impact at Meta. In the last year alone, it has removed petabytes of data across 12.8 million distinct assets, and deleted over 104 million lines of code.
@inproceedings{ShackletonCR0GN23,
title = {Dead Code Removal at Meta: Automatically Deleting Millions of Lines of Code and Petabytes of Deprecated Data},
author = {Will Shackleton and Katriel Cohn-Gordon and Peter C. Rigby and Rui Abreu and James Gill and Nachiappan Nagappan and Karim Nakad and Ioannis Papagiannis and Luke Petre and Giorgi Megreli and Patrick Riggs and James Saindon},
booktitle = {FSE},
year = {2023},
doi = {10.1145/3611643.3613871}
}