一句话：提出以代码 diff 特征为核心、用 LLM 做多语言特征提取的部署风险预测框架，跨数据集 F1 达 0.81。
Prime Video 在直播活动与快速迭代中面临部署风险，而现行变更管控采用一刀切冻结，既有风险预测又依赖开发者元数据与大量历史数据，存在隐私顾虑且难以迁移到新项目。本文提出以 diff-aware 特征为核心的部署风险评估框架：特征直接取自代码改动，涵盖代码级与变更级定量度量，以及编码风格违规、变更类型分类等定性指标；并以 LLM 充当多语言特征提取器，免去语言特定工具。作者在 Prime Video 生产环境与公开数据集 ApacheJIT 上评估，最佳模型检测风险变更的平均召回率 0.83、F1 0.81；消融分析显示增删行数等变更级容量指标噪声大，结构化代码复杂度才是更强的风险信号。
At Amazon Prime Video, we face the critical operational challenge of managing code deployments during live events and rapid feature releases without causing service outages. Current change control approaches use blanket deployment freezes that block all changes regardless of risk, creating significant developer toil. While prior research has explored risky change predictors, these rely on developer-specific metadata or extensive historical data, raising privacy concerns and limiting applicability to new projects. We introduce a framework centered on diff-aware features-characteristics derived directly from code modifications. Our key contribution is the systematic identification of which quantitative metrics (code-level and change-level metrics) and qualitative indicators (coding style violations, change type classification) are necessary for risk prediction. We employ LLMs as multi-language feature extractors, demonstrating their effectiveness for code analysis beyond generation tasks and eliminating the need for language-specific tooling. We evaluated our framework on two datasets: Prime Video’s production environment and the public ApacheJIT dataset. Our best-performing model achieves an average recall of 0.83 and F1 score of 0.81 across both datasets for detecting risky code changes. Notably, ablation analysis reveals that change-level volume metrics (e.g., lines added/deleted) are noisy predictors, while structural code complexity provides a substantially stronger risk signal. These results demonstrate that thoughtful feature curation enables effective change risk assessment across different programming languages and organizational contexts while avoiding privacy concerns.
@inproceedings{abs-2607-06766,
title = {Deployment Risk Assessment using Diff-Aware Features: A Case Study at Prime Video},
author = {Mayur Premkumar Kurup and Hyunjae Suh and Swathi Vaidyanathan and Pranesh Vyas and Srinidhi Madabhushi and Yegor Silyutin},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3834517}
}