一句话：提出VidSumEval——面向学习效果的AI生成编程教学视频摘要评估平台与基准，并配以用户研究验证。
YouTube上的编程教程视频是常用学习资源，但篇幅过长使复习效率低下。AI摘要工具可生成更短版本，但现有评估几乎只用相似度指标衡量表面重叠，无法判断摘要是否真正支持学习。为此，作者提出VidSumEval：一个从以学习为中心视角评估AI生成编程教程摘要的Web平台与基准。基准含C、Java、Python的10个教程，配套VEED与NotebookLM生成的摘要及自动生成理解测验，并支持提交新教程离线评审。作者以被试内研究（n=20）测量测验理解与完整性、清晰性、连贯性评分，并用摘要与原视频的测验分差报告学习效果。
Programming tutorial videos on YouTube are a widely used learning resource, but their length often makes content review inefficient. AI-based summarization tools can produce shorter versions of these videos, yet existing evaluation relies almost exclusively on similarity-based metrics that measure surface overlap rather than whether the summaries actually support learning. We present VidSumEval, a web platform and benchmark for evaluating AI-generated programming tutorial summaries from a learning-centered perspective. The benchmark pairs 10 tutorials in C, Java, and Python with summaries from two commercial engines (VEED and NotebookLM) and auto-generated comprehension quizzes; users can also submit new YouTube tutorials for offline review. We validate the benchmark with a within-subject study (n=20) measuring both quiz-based comprehension and Likert-scale ratings on completeness, clarity, and coherence; learning effectiveness is reported as the per-participant difference between summarized- and original-video quiz scores. The platform is available at https://progsumeval.onrender.com; the source code and dataset are archived at https://doi.org/10.5281/zenodo.21763079; and the screencast is available at https://youtu.be/iBB-5HfRVSk.
@inproceedings{paperbot12546,
title = {VidSumEval: A Web Platform and Benchmark for Evaluating AI-Generated Programming Video Summaries},
author = {Osama AlGhamdi and Ameer AlGhamdi and Faisal Alorabi and Mohammad D. Alahmadi and Abdulmalik Alzahrani and Abdullah Almazroui},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3834594}
}