一句话：首次提出 ActionNet，用图像差分与 CNN 从编程截屏识别九类开发者工作流动作，并用于关键代码帧提取。
编程截屏可用于研究开发者行为、信息需求与知识传播，但图像数据难以分析和索引；现有方法只提取内容，忽略开发者完成任务的工作流动作。本文首次提出从编程截屏中识别工作流动作的技术 ActionNet，利用图像差分和卷积神经网络分析连续帧的对应与变化，识别九类常见开发者动作。作者使用 YouTube 编程截屏评估不同 CNN 配置，并测试跨开发者、工作环境和编程语言的识别性能；在真实工作截屏中展示其用于动作感知的关键代码帧提取应用。
Programming screencasts have two important applications in software engineering context: study developer behaviors, information needs and disseminate software engineering knowledge. Although programming screencasts are easy to produce, they are not easy to analyze or index due to the image nature of the data. Existing techniques extract only content from screencasts, but ignore workflow actions by which developers accomplish programming tasks. This significantly limits the effective use of programming screencasts in downstream applications. In this paper, we are the first to present a novel technique for recognizing workflow actions in programming screencasts. Our technique exploits image differencing and Convolutional Neural Network (CNN) to analyze the correspondence and change of consecutive frames, based on which nine classes of frequent developer actions can be recognized from programming screencasts. Using programming screencasts from Youtube, we evaluate different configurations of our CNN model and the performance of our technique for developer action recognition across developers, working environments and programming languages. Using screencasts of developers’ real work, we demonstrate the usefulness of our technique in a practical application for actionaware extraction of key-code frames in developers’ work.
@inproceedings{ZhaoXC0019,
title = {ActionNet: Vision-Based Workflow Action Recognition From Programming Screencasts},
author = {Dehai Zhao and Zhenchang Xing and Chunyang Chen and Xin Xia and Guoqiang Li},
booktitle = {ICSE},
year = {2019},
doi = {10.1109/icse.2019.00049}
}