一句话：系统映射 100 项研究，揭示游戏化评估缺乏标准模型，多聚焦策略体验或结果影响。
本文针对软件工程教育与实践中游戏化评估缺乏标准程序与模型的问题，开展系统映射研究，分析 2011–2020 年间 100 项游戏化研究。研究从评估程序/模型、评估目的、准则、数据类型、工具与数据分析方法等维度分类。结果显示，64 项研究报告了评估程序，但仅 3 项提出评估模型。评估主要关注两方面：游戏化策略本身（用户体验与感知）以及游戏化对用户和情境的结果与影响。最常用准则包括参与度、绩效、满意度和动机。评估需混合主客观输入与定性定量分析，且依评估焦点不同，数据类型与分析方法各有侧重。
Context: gamification has been used to motivate and engage participants in software engineering education and practice activities. Problem: There is a significant demand for empirical studies for the understanding of the impacts and efficacy of gamification. However, the lack of standard procedures and models for the evaluation of gamification is a challenge for the design, comparison, and report of results related to the assessment of gamification approaches and its effects. Goal: The goal of this study is to identify models and strategies for the evaluation of gamification reported in the literature. Method: To achieve this goal, we conducted a systematic mapping study to investigate strategies for the evaluation of gamification in the context of software engineering. We selected 100 primary studies on gamification in software engineering (from 2011 to 2020). We categorized the studies regarding the presence of evaluation procedures or models for the evaluation of gamification, the purpose of the evaluation, the criteria used, the type of data, instruments, and procedures for data analysis. Results: Our results show that 64 studies report procedures for the evaluation of gamification. However, only three studies actually propose evaluation models for gamification. We observed that the evaluation of gamification focuses on two aspects: the evaluation of the gamification strategy itself, related to the user experience and perceptions; and the evaluation of the outcomes and effects of gamification on its users and context. The most recurring criteria for the evaluation are "engagement", "performance", "satisfaction", and "motivation". Finally, the evaluation of gamification requires a mix of subjective and objective inputs, and qualitative and quantitative data analysis approaches. Depending of the focus of the evaluation (the strategy or the outcomes), there is a predominance of a type of data and analysis.
@inproceedings{paperbot6271,
title = {The Diversity of Gamification Evaluation in the Software Engineering Education and Industry: Trends, Comparisons and Gaps},
author = {Rodrigo Henrique Barbosa Monteiro and Maurício Ronny de Almeida Souza and Sandro Ronaldo Bezerra Oliveira and Carlos dos Santos Portela and Cesar Elias de Cristo Lobato},
booktitle = {ICSE},
year = {2021},
doi = {10.1109/icse-seet52601.2021.00025}
}