一句话：在自动化编程评估系统 Artemis 中集成 GPT-3.5-Turbo 作为 AI 导师，并通过探索性案例研究分析学生交互模式与体验。
随着 AI 进入教育领域，LLM 在自动化编程评估系统（APAS）中作为 AI 导师的效果尚缺乏科学评估。本文在 APAS Artemis 中集成 GPT-3.5-Turbo 作为 AI 导师，开展探索性案例研究：结合经验数据收集与探索性问卷调查，分析学生如何与该 AI 导师交互。研究根据交互模式识别出不同用户类型，并总结其优势（如及时反馈与可扩展性）和挑战（如回答过于泛化、学生担心抑制学习进步）。结果补充了 AI 在教育中角色的讨论，并为 APAS 中 LLM 导师的设计与评估提供经验证据。
With the rapid advancement of artificial intelligence (AI) in various domains, the education sector is set for transformation. The potential of AI-driven tools in enhancing the learning experience, especially in programming, is immense. However, the scientific evaluation of Large Language Models (LLMs) used in Automated Programming Assessment Systems (APASs) as an AI-Tutor remains largely unexplored. Therefore, there is a need to understand how students interact with such AI-Tutors and to analyze their experiences. In this paper, we conducted an exploratory case study by integrating the GPT-3.5-Turbo model as an AI-Tutor within the APAS Artemis. Through a combination of empirical data collection and an exploratory survey, we identified different user types based on their interaction patterns with the AI-Tutor. Additionally, the findings highlight advantages, such as timely feedback and scalability. However, challenges like generic responses and students' concerns about a learning progress inhibition when using the AI-Tutor were also evident. This research adds to the discourse on AI's role in education.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{FrankfordSBKB24,
title = {AI-Tutoring in Software Engineering Education},
author = {Eduard Frankford and Clemens Sauerwein and Patrick Bassner and Stephan Krusche and Ruth Breu},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3639474.3640061}
}