一句话：评估四种 LLM 在 DSA 问题上的正确性、可维护性与解释推理，发现其为推理伙伴但尚不能自主解题。
随着 LLM 进入软件工程教育，学生常借其辅导数据结构与算法（DSA），但尚不清楚它培养真正问题求解能力还是仅提供现成答案，且其在受污染基准之外的 DSA 能力仍缺乏研究。本文评估 GPT-4o、Claude 3.7 Sonnet、Llama 3.2 与 DeepSeek R1，在 80 道 LeetCode Top Interview 150 和 100 道近期周赛题上，从代码正确性、可维护性和解释性推理三个维度考察。结果显示，Claude 与 DeepSeek 在难题上最好，但所有模型在新题上平均成功率下降约 49%。DeepSeek 新题成功率最高（70%）且提示轮次更少，却代码可维护性最差；Claude 成功率较低（50%）但解法最可维护，体现正确性与教学价值的权衡。研究表明 LLM 是推理伙伴，但不足以自主解题，并给出教育整合建议。
Large Language Models (LLMs) are rapidly entering software engineering education. As students turn to them for Data Structures and Algorithms (DSA) tutoring, a critical question emerges: do they foster genuine problem-solving skills or merely supply polished answers? Moreover, their competence in DSA remains underexplored beyond benchmarks likely contaminated with training data. We evaluate four LLMs—GPT-4o, Claude 3.7 Sonnet, Llama 3.2, and DeepSeek R1—on 80 LeetCode Top Interview 150 problems and 100 recent Weekly Contest problems, assessing code correctness, maintainability, and explanatory reasoning as core software engineering (SE) values. Results show that while Claude and DeepSeek perform best on harder problems, all models’ average success rates drop by ∼ 49% on novel problems, revealing a critical gap in translating theoretical insights into effective implementations. On the novel problems, DeepSeek attained the highest success rate (70%) with fewer prompt turns but produced the least maintainable code, whereas Claude solved fewer (50%) yet generated the most maintainable solutions, highlighting a trade-off between correctness and pedagogical value. These findings suggest that LLMs are capable reasoning partners yet insufficient for autonomous problem-solving. Thus, educators must carefully integrate these tools into curricula to emphasize students’ critical reasoning and debugging skills, and develop assessments that leverage or withstand LLM assistance. Our work contributes a rigorous, multidimensional evaluation framework and practical recommendations for AI-assisted learning in SE and computer science (CS) education, underscoring critical evaluation over reliance on automated solutions.
@inproceedings{paperbot12059,
title = {Beyond Answer Engines: LLMs as Reasoning Partners in Data Structures and Algorithms Education},
author = {Saad Zafar Khan and Desiree Leal and Lucas Valença and Ahmad Abdellatif and Mea Wang and Diwakar Krishnamurthy and Ronnie de Souza Santos},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3786580.3786972}
}