一句话：构建240例基准，系统评测四种提示策略下四个LLM检测全部五项SOLID违反的能力，揭示需按设计情境匹配模型与提示。
传统静态分析难以检测SOLID等语义设计缺陷，需理解面向对象设计模式；现有方案多局限于单一原则或单一语言。本文用定制化提示工程评估CodeLlama:70B、DeepSeekCoder:33B、Qwen2.5 Coder:32B与GPT-4o Mini四个LLM检测全部五项SOLID违反的能力，并构建含240个人工验证样例的基准数据集。作者设计四种受zero-shot、few-shot与chain-of-thought启发的提示策略，系统测量其影响。结果显示GPT-4o Mini显著领先，但在DIP等困难原则上仍显吃力；提示策略影响巨大且无普适最优（ENSEMBLE利于OCP，EXAMPLE利于DIP），准确率还受语言特性与代码复杂度显著影响。结论：AI驱动设计分析需按设计情境匹配模型与提示。
Traditional static analysis methods struggle to detect semantic design flaws, such as violations of the SOLID principles, which require a strong understanding of object-oriented design patterns and principles. Existing solutions typically focus on individual SOLID principles or specific programming languages, leaving a gap in the ability to detect violations across all five principles in multi-language codebases. This paper presents a new approach: a methodology that leverages tailored prompt engineering to assess LLMs on their ability to detect SOLID violations across multiple languages. We present a benchmark of four leading LLMs—CodeLlama:70B, DeepSeekCoder:33B, Qwen2.5 Coder:32B, and GPT-4o Mini—on their ability to detect violations of all five SOLID principles. For this evaluation, we construct a new benchmark dataset of 240 manually validated code examples. Using this dataset, we test four distinct prompt strategies inspired by established zero-shot, few-shot, and chain-of-thought techniques to systematically measure their impact on detection accuracy. Our emerging results reveal a stark hierarchy among models, with GPT-4o Mini decisively outperforming others, yet even it struggles with challenging principles like DIP. Crucially, we show that prompt strategy has a dramatic impact, but no single strategy is universally best; for instance, a deliberative ENSEMBLE prompt excels at OCP detection while a hint-based EXAMPLE prompt is superior for DIP violations. Across all experiments, detection accuracy is heavily influenced by language characteristics and degrades sharply with increasing code complexity. These initial findings demonstrate that effective, AI-driven design analysis requires not a single "best" model, but a tailored approach that matches the right model and prompt to the specific design context, highlighting the potential of LLMs to support maintainability through AI-assisted code analysis.
DOI 原文 ·
@inproceedings{paperbot9306,
title = {Are We SOLID Yet? An Empirical Study on Prompting LLMs to Detect Design Principle Violations},
author = {Fatih Pehlivan and Arçin Ülkü Ergüzen and Sahand Moslemi Yengejeh and Mayasah Lami and Anil Koyuncu},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00350}
}