一句话：提出结合建模事件日志与LLM上下文学习的框架，用于生成建模操作合成轨迹并推荐相关操作。
模型驱动工程中，构建精确模型易出错且依赖领域知识；现有智能建模助手需大量训练数据，常因隐私等问题难以获得。本文提出一个概念框架，结合建模事件日志、智能建模助手与基于大语言模型的建模操作生成，在图形化建模环境中记录操作并自动推荐相关操作。作者利用主流LLM生成新的建模事件数据集，并在欧洲项目的工业建模工具链上做概念验证。评估用距离度量检验生成操作真实性，并以真实工业建模制品评价推荐操作。结果表明LLM可生成建模事件，但人工操作准确率更高；数据缺乏时生成式AI可作为替代方案。
Producing accurate software models is crucial in model-driven software engineering (MDE). However, modeling complex systems is an error-prone task that requires deep application domain knowledge. In the past decade, several automated techniques have been proposed to support academic and industrial practitioners by providing relevant modeling operations. Nevertheless, those techniques require a huge amount of training data that cannot be available due to several factors, e.g., privacy issues. The advent of large language models (LLMs) can support the generation of synthetic data although state-of-the-art approaches are not yet supporting the generation of modeling operations. To fill the gap, we propose a conceptual framework that combines modeling event logs, intelligent modeling assistants, and the generation of modeling operations using LLMs. In particular, the architecture comprises modeling components that help the designer specify the system, record its operation within a graphical modeling environment, and automatically recommend relevant operations. In addition, we generate a completely new dataset of modeling events by telling on the most prominent LLMs currently available. As a proof of concept, we instantiate the proposed framework using a set of existing modeling tools employed in industrial use cases within different European projects. To assess the proposed methodology, we first evaluate the capability of the examined LLMs to generate realistic modeling operations by relying on well-founded distance metrics. Then, we evaluate the recommended operations by considering real-world industrial modeling artifacts. Our findings demonstrate that LLMs can generate modeling events even though the overall accuracy is higher when considering human-based operations. In this respect, we see generative AI tools as an alternative when the modeling operations are not available to train traditional IMAs specifically conceived to support industrial practitioners.
DOI 原文 ·
@inproceedings{paperbot8430,
title = {Towards Synthetic Trace Generation of Modeling Operations using In-Context Learning Approach},
author = {Vittoriano Muttillo and Claudio Di Sipio and Riccardo Rubei and Luca Berardinelli and MohammadHadi Dehghani},
booktitle = {ASE},
year = {2024},
doi = {10.1145/3691620.3695058}
}