一句话：提出 Idaka 工具，用 BM25 信息检索与 Alpaca 大模型为软件工程师推荐机器学习最佳实践。
机器学习已广泛用于自动驾驶、医疗诊断等场景，软件工程任务也不例外。然而，不遵循良好的 ML 实践可能导致系统性能下降甚至产生意外结果。尽管存在相关文档与文献，许多非 ML 专家仍倾向于从博客、问答系统等灰色文献中寻找指导，难以提炼出与自身情境相关的知识。为此，作者提出基于用户上下文的 ML 实践推荐系统，并实现了 Idaka 工具作为第一步。Idaka 提供两种获取或生成最佳实践的途径：基于 BM25 的信息检索引擎，以及大语言模型 Alpaca。该平台被设计为支持最佳实践检索工具的对比研究，已在 GitHub 公开并提供演示视频。
Machine learning (ML) is nowadays widely used for different purposes and with several disciplines. From self-driving cars to automated medical diagnosis, machine learning models extensively support users’ daily activities, and software engineering tasks are no exception. Not embracing good ML practices may lead to pitfalls that hinder the performance of an ML system and potentially lead to unexpected results. Despite the existence of documentation and literature about ML best practices, many non-ML experts turn towards gray literature like blogs and Q&A systems when looking for help and guidance when implementing ML systems. To better aid users in distilling relevant knowledge from such sources, we propose a recommender system that recommends ML practices based on the user’s context. As a first step in creating a recommender system for machine learning practices, we implemented Idaka. A tool that provides two different approaches for retrieving/generating ML best practices: i) an information retrieval (IR) engine and ii) a large language model. The IR-engine uses BM25 as the algorithm for retrieving the practices, and a large language model, in our case Alpaca. The platform has been designed to allow comparative studies of best practices retrieval tools. Idaka is publicly available at GitHub: https://bit.ly/idaka. Video: https://youtu.be/cEb-AhIPxnM
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2308-12095,
title = {On Using Information Retrieval to Recommend Machine Learning Good Practices for Software Engineers},
author = {Laura Cabra-Acela and Anamaria Mojica-Hanke and Mario Linares-Vásquez and Steffen Herbold},
booktitle = {FSE},
year = {2023},
doi = {10.1145/3611643.3613093}
}