一句话：提出 SapientML，通过三阶段分治程序合成从语料中学习，为新的预测任务高效生成高质量 ML 流水线。
AutoML 能自动化数据科学家工作，但候选流水线组合搜索空间巨大，现有方法在大型复杂数据集上常生成次优流水线甚至失败。SapientML 从现有数据集及其人工流水线语料中学习，采用三阶段分治式程序合成逐步缩小搜索空间：元学习预测合理 ML 组件，流水线数据流模型细化出少量可行流水线，动态评估选出最佳方案。作者挖掘 Kaggle 构建 170 个数据集、1094 条流水线的语料，在 41 个基准（含 10 个大型真实 Kaggle 数据集）上与 3 个先进 AutoML 工具和 4 个基线对比。结果显示 SapientML 在 27 个基准上最佳或相当，并在 10 个最具挑战基准中 9 个获胜。
Automatic machine learning, or AutoML, holds the promise of truly democratizing the use of machine learning (ML), by substantially automating the work of data scientists. However, the huge combinatorial search space of candidate pipelines means that current AutoML techniques, generate sub-optimal pipelines, or none at all, especially on large, complex datasets. In this work we propose an AutoML technique SapientML, that can learn from a corpus of existing datasets and their human-written pipelines, and efficiently generate a high-quality pipeline for a predictive task on a new dataset. To combat the search space explosion of AutoML, SapientML employs a novel divide-and-conquer strategy realized as a three-stage program synthesis approach, that reasons on successively smaller search spaces. The first stage uses meta-learning to predict a set of plausible ML components to constitute a pipeline. In the second stage, this is then refined into a small pool of viable concrete pipelines using a pipeline dataflow model derived from the corpus. Dynamically evaluating these few pipelines, in the third stage, provides the best solution. We instantiate SapientML as part of a fully automated tool-chain that creates a cleaned, labeled learning corpus by mining Kaggle, learns from it, and uses the learned models to then synthesize pipelines for new predictive tasks. We have created a training corpus of 1,094 pipelines spanning 170 datasets, and evaluated SapientML on a set of 41 benchmark datasets, including 10 new, large, real-world datasets from Kaggle, and against 3 state-of-the-art AutoML tools and 4 baselines. Our evaluation shows that SapientML produces the best or comparable accuracy on 27 of the benchmarks while the second best tool fails to even produce a pipeline on 9 of the instances. This difference is amplified on the 10 most challenging benchmarks, where SapientML wins on 9 instances with the other tools failing to produce pipelines on 4 or more benchmarks.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2202-10451,
title = {SapientML},
author = {Ripon K. Saha and Akira Ura and Sonal Mahajan and Chenguang Zhu and Linyi Li and Yang Hu and Hiroaki Yoshida and Sarfraz Khurshid and Mukul R. Prasad},
booktitle = {ICSE},
year = {2022},
doi = {10.1145/3510003.3510226}
}