一句话：提出首个航空航天时序异常检测基准 ATSADBench，系统评测 LLM 的检测能力与知识注入策略。
时间序列异常检测（TSAD）对航空航天软件的安全可靠至关重要，但 LLM 在此场景的有效性尚缺检验。论文提出首个航空航天 TSAD 基准 ATSADBench，由 3 种模式异常、单/多变量信号与环内/环外反馈组合成 9 类任务、共 108,000 个数据点。作者在 Direct（窗口内直接标注）与 Prediction-Based（基于预测误差）两种范式下评测开源 LLM，将评估下沉到窗口级，并提出告警准确率 AA、延迟 AL 与连续性 AC 三项面向用户的指标，同时考察 few-shot 与 RAG 两种知识注入策略。结果表明：LLM 在单变量任务上表现良好，多变量遥测上 AA 与 AC 接近随机猜测；few-shot 增益有限、RAG 无显著提升；LLM 能捕捉真实异常起始却易误报，few-shot 可缓解而 RAG 反而加剧。
Time series anomaly detection (TSAD) is essential for ensuring the safety and reliability of aerospace software systems. Although large language models (LLMs) provide a promising training-free alternative to unsupervised approaches, their effectiveness in aerospace settings remains under-examined because of complex telemetry, misaligned evaluation metrics, and the absence of domain knowledge. To address this gap, we introduce ATSADBench, the first benchmark for aerospace TSAD. ATSADBench comprises nine tasks that combine three pattern-wise anomaly types, univariate and multivariate signals, and both in-loop and out-of-loop feedback scenarios, yielding 108,000 data points. Using this benchmark, we systematically evaluate state-of-the-art open-source LLMs under two paradigms: Direct, which labels anomalies within sliding windows, and Prediction-Based, which detects anomalies from prediction errors. To reflect operational needs, we reformulate evaluation at the window level and propose three user-oriented metrics: Alarm Accuracy (AA), Alarm Latency (AL), and Alarm Contiguity (AC), which quantify alarm correctness, timeliness, and credibility. We further examine two enhancement strategies, few-shot learning and retrieval-augmented generation (RAG), to inject domain knowledge. The evaluation results show that (1) LLMs perform well on univariate tasks but struggle with multivariate telemetry, (2) their AA and AC on multivariate tasks approach random guessing, (3) few-shot learning provides modest gains whereas RAG offers no significant improvement, and (4) in practice LLMs can detect true anomaly onsets yet sometimes raise false alarms, which few-shot prompting mitigates but RAG exacerbates. These findings offer guidance for future LLM-based TSAD in aerospace software.
DOI 原文 ·
@inproceedings{paperbot9362,
title = {Evaluating Large Language Models for Time Series Anomaly Detection in Aerospace Software},
author = {Yang Liu and Yixing Luo and Xiaofeng Li and Xiaogang Dong and Bin Gu and Zhi Jin},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00291}
}