一句话：提出 TrustVis，一个自动化多维评估框架，通过集成扰动方法与多数投票，可视化评估 LLM 的安全性与鲁棒性。
大语言模型在 NLP 应用中不断取得突破，但其可信度尤其是安全性与鲁棒性仍令人担忧。为应对该问题，本文提出 TrustVis——一个自动化评估框架，可对 LLM 可信度进行全面评估。其核心特性是交互式用户界面，能以直观可视化方式呈现可信度指标。方法上，TrustVis 集成 AutoDAN 等知名扰动方法，并在多种评估方法间采用多数投票，从而提供可靠结果并降低复杂评估流程的使用门槛。在 Vicuna-7b、Llama2-7b 和 GPT-3.5 上的初步案例研究显示，该框架能有效识别安全与鲁棒性漏洞，交互界面还允许用户详细探索结果，以支持有针对性的模型改进。
As Large Language Models (LLMs) continue to revolutionize Natural Language Processing (NLP) applications, critical concerns about their trustworthiness persist, particularly in safety and robustness. To address these challenges, we introduce TrustVis, an automated evaluation framework that provides a comprehensive assessment of LLM trustworthiness. A key feature of our framework is its interactive user interface, designed to offer intuitive visualizations of trustworthiness metrics. By integrating well-known perturbation methods like AutoDAN and employing majority voting across various evaluation methods, TrustVis not only provides reliable results but also makes complex evaluation processes accessible to users. Preliminary case studies on models like Vicuna-7b, Llama2-7b, and GPT-3.5 demonstrate the effectiveness of our framework in identifying safety and robustness vulnerabilities, while the interactive interface allows users to explore results in detail, empowering targeted model improvements. Video Link: https://youtu.be/k1TrBqNVg8g
DOI 原文 ·
@inproceedings{paperbot9550,
title = {TrustVis: A Multi-Dimensional Trustworthiness Evaluation Framework for Large Language Models},
author = {Ruoyu Sun and Da Song and Jiayang Song and Yuheng Huang and Lei Ma},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00380}
}