一句话:针对技术查询答案摘要,构建带真值摘要的基准并提出新方法,以补充用户研究。
软件问答(SQA)站点中,为给定技术查询自动生成答案摘要的方法已有研究,但现有评估几乎只依赖用户研究。这意味着每引入一种新方法都需重新组织用户研究,既耗时又会拖慢方法迭代,且不同用户研究的结果缺乏可比性。因此,社区亟需一个带有 ground truth 摘要的基准,以补充用户研究。然而,此类基准在此之前并不存在。本文围绕这一空白展开:标题表明工作包括构建该基准并提出一种新方法。摘要未给出具体方法细节与量化结果,但明确指出该基准可提供可复现、可比较的评估基础,从而促进技术查询答案摘要方法的开发与比较。
Prior studies have demonstrated that approaches to generate an answer summary for a given technical query in Software Question and Answer (SQA) sites are desired. We find that existing approaches are assessed solely through user studies. Hence, a new user study needs to be performed every time a new approach is introduced; this is time-consuming, slows down the development of the new approach, and results from different user studies may not be comparable to each other. There is a need for a benchmark with ground truth summaries as a complement assessment through user studies. Unfortunately, such a benchmark is non-existent for answer summarization for technical queries from SQA sites.
@inproceedings{paperbot7349,
title = {Answer Summarization for Technical Queries: Benchmark and New Approach},
author = {Chengran Yang and Bowen Xu and Ferdian Thung and Yucen Shi and Ting Zhang and Zhou Yang and Xin Zhou and Jieke Shi and Junda He and Donggyun Han and David Lo},
booktitle = {ASE},
year = {2022},
doi = {10.1145/3551349.3560421}
}