一句话:提出自监督方法 SSQR,无需平行查询语料,通过掩码语言建模实现查询重构并达到有监督水平。
查询重构被广泛用于丰富用户需求、提升代码搜索效果,通常被建模为把原始查询改写成更完整查询的机器翻译任务。但训练这类模型需要大量平行查询对(原查询与重构查询),而这类语料由在线代码搜索引擎掌握且未公开,严重限制了方法的实际可用性。本文提出 SSQR,一种完全不依赖平行查询语料的自监督查询重构方法。受预训练模型启发,SSQR 将查询重构视为在大量无标注查询语料上进行的掩码语言建模任务,在 T5(基于 Transformer 的序列到序列模型)上新增名为 corrupted query completion (CQC) 的预训练目标:随机掩蔽一个完整查询中的词,并训练 T5 预测被掩蔽内容。对于待重构查询,SSQR 先识别可能的扩展位置,再利用预训练 T5 生成合适内容填补空缺,最后依据每个候选扩展的信息增益进行选择。评估结果表明,SSQR 显著优于无监督基线,并能与有监督方法取得有竞争力的性能。
Automatic query reformulation is a widely utilized technology for enriching user requirements and enhancing the outcomes of code search. It can be conceptualized as a machine translation task, wherein the objective is to rephrase a given query into a more comprehensive alternative. While showing promising results, training such a model typically requires a large parallel corpus of query pairs (i.e., the original query and a reformulated query) that are confidential and unpublished by online code search engines. This restricts its practicality in software development processes. In this paper, we propose SSQR, a self-supervised query reformulation method that does not rely on any parallel query corpus. Inspired by pre-trained models, SSQR treats query reformulation as a masked language modeling task conducted on an extensive unannotated corpus of queries. SSQR extends T5 (a sequence-to-sequence model based on Transformer) with a new pre-training objective named corrupted query completion (CQC), which randomly masks words within a complete query and trains T5 to predict the masked content. Subsequently, for a given query to be reformulated, SSQR identifies potential locations for expansion and leverages the pre-trained T5 model to generate appropriate content to fill these gaps. The selection of expansions is then based on the information gain associated with each candidate. Evaluation results demonstrate that SSQR outperforms unsupervised baselines significantly and achieves competitive performance compared to supervised methods.