一句话：提出软件工程领域特定的毒性检测器 ToxiCR，在 19,571 条人工标注代码评审评论上准确率达 95.8%。
开源软件项目中，开发者交互时的毒性言论会对开发者之间的关系造成负面影响，因此需要面向软件工程（SE）领域的毒性检测器。然而既有研究表明，通用的当代毒性检测工具在处理 SE 文本时表现很差。为应对这一挑战，作者开发了 ToxiCR——一个 SE 特定的毒性检测器，并在人工标注的 19,571 条代码评审评论上进行评估。评估中系统性地组合了十种监督学习模型、五种文本向量化器和八种预处理技术（其中两种为 SE 领域特定技术），遍历所有可能的组合。结果表明，ToxiCR 显著优于现有的毒性分类器，准确率达到 95.8%，F1 分数为 88.9%。
Since toxicity during developers’ interactions in open source software (OSS) projects show negative impacts on developers’ relation, a toxicity detector for the Software Engineering (SE) domain is needed. However, prior studies found that contemporary toxicity detection tools performed poorly with the SE texts. To address this challenge, I have developed ToxiCR, a SE-specific toxicity detector that is evaluated with manually labeled 19,571 code review comments. I evaluate ToxiCR with different combinations of ten supervised learning models, five text vectorizers, and eight preprocessing techniques (two of them are SE domain-specific). After applying all possible combinations, I have found that ToxiCR significantly outperformed existing toxicity classifiers with accuracy of 95.8% and an F1 score of 88.9%.
DOI 原文 ·
@inproceedings{paperbot7290,
title = {‘Who built this crap?’ Developing a Software Engineering Domain Specific Toxicity Detector},
author = {Jaydeb Sarker},
booktitle = {ASE},
year = {2022},
doi = {10.1145/3551349.3559508}
}