paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Testing Machine Translation via Referential Transparency

通过指称透明性测试机器翻译
ICSE 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE)2021匹配度 4/10 · 论文属于机器学习系统的动态测试与验证,与用户关注的“PL→AI”中在ML系统上应用分析技术有部分交集,但方法并非静态分析,对象是机器翻译而非Agentic程序或代码,且不涉及浮点、贝叶斯或LLM作为工具,因此匹配度有限。引用 64 inbox
Pinjia He, Clara Meister, Zhendong Su
crossref: 2026-09-12dblp: 2026-09-12arxiv: 2026-09-12

AI 速览

一句话:提出指称透明输入(RTI)方法并实现 Purity,用于检测机器翻译错误,在 Google 与 Bing 上高精度发现大量误译。

机器翻译在神经网络推动下快速进步,但因模型复杂且难以解释,仍不鲁棒,可能产生错误翻译,导致误解、经济损失、人身安全与健康威胁乃至政治冲突。本文提出指称透明输入(RTIs):同一段文本在不同语境中应得到相似翻译;若翻译破坏该性质,就可能是错误。作者实现工具 Purity,用其自动检测此类违反。为评估 RTI,作者用 200 个无标签句子测试 Google Translate 和 Bing Microsoft Translator,分别检出 123 和 142 个错误翻译,精度达 79.3% 和 78.3%。错误类型多样,包括欠译、过译、词/短语误译、错误修饰和逻辑不清。

问题
现代神经机器翻译仍不鲁棒,会产生错误翻译,可能造成误解、经济损失、安全与健康威胁及政治冲突,因此需要有效的验证方法。
方法
提出指称透明输入(RTIs),即在不同上下文中应得到相似翻译的文本,并实现 Purity 工具来检测翻译是否破坏该指称透明性质,从而发现错误。
评估
用 200 个无标签句子测试 Google Translate 和 Bing Microsoft Translator,分别检出 123 和 142 个错误翻译,精度为 79.3% 和 78.3%,错误类型涵盖欠译、过译、词/短语误译、错误修饰和逻辑不清等。
原文摘要(Abstract)

Machine translation software has seen rapid progress in recent years due to the advancement of deep Neural Networks. People routinely use machine translation software in their daily lives for tasks such as ordering food in a foreign restaurant, receiving medical diagnosis and treatment from foreign doctors, and reading international political news online. However, due to the complexity and intractability of the underlying Neural Networks, modern machine translation software is still far from robust and can produce poor or incorrect translations; this can lead to misunderstanding, financial loss, threats to personal safety and health, and political conflicts. To address this problem, we introduce referentially transparent inputs (RTIs), a simple, widely applicable methodology for validating machine translation software. A referentially transparent input is a piece of text that should have similar translations when used in different contexts. Our practical implementation, Purity, detects when this property is broken by a translation. To evaluate RTI, we use Purity to test Google Translate and Bing Microsoft Translator with 200 unlabeled sentences, which detected 123 and 142 erroneous translations with high precision (79.3% and 78.3%). The translation errors are diverse, including examples of under-translation, over-translation, word/phrase mistranslation, incorrect modification, and unclear logic.

链接与引用

DOI 原文 · arXiv · PDF(开放获取) · DBLP

BibTeX
@inproceedings{abs-2004-10361,
  title = {Testing Machine Translation via Referential Transparency},
  author = {Pinjia He and Clara Meister and Zhendong Su},
  booktitle = {ICSE},
  year = {2021},
  doi = {10.1109/icse43902.2021.00047}
}