一句话：提出 TypeBert，用简单 token 序列归纳偏置与大规模数据，在 TypeScript 类型注解预测上超越复杂模型。
TypeScript 的可选类型允许“按需支付”类型化，但也带来类型注解税：为新旧代码补类型需要人力。已有自动方法多采用概率机器学习，归纳偏置从简单 token 序列到复杂图神经网络，并常通过手工设计利用软件形式特性。本文追问：与其为代码部署精巧归纳偏置，能否直接用“大数据”学习与类型相关的自然模式？作者提出 TypeBert，采用 BERT 式简单 token 序列归纳偏置并扩大数据规模。在类型注解任务上，TypeBert 的性能可超过最复杂的模型，表明足够数据下简单模型即可奏效。
TypeScript is a widely used optionally-typed language where developers can adopt “pay as you go” typing: they can add types as desired, and benefit from static typing. The “type annotation tax” or manual effort required to annotate new or existing TypeScript can be reduced by a variety of automatic methods. Probabilistic machine-learning (ML) approaches work quite well. ML approaches use different inductive biases, ranging from simple token sequences to complex graphical neural network (GNN) models capturing syntax and semantic relations. More sophisticated inductive biases are hand-engineered to exploit the formal nature of software. Rather than deploying fancy inductive biases for code, can we just use “big data” to learn natural patterns relevant to typing? We find evidence suggesting that this is the case. We present TypeBert, demonstrating that even with simple token-sequence inductive bias used in BERT-style models and enough data, type-annotation performance of the most sophisticated models can be surpassed.