一句话：研究在程序员在线调查筛选问题中引入时间限制，以在保持可靠性的同时缩短筛选时间。
在线调查中招募具备编程技能的参与者十分困难，而 Amazon MTurk、Qualtrics 等平台允许参与者自报编程知识，非程序员可能因误解或为获取报酬而参与，从而降低数据质量。Danilova 等人此前开发并测试了用于识别非程序员的筛选任务，但最可靠的筛选器往往耗时最长。本文考察引入时间限制是否能够构建更高效、即更快但仍可靠的筛选器。结果显示这一目标可行，作者据此扩展了筛选器池，并就如何改进筛选流程提出建议，为需要程序员参与者的在线调查研究提供了更实用的招募与筛查方案。
Recruiting study participants with programming skill is essential for researchers. As programming is not a common skill, recruiting programmers as participants in large numbers is challenging. Plat-forms like Amazon MTurk or Qualtrics offer to recruit participants with programming knowledge. As this is self-reported, participants without programming experience could still take part, either due to a misunderstanding or to obtain the study compensation. If these participants are not detected, the data quality will suffer. To tackle this, Danilova et al. [11] developed and tested screening tasks to detect non-programmers. Unfortunately, the most reliable screen-ers were also those that took the most time. Since screeners should take as little time as possible, we examine whether the introduction of time limits allows us to create more efficient (i.e., quicker but still reliable) screeners. Our results show that this is possible and we extend the pool of screeners and make recommendations on how to improve the process.
@inproceedings{DanilovaH0N22,
title = {Testing time limits in screener questions for online surveys with programmers},
author = {Anastasia Danilova and Stefan Horstmann and Matthew Smith and Alena Naiakshina},
booktitle = {ICSE},
year = {2022},
doi = {10.1145/3510003.3510223}
}