On the Transportability of Laboratory Results
2021 (English)In: Sociological Methods & Research, ISSN 0049-1241, E-ISSN 1552-8294, Vol. 50, no 3, p. 1452-1452Article in journal (Refereed) Published
Abstract [en]
The "transportability" of laboratory findings to other instances than the original implementation entails the robustness of rates of observed behaviors and estimated treatment effects to changes in the specific research setting and in the sample under study. In four studies based on incentivized games of fairness, trust, and reciprocity, we evaluate (1) the sensitivity of laboratory results to locally recruited student-subject pools, (2) the comparability of behavioral data collected online and, under varying anonymity conditions, in the laboratory, (3) the generalizability of student-based results to the broader population, and (4) with a replication at Amazon Mechanical Turk, the stability of laboratory results across research contexts. For the class of laboratory designs using incentivized games as measurement instruments of prosocial behavior, we find that rates of behavior and the exact behavioral differences between decision situations do not transport beyond specific implementations. Most clearly, data obtained from standard participant pools differ significantly from those from the broader population. This undermines the use of empirically motivated laboratory studies to establish descriptive parameters of human behavior. Directions of the behavioral differences between games, in contrast, are remarkably robust to changes in samples and settings. Moreover, we find no evidence for either anonymity effects nor mode effects potentially biasing laboratory measurement. These results underscore the capacity of laboratory experiments to establish generalizable causal effects in theory-driven designs.
Place, publisher, year, edition, pages
Sage Publications, 2021. Vol. 50, no 3, p. 1452-1452
Keywords [en]
anonymity, experimental methods, external validity, laboratory research, mode effects, online experiments, prosocial behavior, sample effects
National Category
Biomedical Laboratory Science/Technology
Identifiers
URN: urn:nbn:se:liu:diva-178268DOI: 10.1177/0049124119826151ISI: 000678160200017OAI: oai:DiVA.org:liu-178268DiVA, id: diva2:1585563
2021-08-172021-08-172022-12-31