liu.seSearch for publications in DiVA
EnglishSvenskaNorsk
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
The reliability of replications: a study in computational reproductions
Leibniz Ctr Lifelong Learning, Germany.
Univ Leeds, England.
Ludwig Maximilians Univ Munchen, Germany.
Univ Maryland, MD 47405 USA.
Show others and affiliations
2025 (English)In: Royal Society Open Science, E-ISSN 2054-5703, Vol. 12, no 3, article id 241038Article in journal (Refereed) Published
Abstract [en]

This study investigates researcher variability in computational reproduction, an activity for which it is least expected. Eighty-five independent teams attempted numerical replication of results from an original study of policy preferences and immigration. Reproduction teams were randomly grouped into a 'transparent group' receiving original study and code or 'opaque group' receiving only a method and results description and no code. The transparent group mostly verified original results (95.7% same sign and p-value cutoff), while the opaque group had less success (89.3%). Second-decimal place exact numerical reproductions were less common (76.9 and 48.1%). Qualitative investigation of the workflows revealed many causes of error, including mistakes and procedural variations. When curating mistakes, we still find that only the transparent group was reliably successful. Our findings imply a need for transparency, but also more. Institutional checks and less subjective difficulty for researchers 'doing reproduction' would help, implying a need for better training. We also urge increased awareness of complexity in the research process and in 'push button' replications.
Place, publisher, year, edition, pages
ROYAL SOC , 2025. Vol. 12, no 3, article id 241038
Keywords [en]
reliability; replications; computational reproduction; social and behavioural sciences
National Category
Evolutionary Biology
Identifiers
URN: urn:nbn:se:liu:diva-213189DOI: 10.6084/m9.figshare.c.7655134ISI: 001451495100001PubMedID: 40109936OAI: oai:DiVA.org:liu-213189DiVA, id: diva2:1954062
Note

Funding Agencies|German Science Foundation -Deutsche Forschungsgemeinschaft (DFG) [464546557]; Chilean National Research and Development Agency -Agencia Nacional de Investigacion y Desarrollo [ANID/FONDAP/1523A0005]; Mannheim Centre for European Social Research, Germany

Available from: 2025-04-23 Created: 2025-04-23 Last updated: 2025-04-28

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textPubMed

Authority records

Nygård, Olav

Search in DiVA

By author/editor
Nygård, Olav
By organisation
Division of Migration, Ethnicity and Society (REMESO)Faculty of Arts and SciencesREMESO - Institute for Research on Migration, Ethnicity and Society
In the same journal
Royal Society Open Science
Evolutionary Biology

Search outside of DiVA

GoogleGoogle Scholar

doi
pubmed
urn-nbn

Altmetric score

doi
pubmed
urn-nbn
Total: 76 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
v. 2.47.0
|
Accessibility
|
DiVA portal
|
DiVA Log in
|
Contact us
|
LiU University Library
|
SwePub
|
Uppsök