liu.seSearch for publications in DiVA
EnglishSvenskaNorsk
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Variability in the analysis of a single neuroimaging dataset by many teams
Tel Aviv Univ, Israel; Tel Aviv Univ, Israel; Dartmouth Coll, NH 03755 USA.
Univ Innsbruck, Austria.
CALTECH, CA 91125 USA.
Stockholm Sch Econ, Sweden; Univ Innsbruck, Austria.
Show others and affiliations
2020 (English)In: Nature, ISSN 0028-0836, E-ISSN 1476-4687, Vol. 582, p. 84-88Article in journal (Refereed) Published
Abstract [en]

Data analysis workflows in many scientific domains have become increasingly complex and flexible. Here we assess the effect of this flexibility on the results of functional magnetic resonance imaging by asking 70 independent teams to analyse the same dataset, testing the same 9 ex-ante hypotheses(1). The flexibility of analytical approaches is exemplified by the fact that no two teams chose identical workflows to analyse the data. This flexibility resulted in sizeable variation in the results of hypothesis tests, even for teams whose statistical maps were highly correlated at intermediate stages of the analysis pipeline. Variation in reported results was related to several aspects of analysis methodology. Notably, a meta-analytical approach that aggregated information across teams yielded a significant consensus in activated regions. Furthermore, prediction markets of researchers in the field revealed an overestimation of the likelihood of significant findings, even by researchers with direct knowledge of the dataset(2-5). Our findings show that analytical flexibility can have substantial effects on scientific conclusions, and identify factors that may be related to variability in the analysis of functional magnetic resonance imaging. The results emphasize the importance of validating and sharing complex analysis workflows, and demonstrate the need for performing and reporting multiple analyses of the same data. Potential approaches that could be used to mitigate issues related to analytical variability are discussed. The results obtained by seventy different teams analysing the same functional magnetic resonance imaging dataset show substantial variation, highlighting the influence of analytical choices and the importance of sharing workflows publicly and performing multiple analyses.
Place, publisher, year, edition, pages
Nature Publishing Group, 2020. Vol. 582, p. 84-88
National Category
Bioinformatics and Systems Biology
Identifiers
URN: urn:nbn:se:liu:diva-166165DOI: 10.1038/s41586-020-2314-9ISI: 000535225300001PubMedID: 32483374Scopus ID: 2-s2.0-85085279867OAI: oai:DiVA.org:liu-166165DiVA, id: diva2:1437769
Note

Funding Agencies|Austrian Science FundAustrian Science Fund (FWF) [P29362-G27]; Israel Science FoundationIsrael Science Foundation [ISF 2004/15]; Swedish Foundation for Humanities and Social Sciences [NHS14-1719:1]; National Institutes of Health (NIH)United States Department of Health & Human ServicesNational Institutes of Health (NIH) - USA [R24MH117179]; Austrian Science Fund (FWF)Austrian Science Fund (FWF) [SFB F63, P 32686]; Research Foundation Flanders (FWO)FWO; European UnionEuropean Union (EU) [665501]; University of Basel Research Fund; Research Foundation FlandersFWO [12T2517N]; Marie Skodowska-Curie Actions under COFUND [665501]; Obra Social La CaixaLa Caixa Foundation; Vienna Science and Technology Fund [WWTF VRG13-007]; National Natural Science Foundation of ChinaNational Natural Science Foundation of China [71801110, 71971199, 71602175, 71942004]; MOE (Ministry of Education in China) Project of Humanities and Social Sciences [18YJC630268]; China Postdoctoral Science FoundationChina Postdoctoral Science Foundation [2018M633270]; Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germanys Excellence Strategy Science of Intelligence (EXC 2002/1)German Research Foundation (DFG) [390523135]; Brain Canada; Health Canada; NIHUnited States Department of Health & Human ServicesNational Institutes of Health (NIH) - USA [NIH-NIBIB P41 EB019936, NIH-NIMH R01 MH083320, NIH RF1 MH120021, R01 DA041353]; National Institute Of Mental Health of the NIH [R01MH096906]; Canada First Research Excellence Fund; European Unions Horizon 2020 Research and Innovation Programme [785907]; Max Planck SocietyMax Planck SocietyFoundation CELLEX; Dutch foundation LSH-TKI [LSHM16053-SGF]; NSFNational Science Foundation (NSF) [1631325]; T32 Predoctoral Fellowship from the NIHUnited States Department of Health & Human ServicesNational Institutes of Health (NIH) - USA; Deutsche ForschungsgemeinschaftGerman Research Foundation (DFG) [CRC1193]; Australian National Imaging Facility; National Collaborative Research Infrastructure Strategy (NCRIS) capability; VIDI from the Netherlands Organisation for Scientific Research [452-17-013]; Swedish Research CouncilSwedish Research Council; NIH IRPUnited States Department of Health & Human ServicesNational Institutes of Health (NIH) - USA [ZICMH002888, ZICMH002960]; Tianqiao and Chrissy Center for Social and Decision Neuroscience Center Leadership Chair; Belgian Excellence of Science program (EOS) from the FNRS-Belgium [30991544]; EraNET Neuron [R4195]; Ministry of Education of Humanities and Social Science [16YJC630103]; Wellcome TrustWellcome Trust [100309/Z/12/Z]

Available from: 2020-06-09 Created: 2020-06-09 Last updated: 2021-05-05Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textPubMedScopus

Authority records

Hamilton, Paul J.Tinghög, Gustav

Search in DiVA

By author/editor
Hamilton, Paul J.Skagerlund, KennyTinghög, Gustav
By organisation
Center for Social and Affective NeuroscienceFaculty of Medicine and Health SciencesPsychologyFaculty of Arts and SciencesEconomicsDivision of Society and Health
In the same journal
Nature
Bioinformatics and Systems Biology

Search outside of DiVA

GoogleGoogle Scholar

doi
pubmed
urn-nbn

Altmetric score

doi
pubmed
urn-nbn
Total: 308 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • oxford
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
v. 2.45.0
|
Accessibility
|
DiVA portal
|
DiVA Log in
|
Contact us
|
LiU University Library
|
SwePub
|
Uppsök