Crowdsourcing data analysis paper coming out in OBHDP (open access)

We are happy to report that our paper entitled “Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis” is forthcoming at Organizational Behavior and Human Decision Processes.

We examine both paths taken and not taken by multiple analysts who reported radically different analyses and dispersed empirical outcomes, including significant effects in opposite directions for the same research question. We conducted both a multiverse analysis and a qualitative analysis of analyst’s R code (a world’s first as far as we know) to show exactly which subjective decisions drive the reported empirical results. We also present DataExplained, a newly developed tool that is available free of charge to any analyst who wants to make her analytical choices transparent.

You can soon read the entire paper (open access) and access all supplementary materials here.

Reference

Schweinsberg, M., Feldman, M., Staub, N., van den Akker, O. R., van Aert, R. C. M., van Assen, M. A. L. M., … Uhlmann, E. L. (2021). Same data, different conclusions: Radical dispersion in empirical results when independent analysts operationalize and test the same hypothesis. Forthcoming in Organizational Behavior and Human Decision Processes.