Dr Manuela Thomae and Dr Michael Wood participated in an international collaboration – the Reproducibility Project: Psychology – designed to assess the reproducibility of psychological research. To date, this project is the most comprehensive investigation ever undertaken about the rate and predictors of reproducibility in a field of science.
Dr Thomae and Dr Wood submitted their findings in a collaborative research paper entitled Estimating the Reproducibility of Psychological Science and collectively authored by the Open Science Collaboration. The paper is based on the evidence collected internationally from replications of 100 published psychology studies.
“Here at Winchester, we conducted a pre-registered replication of an experiment assessing whether people act in their own individual best interest or in the best interest of the group to which they belong,” said Dr Thomae. “Using male undergraduate students from Israel, the original scientists found that once you disentangle opposing motives, such as helping one’s own versus harming the others, the overriding motivation driving most people to engage in conflict is the desire to help their own – even if it comes at great personal expense.
“We successfully replicated this finding with an undergraduate participant sample from the University of Winchester. However, the overall picture painted by this large-scale replication effort is less optimistic.”
The Reproducibility Project: Psychology found that regardless of the analytic method or criteria used, fewer than half of the replications produced the same findings as the original study.
Mallory Kidwell, one of the project coordinators from the Center for Open Science, concluded: “The results provide suggestive evidence toward the challenges of reproducing research findings, including identifying predictors of reproducibility and practices to improve it.”
Since the Reproducibility Project began in 2011, similar projects have emerged in other fields such as the Reproducibility Project: Cancer Biology, and the discipline of meta-science is expanding – scientific research about scientific research. These and the widespread efforts to improve research transparency and reproducibility are indications that, as suggested by team member Susann Fiedler from the Max Planck Institute for Research on Collective Goods: “Science is actively self-examining and self-correcting to maximise the quality and efficiency of the research process in the service of building knowledge for the public good.”
Dr Thomae, added: “Science is unique from other ways of gaining knowledge by relying on reproducibility to gain confidence in ideas and evidence. Reproducibility means that the results recur when the same data are analysed again, or when new data are collected using the same methods.
“The contribution we hope to make here at Winchester is at least twofold. Firstly, we now have further evidence that ‘In-Group Love’, not ‘Out-Group Hate’ motivates people to engage in intergroup conflict. Our findings add a cross-cultural dimension to the original study and we are now one step closer in showing that we are observing a phenomenon that holds true for humans in general and not just for a specific participant sample, tested at a specific time and in a specific context.
“Secondly, our findings support the large-scale effort of the Reproducibility Project: Psychology and help to assess the reproducibility of psychological research. As psychologists, we often measure ‘invisible things’ – thoughts, ideas, motivations or concepts that are in the mind and at times not easily observable. Hence, it is particularly vital to ensure a high degree of generalisibility and reproducibility of our findings.”