The Replication Crisis in Psychology – Holding Some Theories Lightly

The Replication Crisis in Psychology

The replication crisis in psychology refers to the growing awareness and evidence that many psychological studies’ findings cannot be reliably reproduced. This “crisis” highlights fundamental issues in the field’s research practices, raising concerns about the validity of psychological theories and the robustness of scientific evidence.

The Reproducibility Project, initiated by Brian Nosek (of the University of Virginia) and the Open Science Collaboration in 2011, attempted to replicate 100 published studies from three leading psychology journals. The results, published in 2015, were startling: only 36% of the replications yielded statistically significant results, compared to 97% of the original studies. This stark discrepancy cast doubt on the reliability of many high-profile findings.

The replication crisis has several underlying causes. One major factor is the pervasive use of questionable research practices, such as p-hacking, where researchers manipulate data and analyses until they achieve statistically significant results. In addition, in some instances, researchers have influenced or primed some of the particpants in their experiements. Part of the reason researchers and academics do this is due to the pressure to publish novel and positive findings, which lead to recognition and reward within the field.

Some examples of famous experiments which struggle with replication include the following:

Stanford Prison Experiment:

Conducted by Philip Zimbardo in 1971, the Stanford Prison Experiment is one of psychology’s most infamous studies. It suggested that individuals could quickly adopt abusive behaviors in certain roles, such as prison guards. However, recent critiques and attempts to replicate the findings have highlighted methodological flaws, ethical concerns, and possible researcher bias, casting doubt on the robustness of the original conclusions.

The Marshmallow Test:

Walter Mischel’s Marshmallow Test in the 1960s and 70s purportedly demonstrated that children who could delay gratification (waiting to eat a marshmallow) tended to have better life outcomes. A replication study published in 2018 by Tyler Watts and colleagues found that the effect of delayed gratification on future success was much smaller and less consistent than originally reported, with socioeconomic factors playing a significant role.

Ego Depletion:

Roy Baumeister’s research on ego depletion suggested that self-control is a finite resource that can be exhausted. A large-scale replication attempt by the Many Labs 2 project in 2016 failed to reproduce the ego depletion effect, leading to significant skepticism about the reliability of the original findings.

Social Priming (Elderly Walking Study): John Bargh’s 1996 study on social priming claimed that participants exposed to words related to old age walked more slowly afterward. Subsequent replication attempts, including a notable failure by Doyen and colleagues in 2012, have been unable to replicate these findings consistently, suggesting the original effects might have been overstated or due to methodological issues.

Power Posing:

Amy Cuddy’s 2010 research on power posing claimed that adopting expansive postures could increase feelings of power and alter hormone levels. Multiple replication efforts, such as those by Eva Ranehill et al. in 2015, have failed to find the same physiological effects, and the psychological impacts have been shown to be weaker than initially reported, leading to significant controversy over the validity of the original study.

Growth Mindsets:

The concept of “growth mindset,” developed by psychologist Carol Dweck, posits that individuals who believe their abilities can be developed through effort and learning (a growth mindset) tend to achieve more than those who believe their abilities are fixed (a fixed mindset). While Dweck’s research has had a significant impact on educational practices and motivational psychology, replication efforts have raised questions about the robustness and generalizability of the growth mindset effects.

There are lots of places you can learn more about the replication crisis in the world of psychology, and about how to adopt better research methods. One of the books we enjoyed the most which explores some of these examples is the excellent “Human Kind” by Rutget Bergman.

You might also enjoy our podcast on research in organizations:

There’s a beautiful irony in a replication crisis taking place in the fielf of Psychology.From our reading, some of the drivers of this replication crisis step from the psychology of researchers who are motivated to create novel and eye-catching findings in the quest for recognition and reward, themselves pslychological drivers, perhaps. And this is only human. We’ve not explored this topic in great detail, nor from an academic perspective, but we know it is important. We also know that the world of leadership and management training and development is full of learning resources and programmes built on the foundations of research that has not been replicated. This may be a problem. Then again, some of the learning programme may still be hugely helpful. We also note that the rise of movement towards evidence based management is driven in part by some of these challenges. When in doubt, it’s worth being sceptical about eye-catching findings and checking if they do stand up to replication.

There are no specific addition references for this, beyond those noted in the main text. If you think there should be some, please do let us know.

We’re a small organization who know we make mistakes and want to improve them. Please contact us with any feedback you have on this post. We’ll usually reply within 72 hours.