Psychology’s Replication Crisis

April 13, 2024 Health & Healing No Comments

Since 2010, an effort (Many Labs 2) to replicate 28 of the most renowned and highly cited studies in behavioral psychology (with peer-reviewed protocols) has led to +/- half of them showing drastically different results in comparison to the originals.

This should not be denied. It potentially calls into question substantial parts of scientific knowledge, yet it is no reason to discard science. On the contrary, it should lead us to take science even more seriously, primarily when it influences general policy-making in mind-related matters.

What causes this conundrum.

Several hypotheses have been forwarded:

sloppy research practices, with small and statistically weak studies
fuzziness in data handling, turning them around until something interesting comes out (posthoc ‘cherry picking’ of hypotheses)
publication bias: only publishing the ones with positive results
unfortunately, also: financial and other interests and prejudices.

I would personally add to this what may be most crucial after all:

a huge cultural bias against the influence of the deeper layers of the mind in anything human-related.

The above same arguments have then been ushered toward initial replicating scientists, which is why the massive Many Labs 2 project took place, with 60 times more study subjects than in the studies in question, repeating experiments many times over in many countries (different cultures), checking every possible detail. The results are clear, as shown above.

An interesting cultural sidekick

Those studies that showed positive replication did so in different cultures. Those that diverged also did so in various cultures.

This shows – if we didn’t know yet – that people all over the world are pretty similar. Well, doesn’t it make one happy to see this transcendent result, at least, being replicated? Of course, people behave culturally differently, but to the core, we are the same.

Not only psychology

Failed replication also happens in other natural and social sciences, including in medicine. The probably most interesting article in the latter domain is written by John Ioannidis (*), an extraordinarily courageous and brilliant researcher whom I had the privilege of personally encountering in 2016. I remember we also talked about my investigations into the lack of blinding (as well as blindness proof) of many medical studies, and the implications of this on the concept itself of ‘double-blind studies’ (see <science> in the above green menu).

Unfortunately, Ioannidis has also shown that the quality of the evidence in even very renowned scientific papers/journals does not seem to improve over time.

Again, this is no argument to discard science ― on the contrary!

However, rethinking my own added critique, we should at least acknowledge that the known gaps in our knowledge about what causes illness and healing are not only huge but will also probably grow in anything related to the complexity of the human mind.

Isn’t it high time for the Lisa Revolution, emphasizing the integration of deep respect for the human mind in both personal growth and scientific inquiry?

(*) Ioannidis JP. Why most published research findings are false. PLoS Med. 2005 Aug;2(8):e124. doi: 10.1371/journal.pmed.0020124. Epub 2005 Aug 30.