Are RCTs valid in Psychotherapy?

May 15, 2021 Health & Healing, Psychotherapy No Comments

Randomized Controlled Trials (RCTs) are a lab-based study design in which one tries to minimize confounding factors by comparing groups that are as similar as possible except for the property or properties under study.

Getting physical

These trials are meant to be as close as possible to studies in the field of physics. In these, there is strict delineation between the variables (or characteristics) under study and all others. In a proper study, the distinction is crisp, and the results are quantifiable in a straightforward manner.

A (part of the) world in which this is possible is called a closed world.

Medicine tried and tries to emulate this, and so does psychotherapy.

Open to problems

However, from physics to medicine to psychotherapy, the closed world assumption is increasingly less forthright. This is easily understandable as the degree of complexity increasingly heightens in exponential fashion.

This calls for problems. In an open world, many more biases may confound the results of a trial. Study designers try to alleviate these problems through randomization and control. Groups are compared that are as similar as possible to each other and to the general population except for the characteristics under study.

This turns out to be much more complicated than expected.

Two groups of problems.

Many more, actually, but I discern these two groups as most relevant for why things go principally wrong. With ‘principally,’ I mean: for all studies, abstractly speaking:

biases within the study: group differences that are not accounted for. For instance, in psychotherapy: same kinds of subjects but different kinds of therapists, different degrees of non-consciously engendered placebo effects through side effects, differences in expectation profiles, etc.
extrapolation problems towards the real world. For instance, not feeling studied anymore, the disappearance of other study design elements, changes in real-world contacts, differences with and between groups of the general population, etc.

Plus: implicit theory

This may be seen as a ‘problem of the third kind.’

It is well known that an explicit theory may drive data gathering. For instance, easily gathered data may be higher on the agenda. Thus, the study design may ignore less easily discernible data that are not, therefore, less important. They may make a difference in the real-world that comes as a surprise or may not be easily noticed until much later or in changed circumstances.

Nothing worse than a theory that improperly drives data gathering ― except an implicit theory.

Basic cognitive illusion – basic implicit theory

This is the second part of the Cartesian error ― or should one say disaster. It consists of the idea that everything mentally significant is consciously knowable or even known. [see: “The Basic Cognitive Illusion“]

This provokes a massive problem in medicine, making double-blind studies substantially less blind than generally thought. [see: “Double-Blind in the Balance“] This results in a present-day real-world evidence crisis in medicine.

Also in RCTs where no double-blind is possible, huge problems may be prompted through the same illusion. The present-day reproducibility crisis in psychology may well be substantially engendered through this.

Thus, in both medicine and psychotherapy, we may be witnessing a similarly caused crisis. We also see both fields being plagued by mounting human costs, suffering, and financial costs. May this be profoundly related?

I wonder where it’s leading to. I wonder less about the solution, abstractly seen. It lies in diminishing the illusion/disaster.

Then again, I wonder how. I can only try to humbly do my share.