Sakaluk, J. K., Williams, A. J., Kilshaw, R. E., & Rhyner, K. T. (2019). Evaluating the evidential value of empirically supported psychological treatments (ESTs): A meta-scientific review. Journal of Abnormal Psychology, 128(6), 500-509.
In the 1990s the Clinical Division of the American Psychological Association commissioned a Task Force to identify “Empirically Supported Treatments” (EST). The Task Force decided that psychotherapies that repeatedly showed statistically significant improvements over no treatment, placebos, or another treatment would be designated as “Strongly” supported. They also designated some treatments as “Modestly” supported or with “Controversial” support. The EST movement continues to have a great impact on the practice, research, and funding of psychotherapy. Time-limited, diagnosis-focused therapies, tested in randomized controlled trials became the “gold standard”. Clinicians are expected to practice these ESTs, research agencies focus funding on these models, and some governments and insurance companies provide reimbursements only for these types of therapy. The Empirically Supported Treatments (EST) movement redefined the practice of psychotherapy as short-term, symptom-focused, technically-oriented, and mostly cognitive-behavioral. In this meta-scientific review Sakaluk and colleagues asked: how good is the evidence for the ESTs? The authors were particularly concerned with the quality of the studies from a methodological and statistical point of view: how likely was it that these findings could be replicated, or how reliable were the findings? The good news is that there were few instances (about 10%) of research supporting ESTs in which researchers mis-reported the statistics (i.e., error in the reporting of statistical findings). This is quite a bit lower than previously identified mis-reporting rates (about 50%) in psychological research in general. However, only about 19% of ESTs were supported consistently by high quality studies. Over half of ESTs were supported consistently by poor quality studies. Most of the studies supporting ESTs were not sufficiently powered to detect differences between treatments or conditions. That is, often the sample sizes of patients in the studies were too small, and so the significant results were not likely reliable or perhaps not plausible. Also, those therapies that the EST list defined as having “Strong” support were not backed by more higher quality research compared to therapies considered to have “Moderate” support. In other words, the decision to designate treatments as “Strongly” or “Moderately” supported appears to have almost no relationship with the quality of the research.
Embedded in this dense methodological paper are some troubling findings and important practice implications. The authors suggested that there are a number treatments on the EST list that have dubious research support because the studies of those treatments may not stand up to replication (a critical test in scientific research). It is not clear that ESTs are any more effective than other bona-fide psychotherapies that are not on the list. (Bona-fide psychotherapies are those that are based on a psychological theory, delivered by trained therapists, and in which the patient and therapist develop a relationship). The findings question whether dissemination of and training in ESTs to the exclusion of other psychotherapies can be justified given the quality of the evidence. In other words, it is possible that other bona-fide psychotherapies that are not on the EST list may be just as effective. This does not imply that psychotherapy is not effective or that anything goes when it comes to the practice of psychotherapy. Evidence-based practice in psychotherapy should guide psychotherapists’ clinical choices. However, the EST list is not the final word on what constitutes “evidence-based” practice in psychotherapy, or on what treatments should be researched and funded.