Blog

Sakaluk, J. K., Williams, A. J., Kilshaw, R. E., & Rhyner, K. T. (2019). Evaluating the evidential value of empirically supported psychological treatments (ESTs): A meta-scientific review. Journal of Abnormal Psychology, 128(6), 500-509.

In the 1990s the Clinical Division of the American Psychological Association commissioned a Task Force to identify “Empirically Supported Treatments” (EST). The Task Force decided that psychotherapies that repeatedly showed statistically significant improvements over no treatment, placebos, or another treatment would be designated as “Strongly” supported. They also designated some treatments as “Modestly” supported or with “Controversial” support. The EST movement continues to have a great impact on the practice, research, and funding of psychotherapy. Time-limited, diagnosis-focused therapies, tested in randomized controlled trials became the “gold standard”. Clinicians are expected to practice these ESTs, research agencies focus funding on these models, and some governments and insurance companies provide reimbursements only for these types of therapy. The Empirically Supported Treatments (EST) movement redefined the practice of psychotherapy as short-term, symptom-focused, technically-oriented, and mostly cognitive-behavioral. In this meta-scientific review Sakaluk and colleagues asked: how good is the evidence for the ESTs? The authors were particularly concerned with the quality of the studies from a methodological and statistical point of view: how likely was it that these findings could be replicated, or how reliable were the findings? The good news is that there were few instances (about 10%) of research supporting ESTs in which researchers mis-reported the statistics (i.e., error in the reporting of statistical findings). This is quite a bit lower than previously identified mis-reporting rates (about 50%) in psychological research in general. However, only about 19% of ESTs were supported consistently by high quality studies. Over half of ESTs were supported consistently by poor quality studies. Most of the studies supporting ESTs were not sufficiently powered to detect differences between treatments or conditions. That is, often the sample sizes of patients in the studies were too small, and so the significant results were not likely reliable or perhaps not plausible. Also, those therapies that the EST list defined as having “Strong” support were not backed by more higher quality research compared to therapies considered to have “Moderate” support. In other words, the decision to designate treatments as “Strongly” or “Moderately” supported appears to have almost no relationship with the quality of the research.

Practice Implications

Embedded in this dense methodological paper are some troubling findings and important practice implications. The authors suggested that there are a number treatments on the EST list that have dubious research support because the studies of those treatments may not stand up to replication (a critical test in scientific research). It is not clear that ESTs are any more effective than other bona-fide psychotherapies that are not on the list. (Bona-fide psychotherapies are those that are based on a psychological theory, delivered by trained therapists, and in which the patient and therapist develop a relationship). The findings question whether dissemination of and training in ESTs to the exclusion of other psychotherapies can be justified given the quality of the evidence. In other words, it is possible that other bona-fide psychotherapies that are not on the EST list may be just as effective. This does not imply that psychotherapy is not effective or that anything goes when it comes to the practice of psychotherapy. Evidence-based practice in psychotherapy should guide psychotherapists’ clinical choices. However, the EST list is not the final word on what constitutes “evidence-based” practice in psychotherapy, or on what treatments should be researched and funded.

Wampold, B.E., Flückiger, C., Del Re, A.C., Yulish, N.E., Frost, N.D., …Hilsenroth, M. (2017) In pursuit of truth: A critical examination of meta-analyses of cognitive behavior therapy, Psychotherapy Research, 27, 14-32.

The vast majority of meta-analyses of studies that compare different brands of psychotherapy for any particular disorder indicate that differences between treatments are quite small and clinically trivial. Meta-analyses are an important way of aggregating effect sizes across studies and of providing reliable estimates of the state of a research field. But meta-analyses are not perfect - they rely on judgements made by the researchers that may bias findings. Despite a large body of evidence to the contrary, three meta-analyses in particular have purported to demonstrate that cognitive-behavioral therapy (CBT) is superior to other therapies for some specific disorders. In this paper, Wampold and colleagues critically review these three meta analyses to see if in fact CBT is superior to other psychotherapies. A meta-analysis by Tolin that reported that CBT was more efficacious than other therapies for anxiety and depression was surprising given that it contradicted 5 previous meta-analyses. It turns out that Tolin misclassified some treatments as CBT (including eye movement desensitization and reprocessing [EMDR] and present-centred therapy [PCT]). Further, Tolin made a critical computational error with one of the studies that when corrected wiped out any superiority for CBT. A second meta-analysis by Marcus and colleagues reported small differences in favor of CBT for primary (i.e., target symptoms) outcomes at post-treatment but not at follow up. Wampold and colleagues reported that the small difference at post-treatment was unduly affected by one study in the meta-analysis that showed unusually large effect in favor of CBT (i.e., the study was likely unreliable because its results were so much out of line with all other studies). Further, the purported superiority of CBT disappeared in the longer term. Finally, a meta-analysis by Mayo-Wilson and colleagues published in the prestigious journal Lancet Psychiatry used a network meta-analysis to compare treatments, and reported that CBT was more effective than other psychotherapies. Network meta-analysis relies heavily on indirect comparisons rather than including only studies that directly compared two therapy modalities. For example, if there are only a few studies that compare treatment A to treatment B (AB), one could look at studies of treatment A versus treatment C (AC), and studies of treatment B versus treatment C (BC), and then use the transitive property (remember high school math?) to estimate the effect of AB indirectly from the studies of AC and BC. It turns out that this practice in the context of meta-analysis is unreliable and can grossly over-estimate differences between treatments.

Practice Implication

The vast majority of meta-analyses show that bona-fide psychotherapies are effective, and one therapeutic orientation does not seem to be superior to another. The three meta-analyses that run counter to this conclusion are deeply flawed. To claim that one treatment is more effective than another will limit patients’ access to other treatments. This is concerning, since most time-limited treatments result in about half of patients recovering from their mental health problems. And so many patients and their therapists need more therapeutic options to draw upon. Falsely claiming that one treatment is more effective than others may lead insurance companies and government policy makers to make erroneous decisions to fund only one type of therapy.

Fonagy, P., Lemma, A., Target, M., O'Keeffe, S., Constantinou, M., Ventura Wurman, T., . . . Pilling, S. (2019). Dynamic interpersonal therapy for moderate to severe depression: A pilot randomized controlled and feasibility trial. Psychological Medicine, 1-10. Online first publication. doi:10.1017/S0033291719000928

Most psychotherapies are equally effective when it comes to treating depression. However, no single therapy is uniformly effective, so that about 50% of patients might improve when it comes to symptom reduction. So, although there is a large evidence base for treatments like CBT, therapists and patients need access to a range of available treatments. There is less research on psychodynamic therapies, although a number of trials and meta-analyses indicate their effectiveness to treat depression. In the United Kingdom (UK), the health system may offer a stepped care program that provides patients with low intensity guided self-help based on a CBT model followed by more intensive treatment with CBT or IPT if patients did not benefit from self-help. The UK health system rarely offers Dynamic Interpersonal Therapy (DIT), and DIT has never been studied in a randomized controlled trial within the UK health system. Fonagy and colleagues designed this randomized controlled trial to test the efficacy of DIT when compared to the CBT-oriented self-help program as offered in the UK. The study also included a smaller randomized sample of those who received the intensive version of CBT for depression. In total, 147 participants with moderate to severe depression were randomly assigned to DIT, CBT guided self-help, or the intensive version of CBT. The DIT is informed by attachment theory and by mentalization theory, and it views depressive symptoms as responses to interpersonal difficulties or perceived attachment threats. The results of the trial showed a significantly greater effect of DIT compared to guided self-help with regard to depressive symptoms, overall symptom severity, social functioning, and quality of life at post-treatment. The patients receiving DIT maintained these gains up to 1-year post-treatment. Over half of DIT patients showed clinically significant improvements, but only 9% who received the CBT-based guided self-help achieved such improvement. There were no significant differences on any of the outcomes between DIT and the more intensive version of CBT.

Practice Implications

One of the benefits of DIT, according to the authors, is that it offers a treatment manual and curriculum that enables those without a lot of background in psychodynamic therapies to deliver it. This makes DIT potentially widely-applicable in publicly funded health systems like in the UK, Canada, and others. DIT may offer yet another effective option of psychotherapy to therapists and their patients who experience depressive symptoms. The study also points to the limits of offering only guided self-help to those with moderate to severe depression.

Author email: p.fonagy@ucl.ac.uk

Huey, S. J. & Tilley, J. L. (2018). Effects of mental health interventions with Asian Americans: A review and meta-analysis. Journal of Consulting and Clinical Psychology, 86, 915-930.

Do existing mental health interventions work well for patients of Asian descent? Interventions delivered in the typical way in which they were devised may not be as effective as intended when it comes to culturally diverse groups like Asian Americans. The clinical trials in which the treatments were developed typically are almost exclusively made up of White participants, and most evidence-based treatments do not consider cultural considerations. Culturally responsive psychotherapies that are consistent with the cultural norms, values, and expectations of patients may be more effective. That is, if an evidence-based treatment is not culture specific, it may not be as effective as intended. Even when culture is taken into account in evidence-based treatments, the accommodation tends to be for African American or Hispanic/Latino patients, and not for Asian American patients. Asian American and East Asian heritage is often influenced by Confucian values that emphasize interpersonal harmony, mutual obligations, and respect for hierarchy in relationships. This may mean that patients of Asian descent may be less committed to personal choice, more attuned to others, and more socially conforming. This may lead to cultural differences in cognitive processing and emotional reactions to interpersonal contexts. In this meta-analysis, Huey and colleagues assessed if the effects of evidence-based treatments will be bigger if the treatments were specifically tailored for Asian Americans. Their review included 18 studies with 6,377 participants. Samples included Chinese Americans, Cambodian Americans, Korean Americans, Vietnamese Americans, and other Asian groups. Problems treated included depression, PTSD, smoking, and other concerns. About half of the studies were of CBT, and most (91%) were culturally tailored in some way either for an Asian subgroup or tailored for minorities in general. The mean effect size for evidence-based treatments versus control groups was d = .75, SE = .14, p < .001, indicating a moderate to large effect. Treatments tailored specifically for Asian subgroups (e.g., Chinese Americans) showed the largest effects (d = 1.10), whereas treatment with no cultural tailoring or non-Asian tailoring showed the smallest effects (d = .25).

Practice Implications

Existing psychological treatments are efficacious for Asian Americans, with moderate effects. However, treatments specifically adapted for Asian American subgroups showed the largest effects, indicating that specific cultural adaptations could substantially improve the effectiveness of psychotherapy. Asian Americans face challenges in terms of using and engaging in treatments. Developing culturally specific interventions to improve acceptability of treatment may be one way to make the most therapeutic impact on one of the largest growing racial groups in North America.

Author email: hueyjr@usc.edu

Lilliengren, P., Johansson, R., Lindqvist, K., Mechler, J., & Andersson, G. (2016). Efficacy of experiential dynamic therapy for psychiatric conditions: A meta-analysis of randomized controlled trials. Psychotherapy, 53(1), 90-104.

There is growing research support for the efficacy of short-term psychodynamic psychotherapies to treat common mental health problems. A subtype of short-term psychodynamic psychotherapies is called experiential-dynamic therapy (EDT), which goes by a number of different names such as Fosha’s accelerated experiential-dynamic psychotherapy, and McCullough’s affect phobia therapy. A fundamental assumption of EDT is that conditions like depression, anxiety and personality disorders are by-products of an individual’s attempts to regulate strong emotions associated with adverse experiences in attachment relationships during childhood. When the attachment system and associated affects are re-awakened in current relationships, the individual may engage in maladaptive coping that leads to difficulties in relationships. While EDTs may focus on helping patients to understand how their attachment difficulties lead to inhibitory affects and maladaptive defenses, the treatment favors interventions that facilitate direct experience of underlying emotions in the here and now of the therapy. In this meta-analysis, Lilliengren and colleagues reviewed 28 studies with 1,782 adult patients who had a mood, anxiety, personality, or mixed disorder. Compared to inactive controls, EDT showed a moderate and significant effect at post-treatment (range: d = .39 to .65) and at follow-up assessments (range: d = .26 to .62), with largest effects for depression and anxiety. When researchers compared EDT to cognitive-behavioral therapy (CBT) in five studies, there were no significant effects at post-treatment (d = .02, 95% CI: -.24, .28) or follow-up (d = .07, 95% CI: -.22, .36). The average quality of EDT studies was good. In fact, studies with larger samples, that used blind randomization and assessments, and appropriate statistical tests showed larger effects for EDT. Drop-out rates for EDT (16.3%) were similar to other treatments.

Practice Implications

Experiential-dynamic therapy (EDT), which is a variant of short-term psychodynamic psychotherapy, was more effective than no-treatment and just as effective as evidence-based treatments like CBT. The findings are similar to those reported in many comparative outcome studies in which any bona-fide psychotherapy is effective for many disorders. The average quality of the EDT studies was quite good, suggesting that the findings were reliable and valid, and perhaps underestimating the true effects of EDT.

Author email: peter.lilliengren@psychology.su.se

McLaughlin, S.P.B., Barkowski, S., Burlingame, G.M., Strauss, B., & Rosendahl, J. (2019). Group psychotherapy for borderline personality disorder: A meta-Analysis of randomized-controlled trials. Psychotherapy. Advance online publication. http://dx.doi.org/10.1037/pst0000211

Borderline personality disorder (BPD) is characterized by fear of abandonment, unstable intense relationships, rapid changes in identity and self-image, impulsivity, wide mood swings, periods of intense anger, and ongoing feelings of emptiness. These symptoms sometimes lead to suicidal behavior or non-suicidal self-injury. Often, those with BPD report a very stressful childhood that included sexual and/or physical abuse, and neglect. Borderline personality disorder is the most common of the personality disorders and is associated with severe social psychological impairment such that those with BPD often have unstable employment, are involved in abusive relationships, and engage in risky behaviors. A diagnosis of BPD is also associated with a high rate of mortality due to suicide. Practice guidelines indicate that psychotherapy is a key component to the treatment of BPD. Two psychological treatment approaches that incorporate group interventions are dialectical behavior therapy (DBT) and mentalization-based treatment (MBT). In DBT patients learn specific skills to alter maladaptive ways of regulating emotions in a group context. In MBT, an attachment-based treatment, the focus is on building trust in others through group interactions that generalize to other social relationships. In this meta-analysis, McLaughlin and colleagues reviewed 24 studies that compared group treatment for BPD to treatment as usual, which included a variety of interventions like supportive groups, pharmacotherapy, individual therapy, and others. Some of the treatments for BPD were stand-alone groups and some groups were part of a larger comprehensive program. Participants attended between 12 and 130 sessions, and group size ranged from 4 to 12 members. The meta-analysis revealed that group treatment for BPD versus treatment as usual resulted in moderate to large effect on BPD symptoms: g = .72, CI: [.41, 1.04], p < .001. The effects of group treatment versus treatment as usual on suicidality produced a moderate effect, g = .46, CI: [.22, .71], p < .001. The authors reported similar results for secondary outcomes like depression, anxiety, and general mental health. Drop-out rates were similar between group treatments (26.26%) and treatment as usual (28.26%). There were no differences in the effects of group therapy orientations on any of the outcomes or on drop-out rates.

Practice Implications

The results of this meta-analysis indicated the value of group treatment for BPD not only for core symptoms and suicidality, but also for symptoms related to quality of life (depression, anxiety). Theoretical orientation did not explain any of the findings, suggesting that treatments like DBT and MBT in a group format are equally effective. Therapists and patients can feel confident that group treatment for BPD are among the most effective treatments available.

Author email: mclaughlin.stevie@gmail.com