Blog

Wampold, B.E., Flückiger, C., Del Re, A.C., Yulish, N.E., Frost, N.D., …Hilsenroth, M. (2017) In pursuit of truth: A critical examination of meta-analyses of cognitive behavior therapy, Psychotherapy Research, 27, 14-32.

The vast majority of meta-analyses of studies that compare different brands of psychotherapy for any particular disorder indicate that differences between treatments are quite small and clinically trivial. Meta-analyses are an important way of aggregating effect sizes across studies and of providing reliable estimates of the state of a research field. But meta-analyses are not perfect - they rely on judgements made by the researchers that may bias findings. Despite a large body of evidence to the contrary, three meta-analyses in particular have purported to demonstrate that cognitive-behavioral therapy (CBT) is superior to other therapies for some specific disorders. In this paper, Wampold and colleagues critically review these three meta analyses to see if in fact CBT is superior to other psychotherapies. A meta-analysis by Tolin that reported that CBT was more efficacious than other therapies for anxiety and depression was surprising given that it contradicted 5 previous meta-analyses. It turns out that Tolin misclassified some treatments as CBT (including eye movement desensitization and reprocessing [EMDR] and present-centred therapy [PCT]). Further, Tolin made a critical computational error with one of the studies that when corrected wiped out any superiority for CBT. A second meta-analysis by Marcus and colleagues reported small differences in favor of CBT for primary (i.e., target symptoms) outcomes at post-treatment but not at follow up. Wampold and colleagues reported that the small difference at post-treatment was unduly affected by one study in the meta-analysis that showed unusually large effect in favor of CBT (i.e., the study was likely unreliable because its results were so much out of line with all other studies). Further, the purported superiority of CBT disappeared in the longer term. Finally, a meta-analysis by Mayo-Wilson and colleagues published in the prestigious journal Lancet Psychiatry used a network meta-analysis to compare treatments, and reported that CBT was more effective than other psychotherapies. Network meta-analysis relies heavily on indirect comparisons rather than including only studies that directly compared two therapy modalities. For example, if there are only a few studies that compare treatment A to treatment B (AB), one could look at studies of treatment A versus treatment C (AC), and studies of treatment B versus treatment C (BC), and then use the transitive property (remember high school math?) to estimate the effect of AB indirectly from the studies of AC and BC. It turns out that this practice in the context of meta-analysis is unreliable and can grossly over-estimate differences between treatments.

Practice Implication

The vast majority of meta-analyses show that bona-fide psychotherapies are effective, and one therapeutic orientation does not seem to be superior to another. The three meta-analyses that run counter to this conclusion are deeply flawed. To claim that one treatment is more effective than another will limit patients’ access to other treatments. This is concerning, since most time-limited treatments result in about half of patients recovering from their mental health problems. And so many patients and their therapists need more therapeutic options to draw upon. Falsely claiming that one treatment is more effective than others may lead insurance companies and government policy makers to make erroneous decisions to fund only one type of therapy.

Jennissen, S., Huber, J., Ehrenthal, J.C., Schauenburg, H., & Dinger, U. (2018). Association between insight and outcome of psychotherapy: Systematic review and meta-analysis. The American Journal of Psychiatry. Published Online: https://doi.org/10.1176/appi.ajp.2018.17080847

For many authors, one of the purported mechanisms of change in psychotherapy is insight. In fact, the utility of insight for clients with mental health problems was first proposed over 120 years ago by Freud and Breuer. Briefly, insight refers to higher levels of self-understanding that might result in fewer negative automatic reactions to stress and other challenges, more positive emotions, and greater flexibility in cognitive and interpersonal functioning. Although insight is a key factor in some psychodynamic models, it also plays a role in other forms of psychotherapy. Experiential psychotherapy emphasises gaining a new perspective through experiencing, and for CBT insight relates to becoming more aware of automatic thoughts. Jennissen and colleagues defined insight as patients understanding: the relationship between past and present experiences, their typical relationship patterns, and the associations between interpersonal challenges, emotional experiences, and psychological symptoms. In this study, Jennissen and colleagues conducted a systematic review and meta analysis of the insight-outcome relationship, that is the relationship between client self-understanding and symptom reduction. They reviewed studies of adults seeking psychological treatment including individual or group therapy. The predictor variable was an empirical measure of insight assessed during treatment but prior to when final outcomes were evaluated. The outcome was some reliable and empirical measure related to symptom improvement, pre- to post- treatment. The review turned up 22 studies that included over 1100 patients mostly with anxiety or depressive disorders who attended a median of 20 sessions of therapy. The overall effect size of the association between insight and outcome was r = 0.31 (95% CI=0.22–0.40, p < 0.05), which represents a medium effect. Moderator analyses found no effect of type of therapy or diagnosis on this mean effect size, though the power of these analyses was low.

Practice Implications

The magnitude of the association between insight and outcome is similar to the effects of other therapeutic factors such as the therapeutic alliance. When gaining insight, patients may achieve a greater self-understanding, which allows them to reduce distorted perceptions of themselves, and better integrate unpleasant experiences into their conscious life. Symptoms may be improved by self-understanding because of the greater sense of control and master that it provides, and by the new solutions and adaptive ways of living that become available to clients.

Author email: Simone.Jennissen@med.uni-heidelberg.de

Cristea, I.A., Gentili, C., Cotet, C.D., Palomba, D., Barbui, C., & Cuijpers, P. (2017). Efficacy of psychotherapies for borderline personality disorder: A systematic review and meta-analysis. JAMA Psychiatry. doi:10.1001/jamapsychiatry.2016.4287.

Borderline personality disorder (BPD) is a debilitating disorder characterized by: severe instability of emotions, relationships, and behaviors. More than 75% of those with BPD have engaged in deliberate self-harm, and suicide rates are between 8% and 10%. BPD is the most common of the personality disorders with a high level of functional impairment. Several psychotherapies have been developed to treat BPD. Most notably, dialectical behavior therapy (DBT), cognitive behavioral therapy (CBT), and psychodynamic treatments like mentalization-based and transference-focused psychotherapy. This meta-analysis by Cristea and colleagues examined the efficacy of psychotherapy for BPD. Studies included in the meta-analysis (33 trials of 2256 clients) were randomized controlled trials in which a psychotherapy was compared to a control condition for adults with BPD. For all borderline-relevant outcomes (combined borderline symptoms, self-harm, parasuicidal and suicidal behaviors) yielded a significant but small effect of the psychotherapies over control conditions at post treatment (g = 0.35; 95%CI: 0.20, 0.50). At follow up, there was again a significant effect of the psychotherapies over control conditions with a moderate effect (g = 0.45; 95% CI: 0.15, 0.75). When the different treatment types were looked at separately, DBT (g = 0.34; 95% CI: 0.15, 0.53) and psychodynamic approaches (g = 0.41; 95% CI: 0.12, 0.69) were more effective than control interventions, while CBT (g = 0.24; 95% CI: −0.01, 0.49) was not. The authors also reported a significant amount of publication bias, suggesting that published results may be positively biased in favor of the psychotherapies.

Practice Implications

The results indicate a small effect of psychotherapies at post-treatment and a moderate effect at follow-up for the treatment of BPD. DBT and psychodynamic treatment were significantly more effective than control conditions, whereas CBT was not. However, all effects were likely inflated by publication bias, indicating a tendency to publish only positive findings. Nevertheless, various independent psychotherapies demonstrated efficacy for symptoms of self harm, suicide, and general psychopathology in BPD.

The Great Psychotherapy Debate: Starting in April, 2015 I review parts of The Great Psychotherapy Debate (Wampold & Imel, 2015) in the PPRNet Blog. This is the second edition of a landmark, and sometimes controversial, book that surveys the evidence for what makes psychotherapy work. You can view parts of the book in Google Books.

Change in dysfunctional attitudes or cognitions is one of the specific mechanisms by which cognitive therapy (CT) is thought to be effective in the treatment of depression. In this part of their book, Wampold and Imel discuss the evidence that addresses the specific change mechanisms for CT. The reason they focus on CT is that CT is by far the most researched psychotherapy approach, and there is a substantial number of CT studies that have addressed this issue of change mechanisms. In an early meta analysis, Oei and Free (1995) found a significant relationship between change in cognitions and CT. However, in the same meta analysis, the authors found that CT and non-cognitive therapies did not differ in terms of their effects on cognitions. That is, most treatments, whether CT or not, appeared to change cognitions. In another study, three different interventions (behavioral activation, CT, and CT plus behavioral activation) all resulted in change in cognitions and improved depression. In other words, cognitive interventions do not seem to be needed to alter cognitions and reduce depression. Wampold and Imel argue that nonspecific processes in CT (and other psychotherapies for that matter) are largely responsible for the effectiveness of psychotherapy. For example, there is evidence to suggest that a number of patients show substantial symptom improvement early in treatment before specific cognitive techniques are introduced. Some have argued that this early favourable response is largely due to the effects of client expectations, reassurance, and remoralization rather than the specific procedures of the therapy. Moreover, patients who experience this remoralization early-on may be better at successfully applying techniques taught in CT. A large review of this literature concluded that there was insufficient evidence to support the notion that challenging thoughts was responsible for the positive effects of CT.

Practice Implications

This line of research appears to indicate that the specific practice of challenging thoughts or dysfunctional attitudes is not primarily responsible for patient change in CT. It may be that for any psychological treatment that has a cogent rationale for the disorder and is administered by an acknowledged expert, client progress may be determined largely by contextual factors. These factors may include a therapeutic alliance, client expectations of benefit, and client remoralization, which may in turn allow clients to benefit from the specific interventions of psychological treatments.

Hollon, S.D. & Beck, A.T. (2013). Cognitive and cognitive-behavioral therapies. In M.E. Lambert (Ed.), Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change, 6th Edition (pp. 393-442). New York: Wiley.

Cognitive (CT) and cognitive behavioural therapies (CBT) are among the most empirically supported and widely practiced psychological interventions. CT emphasizes the role of meaning in their models of depression and CT interventions emphasise testing the accuracy of beliefs. More behavioural approaches like CBT see change in terms of classical or operant conditioning of behaviours, in which cognitive strategies are incorporated to facilitate behavioural change. In this section of their chapter, Hollon and Beck review research on CT for depression. Depression is the single most prevalent mental disorder and is a leading cause of disability in the world (see this month’s blog entry on the global burden of depression). Most patients have multiple episodes of depression (i.e., recurrent) and about 25% have episodes that last for 2 years or more (i.e., chronic). CT posits that depressed individuals have negative automatic thoughts that are organized into depressogenic automatic beliefs (or underlying assumptions) that put them at risk for relapse. Automatic beliefs can be organized in latent (or unconscious) schemas often laid down in childhood and activated by later stress that influence the way information is organized. In CT patients are taught to evaluate their beliefs (also called empirical disconfirmation), conduct “experiments” to test their accuracy and to modify core beliefs and reduce maladaptive interpersonal behaviours. Most reviews show that CT for depression is superior to no treatment (with large effects) and at least as effective as alternative psychological or pharmacological interventions. Most patients show a good response to CT with about one third showing complete remission. Although some practice guidelines have concluded that medications are preferred to CBT (or any psychotherapy) for severe depression, more recent meta analyses show that CT is as efficacious as medications and is likely better in the long term. CT also has an enduring effect that protects clients against symptoms returning. Medications, on the other hand suppress depressive symptoms only as long as the patient continues to take the treatment, but medications do not reduce underlying risk. As a result, relapse rates for medication treatment of depression are much higher than for CT. These findings suggest that patients who receive CT learn something that reduces risk for recurrence, which is the single biggest advantage that CT has over medications. Further, CT is free from problematic side effects that may occur with medications.

Practice Implications

CT and CBT are the most tested psychological treatments for depression and the evidence indicates that many patients benefit. CT and CBT are as effective as medications for reducing acute distress related to depression, and even for those with more severe depression when implemented by experienced therapists. CT has an enduring effect not found in medications, may also help prevent future episodes of depression, and may prevent relapse after medications are discontinued.

Handbook of Psychotherapy and Behavior Change: Starting in March 2013 I will review one chapter a month from the Handbook of Psychotherapy and Behavior Change in addition to reviewing psychotherapy research articles. Book chapters have more restrictive copy right rules than journal articles, so I will not provide author email addresses for these chapters. If you are interested, the Handbook table of content and sections of the book can be read on Google Books.

Crits-Christoph, P., Connolly Gibbons, M.B., & Mukherjee, D. (2013). Psychotherapy process-outcome research. In M.E. Lambert (Ed.), Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change, 6^th Edition (pp. 298-340). New York: Wiley.

In this section of their chapter in the Handbook, Crits-Christoph and colleagues (2013) review research on: (1) specific techniques of cognitive behavioural therapy (CBT), and (2) change mechanisms of CBT for depression. Research on techniques and mechanisms of change tests the specific or unique effects of a treatment and the rationale for its use. The first issue addresses whether therapist adherence and competence in using CBT techniques produce desired outcomes in patients. CBT techniques include: following an agenda, reviewing homework, asking about specific beliefs, practicing rational responses with patients, and asking patients to keep thought records. Crits-Christoph and colleagues (2013) report that the research findings on the association between using specific CBT techniques and depression outcomes are mixed. The strongest evidence is for concrete techniques such as setting agendas, reviewing homework, and practicing rational responses. However the number of studies that control for prior symptom change and other factors like therapeutic alliance is small, and so the evidence for the specific effects of CBT techniques remains meagre. The second issue addresses whether targeting depressogenic cognitions with CBT results in positive outcomes. Generally, CBT theory argues that the mechanisms by which CBT works is to focus on core depressogenic schemas (i.e., less consciously long held negative beliefs about the self), conscious negative automatic thoughts, and dysfunctional attitudes (i.e., patterns of automatic thoughts) that lead to or maintain depression. Theoretically, addressing these cognitions in CBT should reduce depressive symptoms. Overall, the research shows that both CBT and medication treatment for depression reduce self-reported negative thinking; that is, the effects on negative thinking were not specific to CBT. Few studies show that changes in cognitions precede changes in depressive symptoms, which is a key CBT tenet. The most promising findings suggest that learning compensatory skills (i.e., finding alternative explanations for negative events and thoughts, and problems solving) may be part of the mechanism by which CBT works, but again this mechanism may not be specific to CBT.

Practice Implications

CBT is an effective treatment for depression. CBT theory suggests that the reason for its effectiveness is the use of specific techniques (i.e., reviewing homework, asking for specific beliefs, practicing rational responses with patients, and asking patients to keep thought records) that target the purported causes of depression (i.e., depressogenic shemas, negative thoughts, and dysfunctional attitudes). Currently there is little research evidence that supports the specificity of CBT techniques or that supports the notion that specific changes in cognitions as a result of CBT reduce depression. Nevertheless, in general, concrete techniques (i.e., setting agendas, reviewing homework, and practicing rational responses) are clinically useful for depressed patients, as is learning compensatory skills like problem solving.