Blog

The Great Psychotherapy Debate: Since in April, 2015 I review parts of The Great Psychotherapy Debate (Wampold & Imel, 2015) in the PPRNet Blog. This is the second edition of a landmark, and sometimes controversial, book that surveys the evidence for what makes psychotherapy work. You can view parts of the book in Google Books.

Wampold, B.E. & Imel, Z.E. (2015). The great psychotherapy debate: The evidence for what makes psychotherapy work (2^nd edition). New York: Routledge.

Why We Should Care About Allegiance Effects in Psychotherapy Research

Allegiance in psychotherapy refers to the degree to which a researcher or therapist believes that the therapy they are studying or delivering is effective. Clients have an expectation that therapists have an explanation for their disorder and that the therapy used to address that explanation will lead to improvements. On their part, psychotherapists choose a therapeutic approach that is consistent with their understanding of psychological distress. Wampold and Imel argue that therapist allegiance is a common factor across therapies that contributes to good patient outcomes. Although allegiance is an important therapeutic factor, it complicates the conduct of psychotherapy research. In a trial comparing two treatments, for example, researchers and therapists tend to be affiliated with one of the treatments and so they believe in the effectiveness of their treatment. They often do not feel the same way about the comparison treatment, or they may desire that their preferred treatment be more effective than the comparison. In medication trials, this allegiance effect can be controlled by a double blind placebo controlled design in which both therapist and patient are not aware of who is receiving which active medication, or who is receiving a placebo. It is impossible to blind therapists in psychotherapy trials – therapists have to know what treatment they are providing. When doing a meta analytic review of psychotherapy trials, it is possible and relatively easy to identify the allegiance of the researchers in a particular study by looking at their past publications, and by reading what they say about the therapies they are comparing. Often, the developer of a treatment manual is a co-author of the trial. Wampold and Imel review several meta analyses that assess the allegiance effects. In three meta analyses from published in 1980, 1999, and 2013 the correlation between ratings of researcher allegiance and effects of psychotherapy on patient outcomes ranged from moderate to large (r = .26 to r = .85). One interesting meta analysis illustrates the magnitude of this effect. The reviewers looked at 69 studies on self statement monitoring (SSM), a type of cognitive therapy developed by Meichenbaum. The average effect of SSM compared to controls in all studies was d = .53 to d = .74, which is moderate. However, effect sizes found in the studies co-authored by Meichenbaum were nearly twice as large, d = d = 1.23. Being a co-investigator in a study of a therapy that one develops, apparently doubles the effect of the treatment on patient outcomes.

Practice Implications

Therapist allegiance to a treatment is important to the effectiveness of the treatment in that therapist allegiance increases the therapist’s confidence in the treatment’s effectiveness and increases a patient’s expectation of getting better. However, when interpreting psychotherapy trials, especially those that pit one type of therapy against another, it is important to keep in mind the researchers’ allegiance. It is rare to see trials that compare two interventions in which the research team is made of up proponents of the two interventions. However such trials are important and necessary.

Roest, A.M., de Jonge, P., Williames, G.D., de Vries, Y.A., Shoevers, R.A., & Turner, E.H. (2015). Reporting bias in clinical trials investigating second-generation antidepressants in the treatment of anxiety disorders: A report of 2 meta-analyses. JAMA Psychiatry, doi:10.1001/jamapsychiatry.2015.15.

Previous research has shown that the effects of antidepressant medications for treating depression may be over estimated by as much as 35%. This occurs because of publication bias, which refers to the tendency among researchers and editors to prefer to publish positive findings, and also occurs due to the occasional practice of the pharmaceutical industry to suppress negative findings. In these meta analyses, Roest and colleagues assess publication bias in the research of antidepressant medications to treat anxiety disorders (i.e., generalized anxiety disorder [GAD], panic disorder [PD), social anxiety disorder [SAD], post traumatic stress disorder [PTSD], and obsessive compulsive disorder [OCD]). Anxiety disorders are very common in the population, with an estimated year-prevalence of 12%. Second generation antidepressants (i.e., selective serotonin reuptake inhibitors and serotonin-norepinephrine reuptake inhibitors) are the primary pharmacologic treatment for anxiety disorders. Roest and colleagues were also interested in outcome reporting bias, which refers to mis-reporting non-positive findings as if they were positive findings, and spin which refers to interpreting non-positive results as beneficial findings. Positive findings refer to the pharmacological agent significantly outperforming a placebo, and non-positive findings refer to pharmacological agents not significantly outperforming a placebo. Pharmaceutical companies in the US must register any trial with the Food and Drug Administration (FDA) prior to starting the trial if the company wishes to apply for US marketing approval. And so, all medication trials and their findings, whether positive or non-positive must be listed in the FDA register. Despite being listed with the FDA, not all trials and findings get published in peer reviewed journals. This causes a problem for reviews and meta analyses that tend only to focus on published trials, and prescribing physicians tend only to read published trials and reviews. In their meta analyses Roest and colleagues compared findings from all FDA registered medication trials to those that were published in peer reviewed journal. Fifty seven trials were registered with the FDA but only 48 were published. Regarding publication bias, the proportion of studies with positive findings indicating efficacy of antidepressant medications in FDA trials was 72%, whereas the proportion of studies with positive findings in trials published in a journal was 96%. Overall, trials were 5 times more likely to be published if they were positive than if they were non-positive. Regarding outcome reporting bias, 3 of 16 trials that were non-positive in the FDA review were reported as positive in journal publications. Regarding spin, an additional 3 of the 16 non-positive trials interpreted non-positive results as if they were positive. Effect sizes in the FDA data was g = .33 indicating a small average effect size of the medications for anxiety disorders, but the effect size in published journals was g = .38 indicating a small to moderate effect. This represents a 15% over estimation of the effects of the antidepressant medications for anxiety disorders in the published literature.

Practice Implications

The effects of antidepressant medications for anxiety disorders appear to be over estimated by 15% in the published literature. This inflation is not as large as the 35% over estimation in the published literature of the effects of antidepressant medications for depression. By contrast, as I reported in a previous PPRNet Blog, publication bias in psychotherapy trials is small and has little impact on the overall estimate of psychotherapy’s efficacy. Effect sizes for psychological interventions for anxiety disorders are moderate to large, g = .73. Combining medications and psychotherapy only modestly improves efficacy of treatments, and medications may interfere with the efficacy of psychological interventions.

Castonguay, L., Barkham, M., Lutz, W., & McAleavey, A. (2013). Practice-oriented research: Approaches and applications. In M.E. Lambert (Ed.), Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change, 6th Edition (pp. 85-133). New York: Wiley.

In this chapter of the Handbook, Castonguay and colleagues (2013) review research methods and results associated with practice research networks (PRN). There is substantial evidence to show that psychotherapists often are not influenced by research findings when they prepare their case formulations and conduct interventions. As a result, clients may not be benefiting fully from nearly 60 years of research in psychotherapy methods and processes. There may be several explanations for this divide. Clinicians may perceive psychotherapy research, especially the emphasis on empirically supported treatments, as limited in its clinical relevance. Researchers may pay limited attention to concerns of clinicians when developing research strategies and treatment manuals. The end result is that clinicians feel disenfranchised from the research field, and therefore unaffected by the findings. Clinicians may pay more attention to psychotherapy research if they were more involved and “owned” the research and findings. One solution is to develop PRN based on a partnership of practitioners and researchers in which they collaborate on all aspects of a study; i.e., generation of ideas, implementation, and publication. Castonguay and colleagues (2013) report on the research generated by several PRNs in the U.S. The American Psychiatric Institute for Research and Education’s PRN (APIRE-PRN) conducted several studies including: one study that found that compared to White patients, African Americans were less likely to be prescribed second generation antipsychotic medications, which are considered to be the treatment of choice by psychiatrists; and a second study that reported that presence of a personality disorder, low Global Assessment of Functioning scores, and seeing a psychiatrist at a discounted fee was associated with treatment non-compliance. The Pennsylvania Psychological Association PRN (PPA-PRN) conducted several studies, including one study that found that better patient outcomes were associated with: higher expectancy for change among clients, lower client interpersonal problems, greater number of therapy sessions, and lower therapist case load. A second PPA-PRN study that I reported in my August 2013 Blog found that therapist efforts to foster client awareness of their emotions, thoughts, and behaviors were perceived as particularly helpful by both clients and therapists. Finally, the National Drug Abuse Treatment Clinical Trials Network (CTN) conducted several studies with the intent of bringing drug abuse researchers into the real world and creating opportunities for clinicians to participate in research. This network completed over 50 trials. For example, in one trial, researchers found evidence for better retention, treatment engagement, and family functioning for brief strategic family therapy compared to treatment as usual.

Practice Implications

A qualitative study reported by Castonguay and colleagues (2010) indicated that clinician involvement in PRN research fostered new learning as well as a sense of community with other professionals with shared goals. Therapists also indicated that their clients perceived their research participation as intrinsically meaningful and an opportunity to contribute to scientific knowledge. However, participating in a PRN had its challenges as well. Clinicians had to remember detailed procedures, at times practitioners had to depart from their clinical routine, and clinicians had to find time to complete questionnaires and other procedures. Castonguay and colleagues (2010) recommended that practice based research procedures in a PRN remain simple and clear, that clinicians have to have ready access to research staff for consultations, and that incentives have to be built in for clients and clinicians to participate. Studies in which research goals and clinical goals are indistinguishable are most likely to succeed.

Bell, E. C., Marcus, D. K., & Goodlad, J. K. (2013). Are the parts as good as the whole? A meta-analysis of component treatment studies. Journal of Consulting and Clinical Psychology, 81, 722-736.

Component studies (i.e., dismantling treatments or adding to existing treatments) may provide a method for identifying whether specific active ingredients in psychotherapy contribute to client outcomes. In a dismantling design, at least one element of the treatment is removed and the full treatment is compared to this dismantled version. In additive designs, an additional component is added to an existing treatment to examine whether the addition improves client outcomes. If the dismantled or added component is an active ingredient, then the condition with fewer components should yield less improvement. Among other things, results from dismantling or additive design studies can help clinicians make decisions about which components of treatments to add or remove with some clients who are not responding. For example, Jacobson and colleagues (1996) conducted a dismantling study of cognitive-behavioral therapy (CBT) for depression. They compared: (1) the full package of CBT, (2) behavioral activation (BA) plus CBT modification of automatic thoughts, and (3) BA alone. This study failed to find differences among the three treatment conditions. These findings were interpreted to indicate that BA was as effective as CBT, and there followed an increased interest in behavioral treatments for depression. However, relying on a single study to influence practice is risky because single studies are often statistically underpowered and their results are not as reliable as the collective body of research. One way to evaluate the collective research is by meta analysis, which allows one to assess an overall effect size in the available literature (see my November, 2013 blog on why clinicians should rely on meta analyses). In their meta analysis, Bell and colleagues (2013) collected 66 component studies from 1980 to 2010. For the dismantling studies, there were no significant differences between the full treatments and the dismantled treatments. For the additive studies, the treatment with the added component yielded a small but significant effect at treatment completion and at follow-up. These effects were only found for the specific problems that were targeted by the treatment. Effects were smaller and non-non-significant for other outcomes such as quality of life.

Practice Implications

Psychotherapists are sometimes faced with a decision about whether to supplement current treatments with an added component, or whether to remove a component that may not be helping. Adding components to existing treatments leads to modestly improved outcomes at least with regard to targeted symptoms. Removing components appears not to have an impact on outcomes. The findings of Bell and colleagues’ (2013) meta analysis suggest that specific components or active ingredients of current treatments’ have a significant but small effect on outcomes. Some writers, such as Wampold, have argued that the small effects of specific components highlight the greater importance of common factors in psychotherapy (i.e., therapeutic alliance, client expectations, therapist empathy, etc.). This may be especially the case when it comes to improving a patient’s quality of life.

Author email: david.marcus@wsu.edu

Driessen, E., Van, H.L., Don, F.J., Peen, J., Kool, S. ....Dekker, J.J. (2013). The efficacy of cognitive-behavioral therapy and psychodynamic therapy in the outpatient treatment of major depression: A randomized clinical trial. American Journal of Psychiatry, 170, 1041-1050.

Psychotherapy is one of the most widely used treatments for major depression. Unfortunately there is no commercial entity like the pharmaceutical industry to support research and development of psychotherapy. As a result, researchers have limited ability to conduct larger-scale studies of comparative treatment effectiveness, of which there are only a handful. Although psychodynamic therapy (PDT) has been used to treat depressed patients for decades, randomized controlled trials of its efficacy are relatively infrequent. A concurrent problem with outcome research in psychotherapy is that sample sizes tend to be too small to actually test if two treatments are equivalent in what is called an “equivalency trial”. Without large samples, all one can conclude is that two treatments are “not significantly different” (a statistical note: an equivalency trial is planned from the outset to have a large enough sample to test the hypothesis that, with 95% certainty, the effect of one treatment falls within a narrow, predetermined margin of the effect of another treatment). The study by Driessen and colleagues was conducted in several sites in Amsterdam, in which 341 patients seeking outpatient psychotherapy for depression in psychiatric clinics were randomized to PDT or cognitive behavioural therapy (CBT). This is largest trial ever of PDT. Participants received 16 weeks of therapy and then were followed up for 1 year. About 40% of patients started with severe depression. Therapists were 93 experienced and well trained therapists who provided one of the two treatments. The main outcome was remission from depression, defined by achieving a low score on a validated observer rating scale. Post treatment remission rates were 21% for CBT and 24% for PDT, indicating that the treatments were equivalent.

Practice Implications

Cognitive-behavioral therapy (CBT) and short-term PDT provided similar outcomes for patients with a major depressive episode, but remission rates at the end of treatment were low for both treatments. Lower remission rates were likely due to the greater level of severity for these patients compared to those seen in primary care settings. The results highlight that even the best available psychological (and pharmacological) treatments yield modest outcomes for more severely depressed patients. Nevertheless, this rare equivalency trial found that both CBT and PDT were equivalent in terms of outcomes for these patients.

Author email: e.driessen@vu.nl

Munder, T., Brütsch, O., Leonhart, R., Gerger, H., & Barth, J. (2013). Researcher allegiance in psychotherapy outcome research: An overview of reviews. Clinical Psychology Review, 33, 501-511.

Although evidence for the efficacy of psychotherapy is largely uncontested, there remains debate about whether one type of treatment is more effective than another. This debate continues despite a recent American Psychological Association (APA) resolution on the equivalent efficacy of most systematic psychotherapy approaches. There are many aspects to this debate (e.g., some treatments are more researched than others and so appear to be better; symptom focused measurements are more sensitive to change and so may favour one treatment over another; some treatments are more amenable to manualization and short term application; etc.). One element of the debate that has received a lot of attention is researcher allegiance. Researcher allegiance refers to researchers preferring one treatment approach over another, and this preference may bias comparative outcome trials in favour of the preferred therapy. Researcher allegiance is measured by assessing primary researchers’ publication history or by their self-declared preference for a particular therapy approach. There exist 30 meta analyses that assessed researcher allegiance since the 1980s. These meta analyses focused on different therapy types, client populations (adults, children), and research designs (randomized trials, naturalistic effectiveness studies). However, some meta analyses have reported contradictory results for the researcher allegiance effect. This could be due to the different foci of the meta analyses (i.e., different treatment approaches, patient populations, age groups, etc.), and also possibly due the allegiance of those conducting the meta analyses. Munder and colleagues (2013) conducted a mega analysis of these meta analyses. As the name implies, a mega analysis aggregates the findings of available meta analyses. Munder and colleagues found a significant moderate effect of researcher allegiance. Researcher allegiance was consistent across patient populations, age groups, outcome measures, type of study design, and year of publication.

Practice Implications

As the APA resolution indicates, psychotherapy is the informed and intentional application of clinical methods and interpersonal stances derived from established psychological principles. Evidence-based practice in psychotherapy is "the integration of the best available research with clinical expertise in the context of patient characteristics, culture and preferences". The results of this mega analysis undermine the claim of some comparative outcome studies that suggest that one evidence-based psychotherapy is more effective than another.

Author email: tmunder@uni-kassel.de