Blog

Owen, J., Drinane, J. M., Idigo, K. C., & Valentine, J. C. (2015). Psychotherapist effects in meta-analyses: How accurate are treatment effects? Psychotherapy, 52(3), 321-328.

One of the ongoing debates in the psychotherapy research literature has to do with the relative efficacy of psychotherapies. Is psychotherapy brand A (CBT, for example) more effective than psychotherapy brand B (psychodynamic therapy, for example)? The most common way to test this question is with randomized controlled trials (RCTs), in which clients are randomly assigned to treatment condition (brand A or B). This study design controls for systematic bias in the results that may be caused by differences between clients. But what about therapists? We know for example that therapist effects (i.e., differences between therapists) account for approximately 5% to 10% of client outcomes. Therapist effects are often larger than the effect of the empirically supported treatment that is being offered. Yet it is almost unheard of for therapists to be randomized to treatments, so therapist effects are not controlled in most psychotherapy trials. As a result the effects of the differences between therapists get statistically rolled into the treatment effects. As Owen and colleagues point out, the impact of not controlling for therapist effects is that some differences between treatments in an RCT will appear statistically significant when in fact they are not. One can control for the effect of therapist differences, thus providing a more accurate estimate of treatment effects, but this is rarely done in published RCTs. So, when these RCTs are summarized in a meta analysis, the meta analysis results are also affected by ignoring therapist effects. In their study, Owen colleagues did something very clever. They took data from 17 recent meta analyses of RCTs that found differences between two interventions. These included meta analyses of studies comparing: CBT vs alternative treatments, bona fide treatments vs non-bona fide treatments, culturally adapted treatments vs those that were not adapted, etc. There are many other meta analyses that show no differences between treatments, but the authors wanted to focus specifically on the 17 that did show differences. Owen and colleagues statistically estimated what would happen to the original study findings of significant differences between treatments if therapist effects on patient outcomes were controlled. They controlled for three different sizes of therapist effects that accounted for: 5% (small), 10% (medium), or 20% (large) of patient outcomes. Even small therapist effects (5%) reduced the number of significant differences between treatments from 100% to 80%. When psychotherapist effects were estimated to be medium (10% - which is the best estimate based on research), the number of significant differences between treatments dropped to 65%. For large therapist effects (20%), the number of significant treatment differences was only 35%.

Practice Implications

I have argued previously that the psychotherapist matters. Placing more time and effort in developing good reflective practice based on quality information and developing therapist skills like empathy, progress monitoring, and identifying and repairing alliance ruptures will result in better patient outcomes. As Owen and colleagues note, when reading an RCT that claims to find significant differences between psychotherapies, ask yourself if they took into account the effects of differences between therapists.

Tao, K. W., Owen, J., Pace, B. T., & Imel, Z. E. (2015). A meta-analysis of multicultural competencies and psychotherapy process and outcome. Journal of Counseling Psychology, 62(3), 337-350.

Cultural factors shape health-related beliefs, behaviors and values. For decades, many have argued that therapist multicultural competence shapes the therapy process and affects patient outcomes. Some therapists have poorer outcomes with patients of racial/ethnic minorities compared to White patients. Multicultural competence refers to the ability to work effectively across many groups including minority groups. In 2008, an American Psychological Association Task Force detailed recommendations for multicultural competencies. Multiculturally competent providers are those who: expand their knowledge of their client’s background, use culturally relevant interventions, and gain awareness of their own assumptions and the impact of these on their therapeutic work. In this meta analysis, Tao and colleagues aimed to assess the relationship between multicultural competence in therapists with therapy processes and client outcomes. They reviewed 18 studies that included over 1600 clients, the vast majority of whom identified as a racial/ethnic minority. Therapist multicultural competence was assessed by client self report. Therapist multicultural competence was highly correlated with therapy processes like: therapeutic alliance (r = .61), client satisfaction (r = .72), and session depth (r = .58). The association between therapist multicultural competence and client symptom outcomes were moderate in size but significant (r = .29). A separate analysis showed that the relationship between multicultural competence and therapy process variables (alliance, satisfaction, depth) were significantly larger that associations with client outcomes.

Practice Implications

Therapists’ abilities to integrate aspects of their client’s cultural narrative into their interventions significantly accounted for difference in outcomes. In other words, clients who perceived their therapist as more culturally sensitive had better outcomes. This was likely related to more positive therapeutic processes (i.e., alliance, satisfaction, session depth) between clients and therapist dyads, within which clients perceived the therapist as multiculturally sensitive. A provider’s ability to recognize how their own personal backgrounds influence their own and clients’ behaviors will result in better therapy processes and improved client outcomes.

Author email: k.tao@utah.edu

The Great Psychotherapy Debate: Since in April, 2015 I review parts of The Great Psychotherapy Debate (Wampold & Imel, 2015) in the PPRNet Blog. This is the second edition of a landmark, and sometimes controversial, book that surveys the evidence for what makes psychotherapy work. You can view parts of the book in Google Books.

Wampold, B.E. & Imel, Z.E. (2015). The great psychotherapy debate: The evidence for what makes psychotherapy work (2^nd edition). New York: Routledge.

Some therapists achieve better patient outcomes than others. This seems obvious on the surface and yet few people talk about it, and the research literature seems to downplay or ignore this fact. To illustrate the differing outcomes achieved by therapists, I reviewed a unique study in the September 2013 PPRNet Blog. In that study, 10 cases were randomly selected from 700 therapists (N = 7000 patients), and therapist outcomes were assessed by averaging their patient outcomes. Depending on the presenting problem, as many as 67% of therapists were reliably effective, but as many as 16% were reliably harmful. Clearly therapists differ. Yet psychotherapy research typically treats therapists as if they are uniformly effective. In their chapter, Wampold and Imel review some of the research that estimates the therapist’s effect on outcomes. In other words, what is the impact of the particular therapist on the patient outcomes? Even in studies in which: (a) therapists are selected as experts to provide a specific type of therapy (i.e., CBT, psychodynamic, interpersonal, etc.), (b) therapists are highly trained to be adherent to a manual with repeated supervision, and (c) patients are randomly assigned to treatments, there remains a significant amount of variability in therapist outcomes. Indeed in many studies the therapist effect is as large or larger than the effect of the intervention that is being delivered. In other words, which therapist a patient gets in a treatment study matters just as much or more than what type of therapy they receive. This is also true in medication trials. Better psychiatrists (i.e., those with overall better patient outcomes) who gave a placebo had better patient outcomes than poorer psychiatrists who gave the active medication. A recent large meta analysis found that about 5% of patient outcomes in controlled psychotherapy trials was attributable to the therapist, and the effect is as high as 7% in naturalistic settings. For treatment of PTSD, therapist effects are as high as 12%. On the surface these look like small effects, but in reality they can have a big cumulative impact. Therapists with the best and worst outcomes differ dramatically. For example in one large study, the best performing therapists had a patient response rate of 80% compared to the worst performing therapists who had only 20% of their patients improve. Which therapist would you want a loved one to see?

Practice Implications

Wampold and Imel reported that that therapist effects generally exceed the effects of the specific treatment that is being tested or provided. Some therapists consistently achieve better patient outcomes than others. What are the characteristics and actions of effective therapists? Factors like therapist allegiance to the therapy, empathy, and the ability to form and maintain an alliance with clients appear to differentiate therapists who consistently have good patient outcomes versus those whose patients tend to have poor outcomes.

Bradley, R., Greene, J., Russ, E., Dutra, L., & Westen, D. (2005). A multidimensional meta-analysis of psychotherapy for PTSD. American Journal of Psychiatry, 162, 214–227.

The psychotherapy research literature on treatment of post traumatic stress disorder (PTSD) has focused on cognitive behavioral therapy (CBT, with exposure and/or cognitive restructuring) and eye movement desensitization and reprocessing (EMDR). Exposure therapy involves confronting memories of the trauma or cues related to the traumatic event. Other CBT skills include developing skills for anxiety management or challenging distorted cognitions. In EMDR the patient is asked to develop an image of the traumatic event while tracking a bilateral stimulus. Most studies demonstrate the effectiveness of CBT for PTSD in the short term. However, many studies have excluded patients with comorbid conditions. For example, patients with PTSD often also have significant other symptoms like depression, substance abuse, other anxiety disorders, and personality disorders. In this meta analysis, Bradley and colleagues were interested in documenting the overall efficacy of psychological treatments for PTSD. They also wanted to report on any evidence on the long term efficacy of treatments for PTSD, and on evidence of the effects of excluding patients with comorbid disorders. Bradley and colleagues included randomized controlled trials published between 1980 and 2003 (i.e., 26 studies representing 1,535 patients). Also, they looked at outcomes defined in a few ways: change in symptoms as documented by the effect size, proportion of patients no longer meeting diagnostic criteria for PTSD (but who may have residual symptoms), and proportion whose symptoms improved significantly. Across all treatments, the average pre to post effect size was large (d = 1.43), and comparisons to control conditions were also large (d = .83). The results suggested that psychotherapy produced substantial effects for PTSD. Differences between types of therapy (CBT, CBT with exposure, EMDR) were negligible. Fifty six percent of patients no longer met criteria for PTSD, and 65% showed improved symptoms. At follow ups, 62% no longer met diagnostic criteria for PTSD and 32% were deemed improved, but the number of studies with follow up data were small (k = 10) and so the results could be unreliable. Of those who started treatment, 78.9% completed the therapy. Of those who were assessed, 30% were excluded because of suicide risk, drug or alcohol abuse, or “other serious comorbidity”.

Practice Implications

Treatment guidelines from the International Society for Traumatic Stress Studies list a number of effective treatments for PTSD. The evidence for efficacy is strongest at post treatment, and more research is necessary to demonstrate efficacy in the longer term. There is currently little evidence that any one treatment approach is more effective than another, and some researchers are debating whether specific interventions like exposure is necessary to treat PTSD. Bradley and colleagues argue that we need more research on alternative treatments for PTSD and research on patients with multiple symptoms and comorbidities.

Author email: rbradl2@emory.edu

Angus, L., Watson, J.C., Elliott, R., Schneider, K., & Timulak, L. (2015) Humanistic psychotherapy research 1990–2015: From methodological innovation to evidence-supported treatment outcomes and beyond. Psychotherapy Research, 25, 330-347.

In this wide-ranging review, Angus and colleagues provide an overview of humanistic psychotherapy research from 1990-2015. For this blog I will focus on the efficacy research that they review. Humanistic psychotherapy addresses how people can come to know themselves and each other, and to fulfill their aspirations. This type of therapy emphasizes the personal, interpersonal, and contexts within which clients reflect on their relationships with the self, others, and the world. Carl Rogers is probably the best known early proponent of humanistic client centred psychotherapy. Humanistic psychotherapy focuses on a genuinely empathic therapeutic relationship to promote in-therapy client emotional experiencing, emphasizes meaning-making, and is person-centred. One of the questions raised by Angus and colleagues was: are humanistic psychotherapies efficacious. Here they mainly summarize a previous review by Elliot and colleagues (2013). In a meta analysis of 191 studies and over 14,000 clients, humanistic psychotherapies are associated with large pre to post therapy client change (g = .93) which are maintained over early (< 12 months) and late (> 12 months) follow ups. Further, in 31 studies of over 2,000 clients, those who received humanistic therapies show large gains compared to those who receive no treatment (g = .76). In 100 studies of over 6,000 clients, humanistic therapies had equivalent outcomes to other therapies (g = .01), including CBT (22 studies, g = -.06). Humanistic therapy was most effective for interpersonal/relational trauma, and depression (for which it is considered an evidence supported treatment). There is also good evidence for the efficacy of humanistic therapy for psychotic conditions. However, humanistic therapies may be less effective than CBT for anxiety problems.

Practice Implications

Humanistic psychotherapy that focuses on a genuinely empathic therapeutic relationship that emphasizes client emotional experiencing and meaning-making is efficacious for a number of mental health problems. Rogers argued that non-judgemental acceptance, warmth, and congruence were necessary for good client outcomes, and an accumulating body of research is supporting these early propositions. The evidence for the importance of therapist empathy to improve client outcomes is particularly compelling.

Author email: langus@yorku.ca

Kivlighan, D.M., Goldberg, S.B., Abbas, M., Pace, B.T., …Wampold, B.E. (2015). The enduring effects of psychodynamic treatments vis-à-vis alternative treatments: A multilevel longitudinal meta-analysis. Clinical Psychology Review, 40, 1-14.

There is a great deal of evidence that indicates uniform efficacy of a variety of psychotherapies for many common disorders. For example, in the July 2014 PPRNet Blog, I reviewed a meta-analysis comparing 7 psychotherapies for depression indicating no differences between the various treatments in terms of patient outcomes. Nevertheless proponents of cognitive behavioural therapy have claimed superiority to alternative treatments for decades. On the other hand proponents of psychodynamic therapies have argued that these treatments focus on personality change rather than symptoms, and so benefits of psychodynamic therapies will be longer lasting. In this meta analysis, Kivlighan and colleagues put these claims to the test. They selected studies in which a psychodynamic therapy was compared to one or more alternative treatment. Both the psychodynamic therapy and the alternative (most often CBT) had to be judged as “bona fide” therapies by independent raters (i.e., they had to be therapies that were delivered in a manner in which they could be expected to be effective by clients and therapists). Outcomes not only included specific symptoms (e.g., depression), but also non-targeted outcomes (e.g., improved self esteem in a study of treatment of anxiety), and personality outcomes. Effect sizes for outcomes were assessed at post-treatment and also at follow-ups. Twenty five studies directly comparing psychodynamic and non-psychodynamic therapies were included, representing 1690 patients. At post treatment, no significant differences were found between psychodynamic and non-psychodynamic treatments on targeted outcomes, non-targeted outcomes, and personality measures (all gs < .10). There was also no significant or meaningful effect of time to follow up on outcomes, indicating no differences between treatment types at any of the follow up periods.

Practice Implications

Psychodynamic and non-psychodynamic treatments were equally effective at post treatment and at follow ups for all outcomes, including personality variables. This challenges the belief that psychodynamic treatments uniquely affect personality and have longer lasting effects compared to other treatments. It also challenges the notion that CBT (by far the most common comparison treatment) is a superior therapy for patient outcomes. Pan-theoretical psychotherapy factors (client contributions, expectations, therapeutic alliance) may be more promising factors in understanding the long term benefits of psychotherapy. 

Author email: kivlighan@wisc.edu