Blog

Handbook of Psychotherapy and Behavior Change: Starting in March 2013 I will review one chapter a month from the Handbook of Psychotherapy and Behavior Change in addition to reviewing psychotherapy research articles. Book chapters have more restrictive copy right rules than journal articles, so I will not provide author email addresses for these chapters. If you are interested, the Handbook table of content and sections of the book can be read on Google Books.

Baldwin, S. & Imel, Z.E. (2013). Therapist effects. In M.E. Lambert (Ed.), Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change, 6^th Edition (pp. 258-297). New York: Wiley.

Does it matter that some therapists are more effective than others? Can less effective therapists be trained to improve their outcomes and relationship quality with patients?  These are important questions not only for our patients’ well-being but also for the long term survival of psychotherapy as a health enterprise. If we do not measure outcomes and help therapists who are less effective, stakeholders (i.e., clients, families, agencies, insurance companies) may stop paying for the services. In the September 2013 blog I discussed a large study that showed that a few therapists were reliably harmful and some therapists were reliably helpful to their patients. That study also reported that most therapists were effective in 5 of 12 problem domains for which their patients sought help. What these findings and the Handbook chapter by Baldwin and Imel (2013) show is that there are significant between-therapist effects (i.e., therapists differed from each other on patient outcomes) and within-therapist effects (i.e., therapist outcomes within their own caseload differed based on the patients’ problems). Baldwin and Imel (2013) reported on their meta analysis in which between-therapist differences accounted for 5% of the outcome variance. That seems small, but it’s not. One study, for example, estimated that for each 100 patients that would be treated, the worst therapist compared to the best therapist would have 6 more patients who deteriorated. I would prefer my loved ones to be seen by the best therapist, even if the difference between best and worst is only 5%. Nevertheless, 95% of the variance in outcomes is within the therapist’s caseload. That is, the patient, other contextual variables, and the therapist-patient relationship are by far the biggest contributors to outcome. As Baldwin and Imel point out, not only are some therapists are more effective for some patients and not others, but also some therapists are better at developing a therapeutic relationship with some patients than with others. Baldwin and Imel reported that, on average, 9% of the variance in the quality of the therapeutic alliance is associated with the therapist – that’s a clinically meaningful effect.

Practice Implications

As Baldwin and Imel (2013) state, ignoring therapist accountability is detrimental to patients and to the mental health field in general. If stakeholders do not see evidence of positive outcomes, then they will withdraw funding, and patients will have even less access to services. Therapists differ in their outcomes, and outcomes also differ within each therapist’s caseload. If a primary goal is to improve therapist performance and patient outcomes, then therapists need to measure outcomes and therapeutic relationship quality. This knowledge about performance with specific patients can help therapists seek continuing education and training to improve outcomes and therapeutic alliances with specific patients for whom the therapist is less effective. This may require continuous outcome monitoring and real-time feedback to therapists regarding their patients’ outcomes (see my September 2013 blog in identifying clients who might deteriorate).

Driessen, E., Van, H.L., Don, F.J., Peen, J., Kool, S. ....Dekker, J.J. (2013). The efficacy of cognitive-behavioral therapy and psychodynamic therapy in the outpatient treatment of major depression: A randomized clinical trial. American Journal of Psychiatry, 170, 1041-1050.

Psychotherapy is one of the most widely used treatments for major depression. Unfortunately there is no commercial entity like the pharmaceutical industry to support research and development of psychotherapy. As a result, researchers have limited ability to conduct larger-scale studies of comparative treatment effectiveness, of which there are only a handful. Although psychodynamic therapy (PDT) has been used to treat depressed patients for decades, randomized controlled trials of its efficacy are relatively infrequent. A concurrent problem with outcome research in psychotherapy is that sample sizes tend to be too small to actually test if two treatments are equivalent in what is called an “equivalency trial”. Without large samples, all one can conclude is that two treatments are “not significantly different” (a statistical note: an equivalency trial is planned from the outset to have a large enough sample to test the hypothesis that, with 95% certainty, the effect of one treatment falls within a narrow, predetermined margin of the effect of another treatment). The study by Driessen and colleagues was conducted in several sites in Amsterdam, in which 341 patients seeking outpatient psychotherapy for depression in psychiatric clinics were randomized to PDT or cognitive behavioural therapy (CBT). This is largest trial ever of PDT. Participants received 16 weeks of therapy and then were followed up for 1 year. About 40% of patients started with severe depression. Therapists were 93 experienced and well trained therapists who provided one of the two treatments. The main outcome was remission from depression, defined by achieving a low score on a validated observer rating scale. Post treatment remission rates were 21% for CBT and 24% for PDT, indicating that the treatments were equivalent.

Practice Implications

Cognitive-behavioral therapy (CBT) and short-term PDT provided similar outcomes for patients with a major depressive episode, but remission rates at the end of treatment were low for both treatments. Lower remission rates were likely due to the greater level of severity for these patients compared to those seen in primary care settings. The results highlight that even the best available psychological (and pharmacological) treatments yield modest outcomes for more severely depressed patients. Nevertheless, this rare equivalency trial found that both CBT and PDT were equivalent in terms of outcomes for these patients.

Author email: e.driessen@vu.nl

Munder, T., Brütsch, O., Leonhart, R., Gerger, H., & Barth, J. (2013). Researcher allegiance in psychotherapy outcome research: An overview of reviews. Clinical Psychology Review, 33, 501-511.

Although evidence for the efficacy of psychotherapy is largely uncontested, there remains debate about whether one type of treatment is more effective than another. This debate continues despite a recent American Psychological Association (APA) resolution on the equivalent efficacy of most systematic psychotherapy approaches. There are many aspects to this debate (e.g., some treatments are more researched than others and so appear to be better; symptom focused measurements are more sensitive to change and so may favour one treatment over another; some treatments are more amenable to manualization and short term application; etc.). One element of the debate that has received a lot of attention is researcher allegiance. Researcher allegiance refers to researchers preferring one treatment approach over another, and this preference may bias comparative outcome trials in favour of the preferred therapy. Researcher allegiance is measured by assessing primary researchers’ publication history or by their self-declared preference for a particular therapy approach. There exist 30 meta analyses that assessed researcher allegiance since the 1980s. These meta analyses focused on different therapy types, client populations (adults, children), and research designs (randomized trials, naturalistic effectiveness studies). However, some meta analyses have reported contradictory results for the researcher allegiance effect. This could be due to the different foci of the meta analyses (i.e., different treatment approaches, patient populations, age groups, etc.), and also possibly due the allegiance of those conducting the meta analyses. Munder and colleagues (2013) conducted a mega analysis of these meta analyses. As the name implies, a mega analysis aggregates the findings of available meta analyses. Munder and colleagues found a significant moderate effect of researcher allegiance. Researcher allegiance was consistent across patient populations, age groups, outcome measures, type of study design, and year of publication.

Practice Implications

As the APA resolution indicates, psychotherapy is the informed and intentional application of clinical methods and interpersonal stances derived from established psychological principles. Evidence-based practice in psychotherapy is "the integration of the best available research with clinical expertise in the context of patient characteristics, culture and preferences". The results of this mega analysis undermine the claim of some comparative outcome studies that suggest that one evidence-based psychotherapy is more effective than another.

Author email: tmunder@uni-kassel.de

Kraus, D. R., Castonguay, L., Boswell, J. F., Nordberg, S. S., & Hayes, J. A. (2011). Therapist effectiveness: Implications for accountability and patient care. Psychotherapy Research, 21, 267-276.

Some patients benefit from psychotherapy, some do not, and a few get worse. Research has indicated that patient motivation, client-therapist match, and client characteristics might be associated with better or worse client outcomes. What about the contribution of the therapist? Do some therapists consistently have patients with better outcomes or with worse outcomes? Are consistently effective therapists effective for most patient problem areas or only some? Answers to these questions have important public health, funding, continuing education, and training implications. In a large study conducted in the U.S., Kraus and colleagues assessed 12 patient domains (sexual functioning, work functioning, violence, social functioning, anxiety, substance abuse, psychosis, quality of life, sleep, suicidality, depression, and mania) with a standardized reliable measure (the Treatment Outcome Package). The measure was used in a variety of public and private clinics and practices. Almost 700 therapists were sampled (including social workers 43%, mental health counsellors 35%, psychologists 10%, others 12%), with an average of 11 years experience. Ten cases were selected from each therapist caseload, so almost 7000 patients were included that received at least 16 sessions of therapy (16 sessions is an adequate dose for 50% of patients to improve – see my August, 2013 blog). The patients were, for the most part, representative of a typical caseload with regard to age, sex, and problem area as compared to previous national (U.S.) research. The authors used a reliable change index to classify patients as reliably improved, unchanged, or reliably worsened. The reliable change index is a way of assessing if change from session 1 to 16 on average exceeded the scale’s measurement error so that the change was considered reliable (i.e., not due to error). Reliable change for each therapist’s 10 patients was calculated so that a therapist could be classified as “effective” (i.e., on average their patients reliably improved), “ineffective” (i.e., on average their patients did not change), or “harmful” (i.e., on average their patients reliably worsened). The frequency of effective therapists ranged from a low of 29% in treating symptoms of sexual dysfunction to a high of 67% in treating symptoms of depression. Harmful therapists ranged from a low of 3% in treating depressive symptoms to a high of 16% in treating symptoms of substance abuse and violence. When looking at competency areas (i.e., areas of reliable effectiveness), the median number of areas of therapist competence was 5 out of 12 problem areas. Only 1 therapist of the approximately 700 therapists was competent in 11 of 12 domains, and none were competent in all 12 domains. Being effective in one domain was not correlated with effectiveness in another domain. So, one cannot infer that if a therapist was effective in treating depression he or she would also be effective in treating social dysfunction, for example.

Practice Implications

There was tremendous variability in therapist skill and areas of competence in this very large sample of therapists. Between 3% and 16% of therapists were classified as reliably harmful to their patients, and between 29% and 67% were reliably effective depending on the problem area they were treating. Therapists who were effective in one domain could be harmful in another. Most therapists had some areas in which they were consistently effective, usually around 5. However, as indicated by previous research, without routine measurement, therapists may not be aware of clients for whom they are consistently helpful or harmful. Routine monitoring of outcomes could guide the matching of client problems to therapists, and could direct therapists to areas for continuing education, training, or personal therapy.

Author email: dkraus@bhealthlabs.com

Handbook of Psychotherapy and Behavior Change: Starting in March 2013 I will review one chapter a month from thenHandbook of Psychotherapy and Behavior Change in addition to reviewing psychotherapy research articles. Book chapters have more restrictive copy right rules than journal articles, so I will not provide author email addresses for these chapters. If you are interested, the Handbook table of content can be viewed on Amazon.

Burlingame, G.M., Strauss, B., & Joyce, A.S. (2013). Change mechanisms and effectiveness of small group treatments. In M.J. Lambert (Ed.) Bergin and Garfield’s handbook of psychotherapy and behaviour change (6^thed.), pp. 640-689. Hoboken, N.J.: Wiley.

Group treatments are the most common types of interventions offered in community, organizational, institutional, and hospital settings. They occur in many contexts including: outpatients, inpatients, day hospital, private practice, community health, support groups, drop-in centres, and educational organizations. Despite the extent of their application, group treatments receive relatively little research attention compared to individual psychotherapy or medication interventions. (Not to mention the pervasive and mistaken notion that group therapy is like doing individual therapy with 8-10 patients at once, or that individual therapy training is sufficient to be expert in group therapy). There are many reasons for this relatively lower amount of research, including the lack of expertise in and understanding of group practice among clinical researchers, and the substantially greater difficulty in running a clinical trial of group therapy (of the latter I have ample experience and war wounds). Nevertheless, Burlingame and colleagues summarized more than 250 studies that estimated the efficacy or effectiveness of group therapy for 12 disorders or populations. The findings indicate good or excellent evidence for the efficacy of group treatments for many disorders or patient groups (e.g., panic, social phobia, OCD, eating disorders, substance abuse, trauma related disorders, coping with breast cancer, schizophrenia, and personality disorders). There are also promising results for other disorders (e.g., mood, pain, and inpatients). Although there are substantially more studies on group CBT, most studies that compare different models (including IPT, psychodynamic, DBT, etc.) often produce equivalent outcomes. There is also lots of evidence that group therapy is as effective as individual therapy or medications for most disorders. In one U.S. study on panic disorder, group psychotherapy was the most cost effective (i.e., cost per rate of improvement) of the interventions ($246) compared to individual therapy ($565) and medications ($447). There is also research on the effects of specific characteristics of groups. For example, research on group composition (i.e., heterogenous vs homogeneous in terms of patient population or functioning) has produced mixed results, though there is emerging evidence that heterogeneous groups tend to benefit those who are lower functioning. Further, research on group cohesion (i.e., the bond between the individual and the group) which is a construct related to but distinct from alliance, is positively associated with treatment outcomes with a moderate effect size.

Practice Implications

Group treatments are as effective as individual therapy or medications, and are likely more cost effective. However group therapy is more complicated to practice and to study. Burlingame and colleagues suggest using empirically validated interventions, and ongoing assessment of client outcomes. They also suggest following the American Group Psychotherapy Association (AGPA) practice guidelines (see the Resources page on our web site), that include best practices for creating a successful group, appropriately selecting clients, preparing clients for group, evidence based interventions, and ethics issues related to group practice. Finally, Burlingame and colleagues emphasize using AGPA recommended measures and resources in developing and assessing a therapy group. These include: (1) group selection and group preparation which may involve handouts for group leaders and members about what to expect and how to get the most from group therapy; (2) assessing group processes repeatedly during group therapy using measures like the Therapeutic Factors Inventory or the Working Alliance Inventory; and (3) measuring client outcomes by using an instrument like the Outcome Questionnaire-45. Repeated measurement and feedback of processes and outcomes to the therapist may improve the group’s effectiveness.

Niemeyer, H., Musch, J., & Pietrowsky, R. (2013). Publication bias in meta-analyses of the efficacy of psychotherapeutic interventions for depression. Journal of Consulting and Clinical Psychology, 81, 58-74.

Meta-analyses are important ways of summarizing effects of medical and psychological interventions by aggregating effect sizes across a large number of studies. (Don’t stop reading, I promise this won’t get too statistical). The aggregated effect size from a meta analysis is more reliable than the findings of any individual study. That is why practice guidelines almost exclusively rely on meta analyses when making practice recommendations (see for example the Resources tab on this web site). However meta analyses are only as good as the data (i.e., studies) that go into them (hence, the old adage: “garbage in, garbage out”). For example, if the studies included in a meta analysis are a biased representation of all studies, then the meta analysis results will be unreliable leading to misleading practice guidelines. One problem that leads to unreliable meta analyses is called publication bias. Publication bias often refers to the tendency of peer reviewed journals not to publish studies with non-significant results (e.g., a study showing a treatment is no better than a control condition). Publication bias may also refer to active suppression of data by researchers or industry. Suppression of research results may occur because an intervention’s effects were not supported by the data, or the intervention was harmful to some study participants. In medical research, publication bias can have dire public health consequences (see this TED Talk). There is lots of evidence that publication bias has lead to a significant over-estimation of the effects of antidepressant medications (see Turner et al (2008) New England Journal of Medicine). Does publication bias exist in psychotherapy research, and if so does this mean that psychotherapy is not as effective as we think? A recent study by Niemeyer and colleagues (2013) addressed this question with the most up to date research and statistical techniques. They collected 31 data sets each of which included 6 or more studies of psychotherapeutic interventions (including published and unpublished studies) for depression. The majority of interventions tested were cognitive behavioral therapy, but interpersonal psychotherapy, and brief psychodynamic therapy were also included. The authors applied sophisticated statistical techniques to assess if publication bias existed. (Briefly, there are ways of assessing if the distribution of effect sizes across data sets fall in a predictable pattern called a “funnel plot” – specific significant deviations from this pattern indicate positive or negative publication bias). Niemeyer and colleagues found minimal evidence of publication bias in published research of psychotherapy for depression. This minimal bias had almost no impact on the size of the effect of psychotherapy for depression.

Practice Implications

This is a very important result indicating that despite a minor tendency toward a selective publication of positive results, the efficacy of all reviewed psychotherapy interventions for depression remained substantial, even after correcting for the publication bias. Niemeyer and colleagues’ findings demonstrate that publication bias alone cannot explain the considerable efficacy of psychotherapy for depression. Psychotherapeutic interventions can still be considered efficacious and recommended for the treatment of depression.

Author email address: helen.niemeyer@hhu.de