Blog

Niemeyer, H., Musch, J., & Pietrowsky, R. (2013). Publication bias in meta-analyses of the efficacy of psychotherapeutic interventions for depression. Journal of Consulting and Clinical Psychology, 81, 58-74.

Meta-analyses are important ways of summarizing effects of medical and psychological interventions by aggregating effect sizes across a large number of studies. (Don’t stop reading, I promise this won’t get too statistical). The aggregated effect size from a meta analysis is more reliable than the findings of any individual study. That is why practice guidelines almost exclusively rely on meta analyses when making practice recommendations (see for example the Resources tab on this web site). However meta analyses are only as good as the data (i.e., studies) that go into them (hence, the old adage: “garbage in, garbage out”). For example, if the studies included in a meta analysis are a biased representation of all studies, then the meta analysis results will be unreliable leading to misleading practice guidelines. One problem that leads to unreliable meta analyses is called publication bias. Publication bias often refers to the tendency of peer reviewed journals not to publish studies with non-significant results (e.g., a study showing a treatment is no better than a control condition). Publication bias may also refer to active suppression of data by researchers or industry. Suppression of research results may occur because an intervention’s effects were not supported by the data, or the intervention was harmful to some study participants. In medical research, publication bias can have dire public health consequences (see this TED Talk). There is lots of evidence that publication bias has lead to a significant over-estimation of the effects of antidepressant medications (see Turner et al (2008) New England Journal of Medicine). Does publication bias exist in psychotherapy research, and if so does this mean that psychotherapy is not as effective as we think? A recent study by Niemeyer and colleagues (2013) addressed this question with the most up to date research and statistical techniques. They collected 31 data sets each of which included 6 or more studies of psychotherapeutic interventions (including published and unpublished studies) for depression. The majority of interventions tested were cognitive behavioral therapy, but interpersonal psychotherapy, and brief psychodynamic therapy were also included. The authors applied sophisticated statistical techniques to assess if publication bias existed. (Briefly, there are ways of assessing if the distribution of effect sizes across data sets fall in a predictable pattern called a “funnel plot” – specific significant deviations from this pattern indicate positive or negative publication bias). Niemeyer and colleagues found minimal evidence of publication bias in published research of psychotherapy for depression. This minimal bias had almost no impact on the size of the effect of psychotherapy for depression.

Practice Implications

This is a very important result indicating that despite a minor tendency toward a selective publication of positive results, the efficacy of all reviewed psychotherapy interventions for depression remained substantial, even after correcting for the publication bias. Niemeyer and colleagues’ findings demonstrate that publication bias alone cannot explain the considerable efficacy of psychotherapy for depression. Psychotherapeutic interventions can still be considered efficacious and recommended for the treatment of depression.

Author email address: helen.niemeyer@hhu.de

Handbook of Psychotherapy and Behavior Change: The Handbook of Psychotherapy and Behavior Change is perhaps the most important compendium of psychotherapy research covering a large number of research areas. The Handbook is updated approximately every 10 years, and the most recent 6^th edition was published in January 2013. In the coming months I will review one chapter a month in addition to commenting on psychotherapy research articles. Book chapters have more restrictive copy right rules about distributing content, so I will not provide author email addresses for these chapters. If you are interested, you can view the table of contents on Amazon.

Lambert, M.J. (2013). The efficacy and effectiveness of psychotherapy. In M.J. Lambert (Ed.) Bergin and Garfield’s handbook of psychotherapy and behaviour change (6^th ed.), pp169-218. Hoboken, N.J.: Wiley.

This comprehensive chapter in the Handbook reviews research on the efficacy and effectiveness of psychotherapy. Lambert’s reviews focus on meta-analyses, which is a way of summarizing effect sizes in a research area. The bottom line is that psychotherapy is effective so that 40% to 60% of clients show substantial benefit in controlled research trials, though the effect is likely smaller in routine practice.  Concurrently, a consistent proportion of adults (5% to 10%) deteriorate during psychotherapy. Patients who receive formal treatment are better off than those who receive no treatment, and bona fide treatments are superior to control conditions that provide only some aspects of effective treatment. When psychotherapy is offered by skilful therapists, on average clients experience appreciable gains and return to normal functioning. Fifty percent of patients achieve clinically significant gains after 8 sessions, and 50% achieve recovery after about 20 sessions of psychotherapy. The effects of psychotherapy tend to be long lasting. For example, only 25% treated depressed patients relapse, whereas 50% of those who receive antidepressants relapse. Research continues to support those therapies that have been rigorously tested, and differences in effectiveness between therapy types (e.g., cognitive behavioural therapy (CBT), psychodynamic, interpersonal, etc.) tend to be small or negligible for many disorders. Cognitive behavioural therapy is still the most tested therapy modality, though other treatments are also accumulating evidence of efficacy. Treatment is likely facilitated by a therapeutic relationship that is characterized by trust, understanding, acceptance, kindness, and warmth. The effect of the therapist providing the therapy is at least as large as the effect of different therapy techniques. That is, some therapists are unusually effective, whereas others may not help the majority of patients who seek their services. Continuous monitoring of outcomes and providing regular feedback to the therapist improves the therapy’s effectiveness.

Practice Implications

Providers and patients can be assured that a broad range of formally defined and tested psychotherapies when provided by skilful therapists are likely to result in appreciable gains in clients including a return to normal functioning. Therapy relationships characterized by trust, understanding, acceptance, and warmth can greatly facilitate change in depression, anxiety, inadequacy, and inner conflicts. When making a decision about which therapy to choose, clients would be wise to consider the therapist as a person at least as much as the type of therapy being offered. Treatment efforts should be based on the best evidence available for treatment types, therapist behaviors, and relationship factors. Routinely monitoring the effects of therapy with each patient will give the therapist ongoing information about their effectiveness and may improve their patients’ outcomes.