Blog

Niemeyer, H., Musch, J., & Pietrowsky, R. (2013). Publication bias in meta-analyses of the efficacy of psychotherapeutic interventions for depression. Journal of Consulting and Clinical Psychology, 81, 58-74.

Meta-analyses are important ways of summarizing effects of medical and psychological interventions by aggregating effect sizes across a large number of studies. (Don’t stop reading, I promise this won’t get too statistical). The aggregated effect size from a meta analysis is more reliable than the findings of any individual study. That is why practice guidelines almost exclusively rely on meta analyses when making practice recommendations (see for example the Resources tab on this web site). However meta analyses are only as good as the data (i.e., studies) that go into them (hence, the old adage: “garbage in, garbage out”). For example, if the studies included in a meta analysis are a biased representation of all studies, then the meta analysis results will be unreliable leading to misleading practice guidelines. One problem that leads to unreliable meta analyses is called publication bias. Publication bias often refers to the tendency of peer reviewed journals not to publish studies with non-significant results (e.g., a study showing a treatment is no better than a control condition). Publication bias may also refer to active suppression of data by researchers or industry. Suppression of research results may occur because an intervention’s effects were not supported by the data, or the intervention was harmful to some study participants. In medical research, publication bias can have dire public health consequences (see this TED Talk). There is lots of evidence that publication bias has lead to a significant over-estimation of the effects of antidepressant medications (see Turner et al (2008) New England Journal of Medicine). Does publication bias exist in psychotherapy research, and if so does this mean that psychotherapy is not as effective as we think? A recent study by Niemeyer and colleagues (2013) addressed this question with the most up to date research and statistical techniques. They collected 31 data sets each of which included 6 or more studies of psychotherapeutic interventions (including published and unpublished studies) for depression. The majority of interventions tested were cognitive behavioral therapy, but interpersonal psychotherapy, and brief psychodynamic therapy were also included. The authors applied sophisticated statistical techniques to assess if publication bias existed. (Briefly, there are ways of assessing if the distribution of effect sizes across data sets fall in a predictable pattern called a “funnel plot” – specific significant deviations from this pattern indicate positive or negative publication bias). Niemeyer and colleagues found minimal evidence of publication bias in published research of psychotherapy for depression. This minimal bias had almost no impact on the size of the effect of psychotherapy for depression.

Practice Implications

This is a very important result indicating that despite a minor tendency toward a selective publication of positive results, the efficacy of all reviewed psychotherapy interventions for depression remained substantial, even after correcting for the publication bias. Niemeyer and colleagues’ findings demonstrate that publication bias alone cannot explain the considerable efficacy of psychotherapy for depression. Psychotherapeutic interventions can still be considered efficacious and recommended for the treatment of depression.

Author email address: helen.niemeyer@hhu.de

Handbook of Psychotherapy and Behavior Change: The Handbook of Psychotherapy and Behavior Change is perhaps the most important compendium of psychotherapy research covering a large number of research areas related to psychotherapy. Starting in March 2013, I will review one chapter a month in addition to reviewing psychotherapy research articles. Book chapters have more restrictive copy right rules than journal articles, so I will not provide author email addresses for these chapters. If you are interested, the Handbook table of content can be viewed on Amazon.

Forand, N.R., DeRubeis, R.J., & Amsterdam, J.D. (2013). Combining medication and psychotherapy in the treatment of major mental disorders. In M.J. Lambert (Ed.) Bergin and Garfield’s handbook of psychotherapy and behaviour change (6^th ed.), pp. 735-774. Hoboken, N.J.: Wiley.

This comprehensive chapter covers evidence for combining medication and psychotherapy for several disorders. This month I report on the section of the chapter on anxiety disorders. Monotherapy of medication or psychotherapy are each effective in treating anxiety disorders, though relapse rates can be high. Simultaneously combining medications and psychotherapy is a common practice that is endorsed by several treatment guidelines. Some may also believe that medication and psychotherapy have additive effects or that those who do not respond to one treatment might respond simultaneously to the other. For panic disorder, short term outcomes slightly favour combined therapy of medications (e.g., antidepressants like SSRIs) and psychotherapy (i.e., that often include exposure). However, long term outcome data indicate that combined treatment was no different than cognitive behavioural therapy (CBT) alone. There is also evidence that medications may interfere with exposure-based treatment of panic disorder so that relapse is greater with combination therapy. It is possible for example that medications may suppress fear-related cognitions thus preventing encoding of corrective information, and/or medication may inhibit extinction learning by suppressing cortisol secretion (in the short term) that facilitates consolidation of memories. The evidence for combining medication and psychotherapy for social anxiety disorder, post traumatic stress disorder, generalized anxiety disorder (GAD), and obsessive compulsive disorder are more mixed but still not clearly supportive of long term superiority of simultaneously combining medications and psychotherapy. Other combination approaches appear to show more promise. For example, there is better evidence for starting with a monotherapy initially and adding an alternative therapy for non-responders. Starting with medications first may allow allows cortisol to normalize over time perhaps reducing medication-induced inhibition of extinction learning. Then treatments such as exposure based CBT or brief dynamic therapy for GAD may be additionally helpful to those who do not respond to medication alone. The existing trials tend not to show evidence of incremental benefit of adding medication after initiating psychotherapy. CBT may be effective in helping individuals taper medications while maintaining treatment gains.

Practice implications

Simultaneously combining medication and psychotherapy for anxiety disorders may be common practice. There is an overall lack of evidence that combining treatments improves outcomes, especially in the longer term. Evidence points to medications interfering with the effectiveness of psychotherapy when they are initiated simultaneously. Compared to monotherapy, combined treatments are more complex, time-consuming, expensive, and expose the patient to increased side effect risk. Combination treatments may be best reserved for those who are refractory to initial monotherapy.