Blog

Okiishi, J., Lambert, M. J., Nielsen, S. L., & Ogles, B. M. (2003). Waiting for supershrink: An empirical analysis of therapist effects. Clinical Psychology & Psychotherapy, 10(6), 361-373.

Much of psychotherapy research has focused on searching for effective psychotherapies rather than focusing on effective psychotherapists. Research on psychotherapies generally assumes that therapists are equally effective or relatively less important to patient outcomes than the interventions themselves. Therapists in clinical trials are trained to follow a manual in an attempt to reduce the therapists’ impact on patient outcomes, and to focus the study on the specific ingredients of the therapy itself. However, research indicates that the degree to which a therapist follows a manual has little bearing on patient outcomes, and that therapists do differ in terms of their patients’ outcomes. In one large study, between 33% and 65% of therapists was ineffective or harmful. Okiishi and colleagues asked if it is possible to identify highly effective therapists (“super-shrinks”) and highly ineffective therapists (“pseudo-shrinks”) based on their patients’ outcomes. The therapists were 56 men and women who treated 1779 clients in a university counselling centre. Each therapist saw at least 15 clients, so that there was a good sampling of therapists’ outcomes across a variety of clients. Therapists had a range of experience, training, and theoretical orientations. Clients were adults who had moderate to severe problems with anxiety, depression, or adjustment. Outcomes were measured after every session, and the average number of sessions was 5.16 (SD = 7.20). On average clients improved so that their level of distress significantly declined. Therapist characteristics (sex, experience, training background, theoretical orientation) did not predict patient outcomes. However, client change varied significantly, so that some clients improved at a faster rate than others, some did not change, and some got worse. There were no differences between therapists in their clients’ level of distress, so therapists had equivalent caseloads in terms of client initial distress. However, therapists significantly differed from each other in terms of their clients’ outcomes. For example, the top 3 therapists consistently had clients who got better (super-shrinks), and the bottom 3 therapists consistently had clients who got worse (pseudo-shrinks).

Practice Implications

One would hope that a loved one would get to see a “super-shrink” therapist, since these therapists seem to consistently have clients who do well in therapy. But what about the average or “pseudo-shrink” therapist– what can be done to elevate their skills and their patients’ outcomes? We’ve discussed in this blog several things therapists can do to improve their outcomes, including: using progress monitoring in their practice, receiving training focused on deliberate practice, and seeking out specific continuing education around developing, maintaining, and repairing the therapeutic alliance.

Solstad, S.M., Castonguay, L.G., & Moltu, C. (2018). Patients’ experiences with routine outcome monitoring and clinical feedback systems: A systematic review and synthesis of qualitative empirical literature. Psychotherapy Research. doi=10.1080/10503307.2017.1326645.

Routine outcome monitoring or progress monitoring involves assessing client outcomes or the therapeutic alliance on a weekly basis in psychotherapy, and then giving feedback to the therapist about how the client is doing relative to the previous week and relative to similar clients. Research on progress monitoring indicates that it improves outcomes and it reduces by half the number of clients who might get worse. Despite its benefits, many therapists are not aware of progress monitoring or are reluctant to use the procedure. Some have expressed concerns that progress monitoring could interfere with the therapeutic relationship. However, very few studies have asked clients about their experiences of progress monitoring. In this synthesis of qualitative studies, Solstad and colleagues reviewed 16 studies in which clients were interviewed about their experiences of progress monitoring. The authors used a procedure in which they identified common themes across the studies and categorized client statements within those themes (e.g., thematic analysis). The authors were interested in identifying what were the hindering and helpful processes in clients’ experiences of their therapists’ use of progress monitoring. Four main themes emerged from the research. First, some clients voiced suspicion of how the progress monitoring data was going to be used and why the procedure was implemented. That is, clients sometimes felt that filling out questionnaires weekly was mainly a bureaucratic exercise, or possibly a means to justify reducing services. Second, some clients felt the questionnaires were not flexible enough to capture the complexity of mental health and of client concerns. The questionnaires often focused on symptoms, but clients were also interested in the therapeutic relationship, family, and social functioning. Third, some clients wanted to be more fully informed about the rationale for progress monitoring so that they could feel more empowered to define their own outcomes and treatment plans. Fourth, some clients found progress monitoring to help them to see graphically their own progress, to become more engaged in treatment planning, and to participate in collaborative and reflective discussions with their therapist.

Practice Implications

If psychotherapists choose to use progress monitoring in their practices, they should make sure that clients know what the data will be used for and that the exercise is not just a bureaucratic process. The practice of outcome or progress monitoring can be used to stimulate reflection not only in the therapist but also in the client. Reviewing the client data together might enhance conversations about therapy, the therapeutic relationship, and help to establish realistic goals for therapy. Therapists might consider not only measuring symptom progress repeatedly, but also measuring the working alliance on a regular basis.

Grenon, R., Carlucci, S., Brugnera, A., Schwartze, D., … Tasca, G. A. (2018). Psychotherapy for eating disorders: A meta-analysis of direct comparisons, Psychotherapy Research, DOI: 10.1080/10503307.2018.1489162

Eating disorders can cause a great deal of physical and mental impairment because of the severity of the symptoms and because of comorbid conditions like depression, anxiety, substance use, and others. Anorexia nervosa (AN) occurs in about 0.5% of the population, bulimia nervosa (BN) occurs in about 1.5% of the population, and binge-eating disorder (BED) occurs in about 3.5% of the population. Treatment guidelines include both cognitive behavioral therapy (CBT) and interpersonal psychotherapy (IPT) as front line interventions for BN and BED. However, results from previous meta analyses of psychological treatments for eating disorders were confounded by not focusing exclusively on randomized controlled trials, mixing studies of adult and adolescent samples, combining an array of outcomes rather than separately reporting primary (eating disorder symptoms) and secondary (interpersonal problems, depression) outcomes, and not distinguishing between bona fide psychotherapies (like CBT, IPT, psychodynamic therapy, and others) from non-bona fide treatments (like self help, behavioral weight loss supportive counseling). Grenon and colleagues conducted a meta analysis of psychotherapies for eating disorders to examine if: psychotherapy is effective compared to a wait list, if bona fide psychotherapy and non-bona fide treatment differ in outcomes, and if one type of psychotherapy (i.e., CBT) was more effective than other bona fide psychotherapies (like IPT, behavior therapy, psychodynamic therapy, dialectical behavior therapy). Their meta analysis included 35 randomized controlled trials of direct comparisons. Psychotherapy was significantly more effective than a wait-list control at post treatment, so that 53.89% of patients were abstinent of symptoms after psychotherapy compared to only 8.92% who were abstinent in the wait-list group. Bona fide psychotherapies (51% abstinent) were significantly more effective than non-bona fide treatments (40% abstinent) at post treatment, and dropout in bona fide psychotherapies (17.5%) was significantly lower than in non-bona fide treatment (29.1%). Further, the difference between CBT and other bona fide psychotherapies was not significant.

Practice Implications

Psychotherapy for eating disorders are effective for patients with BN or BED. There were too few studies of those with AN to come to any conclusions about their treatment. Patients with BN or BED are best treated with a bona fide psychotherapy that involves face to face psychological therapy like CBT, IPT, psychodynamic therapy, dialectical behavior therapy, or behavior therapy. Non-bona fide treatments like self help, behavioral weight loss, and supportive counseling should only be used as an adjunct to bona fide psychotherapy for eating disorders.

Jennissen, S., Huber, J., Ehrenthal, J.C., Schauenburg, H., & Dinger, U. (2018). Association between insight and outcome of psychotherapy: Systematic review and meta-analysis. The American Journal of Psychiatry. Published Online: https://doi.org/10.1176/appi.ajp.2018.17080847

For many authors, one of the purported mechanisms of change in psychotherapy is insight. In fact, the utility of insight for clients with mental health problems was first proposed over 120 years ago by Freud and Breuer. Briefly, insight refers to higher levels of self-understanding that might result in fewer negative automatic reactions to stress and other challenges, more positive emotions, and greater flexibility in cognitive and interpersonal functioning. Although insight is a key factor in some psychodynamic models, it also plays a role in other forms of psychotherapy. Experiential psychotherapy emphasises gaining a new perspective through experiencing, and for CBT insight relates to becoming more aware of automatic thoughts. Jennissen and colleagues defined insight as patients understanding: the relationship between past and present experiences, their typical relationship patterns, and the associations between interpersonal challenges, emotional experiences, and psychological symptoms. In this study, Jennissen and colleagues conducted a systematic review and meta analysis of the insight-outcome relationship, that is the relationship between client self-understanding and symptom reduction. They reviewed studies of adults seeking psychological treatment including individual or group therapy. The predictor variable was an empirical measure of insight assessed during treatment but prior to when final outcomes were evaluated. The outcome was some reliable and empirical measure related to symptom improvement, pre- to post- treatment. The review turned up 22 studies that included over 1100 patients mostly with anxiety or depressive disorders who attended a median of 20 sessions of therapy. The overall effect size of the association between insight and outcome was r = 0.31 (95% CI=0.22–0.40, p < 0.05), which represents a medium effect. Moderator analyses found no effect of type of therapy or diagnosis on this mean effect size, though the power of these analyses was low.

Practice Implications

The magnitude of the association between insight and outcome is similar to the effects of other therapeutic factors such as the therapeutic alliance. When gaining insight, patients may achieve a greater self-understanding, which allows them to reduce distorted perceptions of themselves, and better integrate unpleasant experiences into their conscious life. Symptoms may be improved by self-understanding because of the greater sense of control and master that it provides, and by the new solutions and adaptive ways of living that become available to clients.

Author email: Simone.Jennissen@med.uni-heidelberg.de

Reese, R. J., Duncan, B. L., Bohanske, R. T., Owen, J. J., & Minami, T. (2014). Benchmarking outcomes in a public behavioral health setting: Feedback as a quality improvement strategy. Journal of Consulting and Clinical Psychology, 82(4), 731-742.

Psychotherapy has demonstrated its efficacy in randomized controlled trials. But do these findings in highly controlled studies translate to everyday practice in publicly funded agencies that treat low income clients? Previous research in the US showed that outcomes of treatment-as-usual in public behavioural health agencies are generally not positive, so that only 20 to 35% of clients reliably improved. One approach to improving outcomes is to transport specific evidence-based treatments into practice settings. For example, research on applying CBT for panic and depression in a publicly funded agency resulted in similar outcomes to those achieved in randomized controlled trials. However, an alternative strategy of improving outcomes is to use continuous outcome monitoring, which involves repeated (weekly) measurement of client outcomes with reliable scales, and feedback to therapists on the client’s status relative to previous sessions and relative to other similar clients. Research has demonstrated that this strategy improves client outcomes and reduces the number of clients who deteriorate. In this study, Reese and colleagues examined the outcomes of a large public behavioural health service in the U.S. that treats low-income individuals. The service implemented repeated outcome monitoring of clients with feedback to therapists. Over 5,000 clients mainly with depression, mood, and anxiety disorders were treated by 84 therapists who were licensed at the masters degree or higher. The clients completed the Outcome Rating Scale (a measure of symptom outcome) prior to each session, and the Session Rating Scale (a measure of the therapeutic alliance) after each session. Therapists received two days of training on how to use these measures and on the continuous feedback they were provided in order to improve their treatment of clients and their outcomes. Outcomes from this public behavioural health service were compared to previous large studies in publicly funded settings that implemented specific evidence-based treatments. The findings were similar, with about 42% showing reliable pre- to post-treatment improvement. The results of implementing continuous outcome monitoring with feedback for depressive symptoms were also large and positive (d = 1.34). These effects were similar to benchmarks established in randomized controlled trials of specific psychotherapies.

Practice Implications

Continuous outcome feedback enables therapists to identify clients who are not benefiting
from a given treatment, so that clinicians may collaboratively design different interventions or change their interpersonal stances. The inclusion of outcome monitoring and feedback in this publicly funded psychotherapy system, resulted in outcomes that were: better than what is often seen in such public service settings, equivalent to those public systems that implemented specific evidence based treatments, and similar to those reported in highly controlled randomized trials. The authors concluded that adding routine outcome monitoring and feedback is a viable alternative to transporting specific evidence based treatments to publicly funded psychotherapy programs. The measures used in this study are available free for individuals to use at: betteroutcomesnow.com.

Author email: jeff.reese@uky.edu

Munder, T., Fluckiger, C., Leichsenring, F, Abbass, A.A., Hilsenroth, M.J., … Wampold, B.E. (2018). Is psychotherapy effective? A re-analysis of treatments for depression. Epidemiology and Psychiatric Sciences, 1-7.

Based on a deeply flawed review in 1952, Hans Eysenck declared that psychotherapy was no more effective than custodial care for treating mental disorders. Later, he qualified this by stating that behaviour therapy was effective and other forms of psychotherapy were not. These statements touched off decades of angst and debate in the psychotherapy community, and also resulted in a great deal of research about psychotherapy’s effectiveness. By the 1970s the new research technique of meta-analysis was developed and was applied to psychotherapy research. In their seminal meta analysis of controlled studies, Smith and Glass found that psychotherapy was useful and with large effects compared to no treatment. And yet the debate continues. In 2018, Cuijpers argued that waitlist control groups (i.e., a common control condition in psychotherapy studies in which patients receive no treatment) are an inappropriate comparison leading to exaggerated estimates of the effects of psychotherapy. Recently, Munder and colleagues argued that waitlist controls are a way of estimating the natural course of the disorder (what would happen with no treatment) plus the effect of expecting to receive treatment (client expectations of receiving treatment tend to have a positive impact on symptoms). In fact, research shows that pre- to post-study effect sizes for the waiting period is approximately g = .40, or a medium effect. In other words, waiting for therapy in a study results in a moderate proportion of individuals getting better on their own without treatment. Therefore, Munder and colleagues argued that comparing psychotherapy to a waitlist control is appropriate and may be a conservative estimate of psychotherapy’s effects (i.e., psychotherapy has to outperform the effects of clients expecting treatment to help them). In their meta analysis, Munder and colleagues re-analysed 71 studies of psychotherapy for depression compared to a waitlist control condition. They found that the effect size in favour of psychotherapy was g = 0.75 (SE = 0.09) indicating a moderate to large effect. Psychotherapy was also more effective than care as usual (i.e., compared to another intervention that was not psychotherapy), g = 0.31 (SE = 0.11). There were no differences between types of psychotherapy (CBT, IPT, PDT, etc.) for depression outcomes.

Practice Implications

Despite various attempts during the history of psychotherapy to downplay or disparage its efficacy, research continues to show that psychotherapy is in fact effective. The average effect size compared to the natural history of depression is moderate to large (and that is likely an under-estimate). Again, there is no evidence that one type of psychotherapy is superior to another for treating depression. It is time for the field to move beyond questions of efficacy of psychotherapy and of the relative efficacy of different treatments, and look to understanding therapist interpersonal stances, client characteristics, and relationship factors that may improve outcomes from psychotherapy.