Blog

Reese, R. J., Duncan, B. L., Bohanske, R. T., Owen, J. J., & Minami, T. (2014). Benchmarking outcomes in a public behavioral health setting: Feedback as a quality improvement strategy. Journal of Consulting and Clinical Psychology, 82(4), 731-742.

Psychotherapy has demonstrated its efficacy in randomized controlled trials. But do these findings in highly controlled studies translate to everyday practice in publicly funded agencies that treat low income clients? Previous research in the US showed that outcomes of treatment-as-usual in public behavioural health agencies are generally not positive, so that only 20 to 35% of clients reliably improved. One approach to improving outcomes is to transport specific evidence-based treatments into practice settings. For example, research on applying CBT for panic and depression in a publicly funded agency resulted in similar outcomes to those achieved in randomized controlled trials. However, an alternative strategy of improving outcomes is to use continuous outcome monitoring, which involves repeated (weekly) measurement of client outcomes with reliable scales, and feedback to therapists on the client’s status relative to previous sessions and relative to other similar clients. Research has demonstrated that this strategy improves client outcomes and reduces the number of clients who deteriorate. In this study, Reese and colleagues examined the outcomes of a large public behavioural health service in the U.S. that treats low-income individuals. The service implemented repeated outcome monitoring of clients with feedback to therapists. Over 5,000 clients mainly with depression, mood, and anxiety disorders were treated by 84 therapists who were licensed at the masters degree or higher. The clients completed the Outcome Rating Scale (a measure of symptom outcome) prior to each session, and the Session Rating Scale (a measure of the therapeutic alliance) after each session. Therapists received two days of training on how to use these measures and on the continuous feedback they were provided in order to improve their treatment of clients and their outcomes. Outcomes from this public behavioural health service were compared to previous large studies in publicly funded settings that implemented specific evidence-based treatments. The findings were similar, with about 42% showing reliable pre- to post-treatment improvement. The results of implementing continuous outcome monitoring with feedback for depressive symptoms were also large and positive (d = 1.34). These effects were similar to benchmarks established in randomized controlled trials of specific psychotherapies.

Practice Implications

Continuous outcome feedback enables therapists to identify clients who are not benefiting
from a given treatment, so that clinicians may collaboratively design different interventions or change their interpersonal stances. The inclusion of outcome monitoring and feedback in this publicly funded psychotherapy system, resulted in outcomes that were: better than what is often seen in such public service settings, equivalent to those public systems that implemented specific evidence based treatments, and similar to those reported in highly controlled randomized trials. The authors concluded that adding routine outcome monitoring and feedback is a viable alternative to transporting specific evidence based treatments to publicly funded psychotherapy programs. The measures used in this study are available free for individuals to use at: betteroutcomesnow.com.

Author email: jeff.reese@uky.edu

Munder, T., Fluckiger, C., Leichsenring, F, Abbass, A.A., Hilsenroth, M.J., … Wampold, B.E. (2018). Is psychotherapy effective? A re-analysis of treatments for depression. Epidemiology and Psychiatric Sciences, 1-7.

Based on a deeply flawed review in 1952, Hans Eysenck declared that psychotherapy was no more effective than custodial care for treating mental disorders. Later, he qualified this by stating that behaviour therapy was effective and other forms of psychotherapy were not. These statements touched off decades of angst and debate in the psychotherapy community, and also resulted in a great deal of research about psychotherapy’s effectiveness. By the 1970s the new research technique of meta-analysis was developed and was applied to psychotherapy research. In their seminal meta analysis of controlled studies, Smith and Glass found that psychotherapy was useful and with large effects compared to no treatment. And yet the debate continues. In 2018, Cuijpers argued that waitlist control groups (i.e., a common control condition in psychotherapy studies in which patients receive no treatment) are an inappropriate comparison leading to exaggerated estimates of the effects of psychotherapy. Recently, Munder and colleagues argued that waitlist controls are a way of estimating the natural course of the disorder (what would happen with no treatment) plus the effect of expecting to receive treatment (client expectations of receiving treatment tend to have a positive impact on symptoms). In fact, research shows that pre- to post-study effect sizes for the waiting period is approximately g = .40, or a medium effect. In other words, waiting for therapy in a study results in a moderate proportion of individuals getting better on their own without treatment. Therefore, Munder and colleagues argued that comparing psychotherapy to a waitlist control is appropriate and may be a conservative estimate of psychotherapy’s effects (i.e., psychotherapy has to outperform the effects of clients expecting treatment to help them). In their meta analysis, Munder and colleagues re-analysed 71 studies of psychotherapy for depression compared to a waitlist control condition. They found that the effect size in favour of psychotherapy was g = 0.75 (SE = 0.09) indicating a moderate to large effect. Psychotherapy was also more effective than care as usual (i.e., compared to another intervention that was not psychotherapy), g = 0.31 (SE = 0.11). There were no differences between types of psychotherapy (CBT, IPT, PDT, etc.) for depression outcomes.

Practice Implications

Despite various attempts during the history of psychotherapy to downplay or disparage its efficacy, research continues to show that psychotherapy is in fact effective. The average effect size compared to the natural history of depression is moderate to large (and that is likely an under-estimate). Again, there is no evidence that one type of psychotherapy is superior to another for treating depression. It is time for the field to move beyond questions of efficacy of psychotherapy and of the relative efficacy of different treatments, and look to understanding therapist interpersonal stances, client characteristics, and relationship factors that may improve outcomes from psychotherapy.

Pybis, J., Saxon, D., Hill, A., & Barkham, M. (2017). The comparative effectiveness and efficiency of cognitive behaviour therapy and generic counselling in the treatment of depression: Evidence from the 2nd UK National Audit of psychological therapies. BMC Psychiatry, 17, 215. DOI 10.1186/s12888-017-1370-7

Over a decade ago the United Kingdom (UK) invested large sums of public dollars to fund the Increasing Access to Psychotherapy (IAPT) program. In IAPT, most patients receive cognitive behavioral therapy (CBT) as first-line treatment for depression or anxiety, and may receive generic counseling as second line treatment. One of the admirable aspects of IAPT is that the program consistently assesses outcomes, makes its data available for analyses, and publishes yearly reports on their outcomes. In this very large study, Pybis and colleagues assess whether CBT and generic counseling have different outcomes for patients with depression or anxiety. Over 33,000 patients who received treatment at one of 103 sites were in the study. Most patients (about 23,000) receiving CBT, and the others (about 10,000) receiving generic counseling. Two-thirds of the patients were female, most (84%) were white British, and the mean age was 41 (SD = 13.86). CBT focused on changing negative thoughts and behaviors in order to improve depressive symptoms. Generic counselling was harder to define, though the authors described these therapists as practicing in an integrative manner by bringing skills from training in different forms of psychotherapy. Generic counseling therapists did not focus on giving advice or opinions, but rather on helping clients understand themselves better. Pre- to post-treatment effect sizes for CBT (0.94 [0.92, 0.95]) and generic counseling (0.95 [0.92, 0.98]) were equivalent for depression outcomes. In CBT 50.4% of patients reliably improved, whereas 49.6% reliably improved if they received generic counseling. The average number of sessions attended by patients in the two treatments (CBT = 8.9 [6.34]; counseling = 7.5 [5.54]) were also equivalent. However, there were significant site effects. That is, a moderate and significant amount of patient outcomes (15%) could be accounted for by the site at which they received treatment (i.e., some sites or clinics had better outcomes than others).

Practice Implications

Generic counseling as provided in the IAPT in the UK was as effective as structured CBT for reducing symptoms of depression. However, almost half of patients did not improve in either treatment. Generic counseling was likely a label used to describe integrative psychotherapy that followed principles from a variety of psychotherapies that were based on psychological principles. There were much larger site/clinic effects than treatment modality effects, so that clients in some clinics had better than clients who received treatment in other clinics. This is consistent with research on therapist effects that show that some therapists are more effective than others, regardless of their orientation. This research suggests that training therapists to be more effective by improving their facilitative interpersonal skills may yield better outcomes for clients.

Davis, D. E., DeBlaere, C., Owen, J., Hook, J. N., Rivera, D. P., Choe, E., . . . Placeres, V. (2018). The multicultural orientation framework: A narrative review. Psychotherapy, 55(1), 89-100.

Many therapists have better outcomes with White or European clients than clients from diverse racial or ethnic minorities, and this might be due to racial and ethnic microaggressions that sometimes occur in therapy. Microaggression refer to intentional or unintentional brief commonplace verbal, behavioural, or environmental indignities that are experienced as derogatory or negative by racial and ethnic minority clients. A multicultural orientation refers to how the cultural worldviews, values, and beliefs of clients and therapists interact to co-create a relational experience in therapy. Therapist multicultural orientation has three elements. First, cultural humility, in which a therapist is able to maintain an interpersonal stance that is open to the client’s experience of cultural identity. Second, cultural opportunity, in which the therapist uses events in therapy to explore a client’s cultural identity in depth. Third, cultural comfort in which a therapist feels at ease, open, and calm with diverse clients. These elements are important in order to negotiate a therapeutic alliance (i.e. agreement on tasks and goals of therapy, and the emotional bond between client and therapist). In this narrative review, Davis and colleagues look at the small existing research on multicultural orientation and how that research can inform therapists’ practices. The authors found that in the two studies on the topic, greater therapist cultural humility was associated with better client outcomes. Several studies found that cultural humility was associated with a positive therapeutic alliance, and that therapist cultural humility was associated with fewer microaggressions as experienced by racial and ethnic minority clients. Finally, missed opportunities by therapists to explore the meaning of culture and identity were associated with negative client outcomes. Presumably, such missed opportunities meant that therapists did not recognize and repair cultural ruptures.

Practice Implications

The research on multicultural orientation suggests several practice implications. (1) Cultural humility requires therapists to explore their automatic cultural assumptions because if they remain unexplored they may be harmful to clients. (2) Therapists should overtly discuss the importance of cultural identities with clients in order to help both therapist and client develop a more complex understanding of the issues that bring the client to therapy. (3) A strong therapeutic alliance may require the therapist to incorporate their client’s cultural worldview and perspective when conceptualizing the client’s problems. (4) Depending on the client’s cultural worldview, therapists may consult with the client’s family and/or spiritual leaders when negotiating a culturally acceptable way of addressing the client’s problems. (5) Therapists need to identify for themselves when their values conflict with those of the client, and seek consultation or supervision when they do.

Cameron, S. K., Rodgers, J., & Dagnan, D. (2018). The relationship between the therapeutic alliance and clinical outcomes in cognitive behaviour therapy for adults with depression: A meta‐analytic review. Clinical Psychology and Psychotherapy, DOI: 10.1002/cpp.2180.

The therapeutic alliance refers to the collaborative agreement between therapist and client on the tasks of therapy (homework, treatment approach, intervention style) and goals of therapy (to reduce depressive symptoms, to improve interpersonal relationships, to cope better with stress), plus the emotional bond between therapist and client. The alliance is part of a larger concept of therapeutic relationship that also includes the real relationship between client and therapist and the transference relationship (maladaptive relational patterns in the client based on a history of relationships with parental figures). The alliance is thought to be a common factor across different therapeutic orientations, including cognitive behavioral therapy (CBT), time-limited psychodynamic psychotherapy (TLPP), and interpersonal psychotherapy (IPT). In fact, the alliance is known to have a moderate and robust relationship to client outcomes regardless of who rates the alliance (therapist, client, observer), which measure is used, and when in therapy the alliance is rated (early, middle, late). Although Beck emphasized the alliance as a key therapeutic principle in CBT, some CBT writers argue that the alliance is not so important. In this study, Cameron and colleagues conducted a meta analysis of 13 randomized controlled trials that assessed the relationship between therapeutic alliance and CBT outcomes for depression in adult clients. The overall mean correlation between therapeutic alliance and outcome was r = 0.26 (95% CI [.19–.32]), which indicates a moderate and significant relationship. This is very close to the value found in a larger meta analysis of over 200 alliance – outcome studies.

Practice Implications

The study demonstrates the importance of the therapeutic alliance to client outcomes in CBT. The association was at similar levels to those found in other types of therapy. Therapists conducting CBT should attend to building and maintaining an alliance, which provides a context to facilitate CBT interventions. If a client is not completing homework for example, it is likely that there is a lack of agreement on tasks of therapy, and this part of the alliance may need to be renegotiated. Therapists may also benefit from routinely assessing the alliance in therapy with their clients on a session by session basis using short and easy to use measures. Reviewing these scales regularly can alert a therapist to potential problems in the alliance and the need to repair any tensions or ruptures.

Cologan, J., Schweiter, R.D., & Nolte, T. (2017). Therapist reflective functioning, therapist attachment style, and therapist effectiveness. Administration Policy and Mental Health, DOI: 10.1007/s10488-017-0790-5.

Differences between therapists account for about 8% of patient outcomes, which is a moderate effect and therefore an important factor. Constructs such as therapist personality characteristics and facilitative interpersonal skills may play a key role in how effective therapists can be with their clients. An important therapist quality might be reflective functioning, or mentalization. Reflective functioning refers to the ability to conceptualize, identify, and understand mental states in oneself and in others, and how mental states affect behaviour and functioning.  For example, reflective functioning is the basis for predicting others’ behaviors, understanding social nuances and others’ intentions, and also one’s own behaviors and internal experiences. Fundamentally for a therapist, reflective functioning is necessary for empathy, which is a key therapeutic quality. Another key issue for therapists might be their own attachment security, or their characteristic ways of relating to others in interpersonal relationships. Securely attached therapists (those who have a positive view of self and others in relationships) may be able to develop a better therapeutic alliance with clients. Insecurely attached therapists (those who are avoidant in relationships or who are preoccupied in relationships), may struggle to a greater extent with developing and maintaining an alliance. In this study, Cologan and colleagues assessed reflective functioning and attachment security in 25 therapists from different theoretical orientations who treated 1001 adult clients who mostly had problems with depression or anxiety. Client outcomes were measured pre and post treatment. On average clients experienced a reduction in their symptoms after psychotherapy. Clients of therapists with higher levels of reflective functioning experienced better outcomes. Therapist attachment insecurity did not have a direct effect on client outcomes.

Practice Implications

As with other studies, therapists in this study varied in their outcomes, so that some had better outcomes than others. Level of therapist reflective functioning (ability to mentalize) accounted for a large proportion of this difference. Therapists who had greater skills with understanding their own and clients’ behaviors in terms of mental states (intentions, motivations, psychological and emotional needs, internal conflicts) likely were better able to empathize and develop an alliance with their clients. These are skills that therapists can learn with practice, consultation, personal therapy, and training.