Blog

Reese, R. J., Duncan, B. L., Bohanske, R. T., Owen, J. J., & Minami, T. (2014). Benchmarking outcomes in a public behavioral health setting: Feedback as a quality improvement strategy. Journal of Consulting and Clinical Psychology, 82(4), 731-742.

Psychotherapy has demonstrated its efficacy in randomized controlled trials. But do these findings in highly controlled studies translate to everyday practice in publicly funded agencies that treat low income clients? Previous research in the US showed that outcomes of treatment-as-usual in public behavioural health agencies are generally not positive, so that only 20 to 35% of clients reliably improved. One approach to improving outcomes is to transport specific evidence-based treatments into practice settings. For example, research on applying CBT for panic and depression in a publicly funded agency resulted in similar outcomes to those achieved in randomized controlled trials. However, an alternative strategy of improving outcomes is to use continuous outcome monitoring, which involves repeated (weekly) measurement of client outcomes with reliable scales, and feedback to therapists on the client’s status relative to previous sessions and relative to other similar clients. Research has demonstrated that this strategy improves client outcomes and reduces the number of clients who deteriorate. In this study, Reese and colleagues examined the outcomes of a large public behavioural health service in the U.S. that treats low-income individuals. The service implemented repeated outcome monitoring of clients with feedback to therapists. Over 5,000 clients mainly with depression, mood, and anxiety disorders were treated by 84 therapists who were licensed at the masters degree or higher. The clients completed the Outcome Rating Scale (a measure of symptom outcome) prior to each session, and the Session Rating Scale (a measure of the therapeutic alliance) after each session. Therapists received two days of training on how to use these measures and on the continuous feedback they were provided in order to improve their treatment of clients and their outcomes. Outcomes from this public behavioural health service were compared to previous large studies in publicly funded settings that implemented specific evidence-based treatments. The findings were similar, with about 42% showing reliable pre- to post-treatment improvement. The results of implementing continuous outcome monitoring with feedback for depressive symptoms were also large and positive (d = 1.34). These effects were similar to benchmarks established in randomized controlled trials of specific psychotherapies.

Practice Implications

Continuous outcome feedback enables therapists to identify clients who are not benefiting
from a given treatment, so that clinicians may collaboratively design different interventions or change their interpersonal stances. The inclusion of outcome monitoring and feedback in this publicly funded psychotherapy system, resulted in outcomes that were: better than what is often seen in such public service settings, equivalent to those public systems that implemented specific evidence based treatments, and similar to those reported in highly controlled randomized trials. The authors concluded that adding routine outcome monitoring and feedback is a viable alternative to transporting specific evidence based treatments to publicly funded psychotherapy programs. The measures used in this study are available free for individuals to use at: betteroutcomesnow.com.

Author email: jeff.reese@uky.edu

Boswell, J. F., Constantino, M. J., Oswald, J. M., Bugatti, M., Goodwin, B., & Yucel, R. (2018). Mental health care consumers’ relative valuing of clinician performance information. Journal of Consulting and Clinical Psychology, 86(4), 301-308.

Research has shown that some therapists are more effective than others both in terms of their overall effectiveness and in terms of their effectiveness with specific patient problems. Further, despite advances in medicine on this topic, there is little or no information provided to patients about a therapist’s track record on overall effectiveness. In any case, little is known about what patients value in psychotherapists and how much they are willing to give up in order to get what they value. For example, do patients prefer therapists who are highly effective for most problems, and would they be willing to tolerate a poorer therapeutic relationship in order to work with such a highly effective therapist? In this study, Boswell and colleagues employed a relative valuing procedure often used in economics to assess the relative value to patients of different therapist characteristics and performance. Patients were asked how much they were willing to give up on one therapist characteristic (therapist’s overall effectiveness with clients [i.e., overall track record]) in order to receive more of some other characteristic (therapist specific effectiveness in a problem domain, a better therapeutic alliance, lower cost of therapy). The study included 403 patients treated in mental health clinics in the U.S. Patient characteristics were typical of those seen in such clinics – predominantly they had problems with depression or anxiety, were 41 years old on average, mostly women (68.5%), and receiving individual psychotherapy (89.3%). In general, patients highly valued a therapist with a track record of general overall effectiveness. However, patients were willing to give up more of their therapists overall effectiveness if the therapist had a track record of successfully treating their specific problem (e.g., therapist A has lower general efficacy but has demonstrated greater specific efficacy for depression). Patients were also willing to sacrifice therapist general effectiveness in order to pay less for therapy (vs paying a higher fee for a more effective therapist), and in order to work with a provider with whom they would have a better therapeutic alliance (vs a lower alliance with a more generally effective therapist). Surprisingly, patients placed a lower value on factors like therapist gender and race. Younger patients put greater value on therapist performance data (i.e., their track record data), suggesting a generational effect in which younger clients tend to prefer to make decisions based on available data.

Practice Implications

Patients were willing to give up some therapist general effectiveness in order to work with someone who has a track record of being effective for their specific problem, who costs less, and with whom they could have a better therapeutic alliance. Fortunately, therapist general efficacy and domain specific efficacy tend to be highly correlated, and so patients may not have to choose between these. The findings also suggest that patients may be willing to see a therapist who is less generally effective if it meant they could have a good relational experience with the therapist. Research indicates that therapists are able to improve their outcomes and therapeutic alliances with additional training and deliberate practice.

Schermuly-Haupt, M. L., Linden, M., & Rush, A. J. (2018). Unwanted events and side effects in cognitive behavior therapy. Cognitive Therapy and Research, 42(3), 219-229.

Unwanted events are negative consequences for clients that may or may not be related to treatment (i.e., events outside of therapy or inside of therapy that may negatively affect clients).  These might include: occupational problems, stigmatization, strains in personal relationships, changes in the social network, patients feeling overwhelmed, undermined self-efficacy, deterioration of symptoms, emergence of new symptoms, suicidality, and others. Side effects refer to negative reactions in clients directly related to appropriately delivered therapy. Research estimates that between 5% and 20% of patients report side effects of psychotherapy. One could argue that side effects may be inevitable even in well-delivered therapy, and therapists who are aware of the potential for side effects may be better equipped to help clients to manage. In this study, Schermuly-Haupt, interviewed 100 psychotherapists who provided CBT in outpatient clinics in Germany about side effects among their clients. All therapists were supervised as part of their work and so the authors assumed the therapy was appropriately delivered. Therapists had on average 5 years of experience and were trained to provide CBT. The interview asked therapists about their most recent treatment case in which the client attended at least 10 sessions. Clients typically had major depression, an anxiety disorder, or a personality disorder, and had attended 28 sessions of therapy on average. During the interview, therapists identified if an unwanted event occurred for a client from a standardized list, and then rated the duration and severity of the effects. They also rated the degree to which the unwanted event was directly related to therapy (i.e., a side effect). Prior to the interview, only 26% of therapists reported their client experienced side effects. However, the interview process found that almost all clients experienced an unwanted event (98%) that may or may have been related to therapy, and 43% experienced at least one side effect that was at least somewhat related to treatment. The most frequent side effects were: “negative wellbeing/distress” (27% of clients), “deterioration of symptoms” (9% of clients) and “strains in family relations” (6% of clients). Of the therapists, 46% rated the side effects as at least moderately severe, and 8.8% of side effects were rated as persistent (lasting more than a month).

Practice Implications

Unwanted events outside of therapy are very common among our clients, but so are side effects from appropriately delivered treatment. Psychotherapy is not always harmless, and it may be best to acknowledge and prepare both clients and therapists for side effects. These may represent ruptures in the alliance that can be managed through alliance-focused therapy, for example. That is, side effects may be caused a mismatch between the goals of a therapist and client, or a disagreement on how to proceed in therapy given what a client needs at the time. Goals and tasks of therapy may need to be renegotiated following the experience of a side effect.

Andrzej Werbart & Mo Wang (2012). Predictors of not starting and dropping out from psychotherapy in Swedish public service settings, Nordic Psychology, 64, 128-146.

There are few empirical studies looking at patients who are offered but who do not take up psychotherapy. This is a particularly important issue in publicly funded psychotherapy programs in which large numbers of patients who need mental health services to not access the service or leave before receiving adequate treatment. Evidence from the Improving Access to Psychotherapy (IAPT) program in the United Kingdom suggests that about half of patients who are offered psychotherapy either do not take it up or drop out prematurely and unilaterally. Knowledge about what determines treatment rejection or dropping out is critical in designing and developing publicly funded psychotherapy so that not only access but also patient outcomes are improved. In this study from the national Swedish psychotherapy program that is publicly funded, Werbart and colleagues looked at data from 13 clinics in which 189 therapists treated almost 1400 patients. Therapists were experienced (median experience = 5 years), and most received advanced psychotherapy training. Patients had a wide array of problems and severity. Of the patients, 13.6% never started therapy even though they were referred and assessed for treatment, and of those who started 17.4% dropped out of treatment. So a total of 31% never received adequate treatment and did not benefit for psychotherapy. Patients who never started therapy tended to be younger, unemployed, and with higher levels of mental illness. Patients who remained in therapy once they started tended to be older, had more problems with trauma or loss, and had more severe illness although they were not a danger to themselves or others. Never starting treatment and dropping out were both associated with clinics that had greater institutional instability. Clinic instability was defined as a clinic with: unclear treatment goals and guidelines, not well adapted to providing psychotherapy, unclear policies around who and how therapy is conducted, less cooperation among professionals, and financial problems.

Practice Implications

Jurisdictions around the world, including in Canada, are looking to offer publicly funded psychotherapy, yet there is little research to guide how to improve uptake and retention of patients within the system. Such systems might focus pre-therapy efforts to retain patients who are younger and with greater mental health problems. In particular, public systems need to pay attention to clinic and institutional stability. How patients experience the clinic environment (as welcoming and integrated), how treating professionals cooperate, the clarity and structure of treatment guidelines and goals, and the financial stability of a clinic all appear to have an impact on whether patients actually access and complete a course of psychotherapy.

Pybis, J., Saxon, D., Hill, A., & Barkham, M. (2017). The comparative effectiveness and efficiency of cognitive behaviour therapy and generic counselling in the treatment of depression: Evidence from the 2nd UK National Audit of psychological therapies. BMC Psychiatry, 17, 215. DOI 10.1186/s12888-017-1370-7

Over a decade ago the United Kingdom (UK) invested large sums of public dollars to fund the Increasing Access to Psychotherapy (IAPT) program. In IAPT, most patients receive cognitive behavioral therapy (CBT) as first-line treatment for depression or anxiety, and may receive generic counseling as second line treatment. One of the admirable aspects of IAPT is that the program consistently assesses outcomes, makes its data available for analyses, and publishes yearly reports on their outcomes. In this very large study, Pybis and colleagues assess whether CBT and generic counseling have different outcomes for patients with depression or anxiety. Over 33,000 patients who received treatment at one of 103 sites were in the study. Most patients (about 23,000) receiving CBT, and the others (about 10,000) receiving generic counseling. Two-thirds of the patients were female, most (84%) were white British, and the mean age was 41 (SD = 13.86). CBT focused on changing negative thoughts and behaviors in order to improve depressive symptoms. Generic counselling was harder to define, though the authors described these therapists as practicing in an integrative manner by bringing skills from training in different forms of psychotherapy. Generic counseling therapists did not focus on giving advice or opinions, but rather on helping clients understand themselves better. Pre- to post-treatment effect sizes for CBT (0.94 [0.92, 0.95]) and generic counseling (0.95 [0.92, 0.98]) were equivalent for depression outcomes. In CBT 50.4% of patients reliably improved, whereas 49.6% reliably improved if they received generic counseling. The average number of sessions attended by patients in the two treatments (CBT = 8.9 [6.34]; counseling = 7.5 [5.54]) were also equivalent. However, there were significant site effects. That is, a moderate and significant amount of patient outcomes (15%) could be accounted for by the site at which they received treatment (i.e., some sites or clinics had better outcomes than others).

Practice Implications

Generic counseling as provided in the IAPT in the UK was as effective as structured CBT for reducing symptoms of depression. However, almost half of patients did not improve in either treatment. Generic counseling was likely a label used to describe integrative psychotherapy that followed principles from a variety of psychotherapies that were based on psychological principles. There were much larger site/clinic effects than treatment modality effects, so that clients in some clinics had better than clients who received treatment in other clinics. This is consistent with research on therapist effects that show that some therapists are more effective than others, regardless of their orientation. This research suggests that training therapists to be more effective by improving their facilitative interpersonal skills may yield better outcomes for clients.

Solomonov, N. & Barber, J.P. (2018). Patients’ perspectives on political self-disclosure, the therapeutic alliance, and the infiltration of politics into the therapy room in the Trump era. Journal of Clinical Psychology, DOI: 10.1002/jclp.22609.

Most studies of psychotherapy do not take into account the current political climate, and most therapists do not think about the impact of their politics on clients. Studies have focused on the effects of large historical-political events on therapy, but mainly in terms of client reactions to the events. Such studies typically assume that therapist and client shared or agreed on perspectives of the event. However, the 2016 U.S. presidential election was extremely polarizing and may represent one of those events in which clients and therapists do not agree. What if clients and therapists disagreed about the experience of the election and its aftermath – what might be the impact on their therapeutic alliance? To what extent are polarizing politics discussed in therapy, and how are these discussions experienced by clients? Solomonov and Barber conducted a national survey among 604 psychotherapy clients from the 50 U.S. states. The mean age of the sample was 33.82 years (SD = 11.10), 57% were women, 58% were Caucasian, 48% indicated that they voted for Hilary Clinton and 32% indicated that they voted for Donald Trump. Overall, 64% of patients indicated that they had spoken about politics with the therapist (66% of Trump supporters and 70% of Clinton supporters). Among Trump supporters, 38% of clients indicated that their therapist was a Republican, whereas 35% thought their therapist was a Democrat. Among Clinton supporters, only 14% said their therapist was a Republican and 64% perceived their therapist was a Democrat. Thirty percent of clients reported that their therapist explicitly disclosed their political views, and 38% of clients reported that even though their therapist did not explicitly disclose their political views the client could easily guess the therapist’s views. Clients who believed their therapist shared their political views reported significantly higher therapeutic alliance with the therapist than those who believed their therapist did not share their views. Clients who voted for Clinton reported significant increases in expression of negative feelings from before to after the election, whereas Trump supporters did not report a significant increase in negative feelings. Neither Trump nor Clinton supporters reported an increase in positive emotions pre and post election.

Practice Implications

About two thirds of clients in the U.S. have political discussions with their therapists, and almost half wanted to talk more about politics during sessions. Even though general self-disclosure among therapists is relatively infrequent, political self-disclosure among therapists about the 2016 U.S. election seemed to occur much more frequently. It is possible that political instability and the polarizing political climate in the U.S. may contribute to more self-disclosure of a political kind among therapists. This could have an impact on therapy. Clients who perceived their therapists to share political views reported a better therapeutic alliance than those who had divergent political views from their therapist. Similarities in values between therapist and client have long been known to be associated with the therapeutic alliance. The study demonstrates that in the current political climate in the U.S., client perceptions of shared or divergent values with therapists make their way into the therapeutic space.