Blog

Reese, R. J., Duncan, B. L., Bohanske, R. T., Owen, J. J., & Minami, T. (2014). Benchmarking outcomes in a public behavioral health setting: Feedback as a quality improvement strategy. Journal of Consulting and Clinical Psychology, 82(4), 731-742.

Psychotherapy has demonstrated its efficacy in randomized controlled trials. But do these findings in highly controlled studies translate to everyday practice in publicly funded agencies that treat low income clients? Previous research in the US showed that outcomes of treatment-as-usual in public behavioural health agencies are generally not positive, so that only 20 to 35% of clients reliably improved. One approach to improving outcomes is to transport specific evidence-based treatments into practice settings. For example, research on applying CBT for panic and depression in a publicly funded agency resulted in similar outcomes to those achieved in randomized controlled trials. However, an alternative strategy of improving outcomes is to use continuous outcome monitoring, which involves repeated (weekly) measurement of client outcomes with reliable scales, and feedback to therapists on the client’s status relative to previous sessions and relative to other similar clients. Research has demonstrated that this strategy improves client outcomes and reduces the number of clients who deteriorate. In this study, Reese and colleagues examined the outcomes of a large public behavioural health service in the U.S. that treats low-income individuals. The service implemented repeated outcome monitoring of clients with feedback to therapists. Over 5,000 clients mainly with depression, mood, and anxiety disorders were treated by 84 therapists who were licensed at the masters degree or higher. The clients completed the Outcome Rating Scale (a measure of symptom outcome) prior to each session, and the Session Rating Scale (a measure of the therapeutic alliance) after each session. Therapists received two days of training on how to use these measures and on the continuous feedback they were provided in order to improve their treatment of clients and their outcomes. Outcomes from this public behavioural health service were compared to previous large studies in publicly funded settings that implemented specific evidence-based treatments. The findings were similar, with about 42% showing reliable pre- to post-treatment improvement. The results of implementing continuous outcome monitoring with feedback for depressive symptoms were also large and positive (d = 1.34). These effects were similar to benchmarks established in randomized controlled trials of specific psychotherapies.

Practice Implications

Continuous outcome feedback enables therapists to identify clients who are not benefiting
from a given treatment, so that clinicians may collaboratively design different interventions or change their interpersonal stances. The inclusion of outcome monitoring and feedback in this publicly funded psychotherapy system, resulted in outcomes that were: better than what is often seen in such public service settings, equivalent to those public systems that implemented specific evidence based treatments, and similar to those reported in highly controlled randomized trials. The authors concluded that adding routine outcome monitoring and feedback is a viable alternative to transporting specific evidence based treatments to publicly funded psychotherapy programs. The measures used in this study are available free for individuals to use at: betteroutcomesnow.com.

Author email: jeff.reese@uky.edu

Pybis, J., Saxon, D., Hill, A., & Barkham, M. (2017). The comparative effectiveness and efficiency of cognitive behaviour therapy and generic counselling in the treatment of depression: Evidence from the 2nd UK National Audit of psychological therapies. BMC Psychiatry, 17, 215. DOI 10.1186/s12888-017-1370-7

Over a decade ago the United Kingdom (UK) invested large sums of public dollars to fund the Increasing Access to Psychotherapy (IAPT) program. In IAPT, most patients receive cognitive behavioral therapy (CBT) as first-line treatment for depression or anxiety, and may receive generic counseling as second line treatment. One of the admirable aspects of IAPT is that the program consistently assesses outcomes, makes its data available for analyses, and publishes yearly reports on their outcomes. In this very large study, Pybis and colleagues assess whether CBT and generic counseling have different outcomes for patients with depression or anxiety. Over 33,000 patients who received treatment at one of 103 sites were in the study. Most patients (about 23,000) receiving CBT, and the others (about 10,000) receiving generic counseling. Two-thirds of the patients were female, most (84%) were white British, and the mean age was 41 (SD = 13.86). CBT focused on changing negative thoughts and behaviors in order to improve depressive symptoms. Generic counselling was harder to define, though the authors described these therapists as practicing in an integrative manner by bringing skills from training in different forms of psychotherapy. Generic counseling therapists did not focus on giving advice or opinions, but rather on helping clients understand themselves better. Pre- to post-treatment effect sizes for CBT (0.94 [0.92, 0.95]) and generic counseling (0.95 [0.92, 0.98]) were equivalent for depression outcomes. In CBT 50.4% of patients reliably improved, whereas 49.6% reliably improved if they received generic counseling. The average number of sessions attended by patients in the two treatments (CBT = 8.9 [6.34]; counseling = 7.5 [5.54]) were also equivalent. However, there were significant site effects. That is, a moderate and significant amount of patient outcomes (15%) could be accounted for by the site at which they received treatment (i.e., some sites or clinics had better outcomes than others).

Practice Implications

Generic counseling as provided in the IAPT in the UK was as effective as structured CBT for reducing symptoms of depression. However, almost half of patients did not improve in either treatment. Generic counseling was likely a label used to describe integrative psychotherapy that followed principles from a variety of psychotherapies that were based on psychological principles. There were much larger site/clinic effects than treatment modality effects, so that clients in some clinics had better than clients who received treatment in other clinics. This is consistent with research on therapist effects that show that some therapists are more effective than others, regardless of their orientation. This research suggests that training therapists to be more effective by improving their facilitative interpersonal skills may yield better outcomes for clients.

Cologan, J., Schweiter, R.D., & Nolte, T. (2017). Therapist reflective functioning, therapist attachment style, and therapist effectiveness. Administration Policy and Mental Health, DOI: 10.1007/s10488-017-0790-5.

Differences between therapists account for about 8% of patient outcomes, which is a moderate effect and therefore an important factor. Constructs such as therapist personality characteristics and facilitative interpersonal skills may play a key role in how effective therapists can be with their clients. An important therapist quality might be reflective functioning, or mentalization. Reflective functioning refers to the ability to conceptualize, identify, and understand mental states in oneself and in others, and how mental states affect behaviour and functioning.  For example, reflective functioning is the basis for predicting others’ behaviors, understanding social nuances and others’ intentions, and also one’s own behaviors and internal experiences. Fundamentally for a therapist, reflective functioning is necessary for empathy, which is a key therapeutic quality. Another key issue for therapists might be their own attachment security, or their characteristic ways of relating to others in interpersonal relationships. Securely attached therapists (those who have a positive view of self and others in relationships) may be able to develop a better therapeutic alliance with clients. Insecurely attached therapists (those who are avoidant in relationships or who are preoccupied in relationships), may struggle to a greater extent with developing and maintaining an alliance. In this study, Cologan and colleagues assessed reflective functioning and attachment security in 25 therapists from different theoretical orientations who treated 1001 adult clients who mostly had problems with depression or anxiety. Client outcomes were measured pre and post treatment. On average clients experienced a reduction in their symptoms after psychotherapy. Clients of therapists with higher levels of reflective functioning experienced better outcomes. Therapist attachment insecurity did not have a direct effect on client outcomes.

Practice Implications

As with other studies, therapists in this study varied in their outcomes, so that some had better outcomes than others. Level of therapist reflective functioning (ability to mentalize) accounted for a large proportion of this difference. Therapists who had greater skills with understanding their own and clients’ behaviors in terms of mental states (intentions, motivations, psychological and emotional needs, internal conflicts) likely were better able to empathize and develop an alliance with their clients. These are skills that therapists can learn with practice, consultation, personal therapy, and training.

Eubanks, C. F., Burckell, L. A., & Goldfried, M. R. (2017, December 21). Clinical consensus strategies to repair ruptures in the therapeutic alliance. Journal of Psychotherapy Integration. Advance online publication.

Research is clear that the therapeutic alliance (i.e., agreement on tasks and goals of therapy, and the bond between client and therapist) is an important predictor of client outcomes across theoretical orientations. It is also clear that ruptures or strains in the alliance occur often and can have a negative effect on client outcomes. One can define two types of ruptures: (1) withdrawal ruptures, in which the client moves away from the therapist by shutting down, changing the focus, or not completing session assignments; and (2) confrontation ruptures, in which the client moves against the therapist so that the relationship quality is low, the client is not collaborative, and the client does not agree with the goals of therapy. Repairing alliance ruptures can have a positive effect on client outcomes, and therapists can learn to repair alliance ruptures. What are the best strategies that a therapist can use to repair alliance ruptures? In this study of expert consensus, Eubanks and colleagues surveyed clinicians in three broad and different surveys. In the first survey, the authors asked 330 professional social workers and psychologists from a variety of theoretical orientations to describe situations in which they encountered alliance ruptures in clinical practice. The researchers categorized situations described by clinicians as withdrawal ruptures or as confrontation ruptures, and then the authors selected those scenarios that best represented each type of rupture. In a second independent survey, 177 clinicians indicated how they would advise a colleague seeking consultation to respond to each scenario of a therapeutic alliance rupture. Clinicians generated between 35 and 45 strategies to repair each type of alliance rupture. In the final part of the survey, training directors in psychology and social work programs nominated peer experts to rate the strategies for alliance repair, so that 134 peer-nominated expert clinicians provided ratings. There was a high level of consensus among experts such that between 55% and 74% agreed on effective strategies to repair alliance ruptures. Experts agreed that during the session in which the alliance rupture occurred therapists should: explore and empathize with the client`s anger at the therapist, and validate or legitimize the client`s position on the issue related to the rupture. Experts also agreed that in future sessions clinicians can use other strategies like: helping the client manage and cope with painful feelings related to the rupture, helping the client clarify and explore their emotions related to the rupture, and exploring the meaning and patterns of problematic relationships outside of therapy.

Practice Implications

Experts agreed that the best strategies to repair therapeutic alliance ruptures were to deal with the therapeutic bond (e.g., explore and empathize with the client`s anger at the therapist) and to validate the client`s position on the issue related to the rupture. Other strategies like helping the client cope with their reactions and feelings, and exploring the meaning and patterns related to the client`s response were also rated as helpful. Less helpful strategies included therapists communicating about the limits of therapy, and therapist self-disclosure of their reaction to the rupture.

Anderson, T., Crowley, M. E. J., Himawan, L., Holmberg, J. K., & Uhlin, B. D. (2016). Therapist facilitative interpersonal skills and training status: A randomized clinical trial on alliance and outcome. Psychotherapy Research, 26(5), 511-529.

Research on therapist effects indicates that there are differences between therapists so that some therapists are more effective than others. Therapist effects account for about 9% of client outcomes, which represents a moderate and therefore important effect. Differences between therapists do not seem to be accounted for by differing levels of adherence to or competence in delivering a manualized treatment. However, some researchers argue that therapist effects can be accounted for by differing level of facilitative interpersonal skills. That is, therapists vary in the level of interpersonal skills, and this difference accounts for a significant proportion of client outcomes. Therapist facilitative interpersonal skills might include: empathy, positive regard, warmth, ability to establish and repair therapeutic alliances, verbal fluency, emotional expression, and the ability to enhance client expectations of improvement. In this unique analogue study, Anderson and colleagues selected 23 “therapists” who were rated as very high or as very low on facilitative interpersonal skills. For example, highly skilled “therapists” scored high on a self-report measure of social skills and also demonstrated high interpersonal skills in their responses to video vignettes of therapy. Therapists also differed on their training status: half of the “therapists” were advanced clinical psychology graduate students, and the other half were graduate students from other programs (social sciences, humanities) who had no clinical training at all. The 66 clients were volunteers from a large undergraduate student research pool who met diagnostic criteria for a mental disorder (anxiety or depression) and were moderately to highly distressed. Clients were randomly assigned to receive treatment or to a wait-list control condition, so that 46 clients (2 per therapist) received treatment and 22 received no treatment. Compared to those in the control condition, clients who received treatment on average improved in terms of level of distress, regardless of which “therapist” they were assigned to. The training status of “therapists” (those with clinical training versus those without clinical training) had no effect on client outcomes or on the therapeutic alliance. Compared to “therapists” with low facilitative interpersonal skills, those with high interpersonal skills (regardless of training status) had significantly better client outcomes and significantly higher levels of the alliance.

Practice Implications

This was an analogue study in which some “therapists” were non-clinicians, so one must take the results with a grain of salt. Nevertheless, clients started out distressed, had a diagnosable disorder, and on average they achieved significant reduction in distress if they received therapy. Whether “therapists” had any clinical training did not affect outcomes, that is, non-clinical “therapists” did just as well as clinical trainees. However, higher “therapist” facilitative interpersonal skills regardless of training status lead to better client outcomes. These findings provide support for the notion that a therapist who is: empathic, warm, able to establish and repair therapeutic alliances, verbally fluent, emotionally expressive, and able to enhance client expectations of improvement will be more effective in reducing their clients’ levels of distress.

Author email: andersot@ohio.edu

Erekson, D. M., Janis, R., Bailey, R. J., Cattani, K., & Pedersen, T. R. (2017). A longitudinal investigation of the impact of psychotherapist training: Does training improve client outcomes? Journal of Counseling Psychology, 64(5), 514-524.

The research on the effects of therapist training on client outcomes has not been very encouraging. Most studies indicate that more therapist training, better adherence to and competence in a treatment manual, and greater experience are not related to improved client outcomes. The profession would like to think that therapists affect client outcomes so that more training and experience might be related to better outcomes. One could argue that the research in this area is hampered by many studies not following the same therapists across time over stages of training. That is, many studies compare client outcomes between novices and licensed professionals – but these studies do not really address the question “does an individual therapist get better as he or she accumulates more years of training and experience?” In this unique study, Erekson and colleagues track client outcomes of 22 therapists over a 10 year period starting from the therapists’ early training in a doctoral program in psychology to their first years as licensed psychologists working in a counselling centre. On average, the psychotherapists saw 183.95 (SD = 103.23) student clients during that time (range: 62 to 449 clients). The clients primarily had clinically impairing problems with anxiety and depression. Stages of training were defined as: graduate trainee, intern, post-doctoral fellow, and licensed professional. The average client moderately improved (d = .72) in terms of symptoms from the start to the end of their therapy. When looking at therapists’ effects across stages of training, the authors controlled for client initial severity and size of therapist caseload. The results indicated that 4 of the 22 therapists improved in their client outcomes over stages of training, 10 remained the same, and 8 therapists worsened over time. On average, client outcomes remained the same across a therapist’s stage of training. However, average client rate of change (i.e., how quickly a therapist’s client improved) became slower as therapists achieved more training and experience.

Practice Implications

This study adds to the weight of evidence that therapist training and experience as currently conceptualized do not result in better outcomes among clients. One possible explanation for why psychotherapist trainees do as well or better than when they are licensed professionals may lie in the structure of training programs. Trainees in graduate school and internships typically receive a high level of supervision and learning experiences, and must deliberately report client progress on an ongoing basis. Therapists who are licensed professionals are not required to maintain these practices, and so they may not be practicing deliberately. Researchers and clinical writers identify deliberate practice as an important means by which practicing psychotherapists can maintain and improve their skills in interpersonal effectiveness and therapeutic alliance.

Author email: david_erekson@byu.edu