Blog

Owen, J., Wampold, B. E., Kopta, M., Rousmaniere, T., & Miller, S. D. (2016). As good as it gets? Therapy outcomes of trainees over time. Journal of Counseling Psychology, 63, 12-19.

Does psychotherapy training improve trainees’ knowledge and skills? Do trainees improve in their ability to produce positive client outcomes over time? The research on training psychotherapists is mostly inconclusive. Some studies show little or no difference between trainees and experienced therapists, and others found no association between level of experience and client outcomes. On the other hand, some researchers have found a relationship between training and competence in delivering a particular type of treatment. Overall, the research seems to show that there is a lot of variability between therapists in their outcomes and on how training affects their practice and their clients’ outcomes. However, rarely do these studies assess outcomes within the same trainee over time as they accumulate more training. In this study, Owen and colleagues evaluate if psychotherapy trainees’ client outcomes improved with training over time. They assessed 114 psychology trainees at different levels of training in 47 clinics across the U.S. These training therapists saw over 1100 clients over at least a 12-month period, and many therapists were followed for three years. The average client improved, but with small effects (d = .31, CIs not reported). Therapists were more effective with clients who were more distressed (d = .66) than clients who were less distressed (d = .10), probably because more distressed clients had more room to improve. Trainees’ outcomes improved significantly over time, although their average improvement over time was small. Most importantly, trainees’ improvements over time varied so that the researchers were able to identify four patterns of change over a three year period of training: (1) one group of trainees started out with moderately good outcomes and their outcomes remained moderately good over time; (2) a second group started out with small positive effects in their client outcomes and they improved to achieve moderately good outcomes by their third year; (3) a third group of trainees started out with small positive client outcomes but their outcomes got worse by their third year; and (4) a fourth group started out with poor outcomes and improved to achieve small positive outcomes by year 3 of their training.

Practice Implications

Trainees appear to have various trajectories in their ability to foster positive client outcomes over time, and, at times, that trajectory is negative. Trainees whose outcomes get worse over time (group 3) or who do not achieve at least moderately good outcomes (group 4) may need specific training to foster better interpersonal effectiveness, empathy, management of countertransference, and humility. In general, therapists should assess their clients’ outcomes with progress monitoring tools in order to use the feedback to improve their outcomes over time. If outcomes are not positive on average, then therapists should consider remediation, further training, or consultation.

Saxon, D., Barkham, M., Foster, A., & Parry, G. (2016). The contribution of therapist effects to patient dropout and deterioration in the psychological therapies. Clinical Psychology and Psychotherapy, DOI: 10.1002/cpp.2028.

Sometimes patients experience negative outcomes in psychotherapy. For example, some patients drop out of therapy (i.e., they unilaterally decide to leave therapy before making any progress or before the endpoint planned with the therapist). In a previous meta-analysis of 669 studies, dropout rates ranged from 17% to 26% in psychotherapy trials. In this study, Saxon and colleagues were interested in the therapist effect on drop out. In other words, what is the impact of the individual therapist on negative outcomes like patients unilaterally terminating treatment? To examine the therapist effect one can look at differences between therapists in the average number of patients who drop out within their caseload. The authors looked at over 10,000 patients seen by 85 therapists from 14 sites in the United Kingdom initiative for Improving Access to Psychological Therapies. Therapists were selected if they saw more than 30 patients, and patients were included if they attended more than one session of therapy. Patient mean age was 40.3 (SD = 13.0), 71.2% were women, most were White (95%) and employed (76%). Of all the patients, 76.8% had some level of depression and 82.7% had some level of anxiety. Over 90% of the patients scored in the clinical range for symptom severity at pre-treatment. Patient symptom severity seen by a particular therapist was controlled in this study so that therapists who tended to treat severe cases were not penalized (i.e., case mix was controlled). Patients who dropped out represented 33.8% of the sample, with over half of these patients unilaterally terminating before the third session. The mean number of sessions for treatment completers was 6.1 (SD = 2.68). Therapist differences (i.e., the therapist effect) accounted for 12.6% (CI = 9.1, 17.4) of the patient drop out variance. In other words, about a quarter of therapists had a significantly greater number of drop outs compared to the average therapist. The mean dropout rate for the average therapist was 29.7% (SD = 6.4), the mean dropout rate for the above average therapist was 12.0% (SD = 7.3), whereas the mean dropout rate for the below average therapist was 49.0% (SD = 10.4).

Practice Implications

Who a patient gets as a therapist appears to have an important impact on whether the patient remains in therapy. Almost half of clients dropped out if they saw a poorly performing therapist (and nearly a quarter of therapists were poorly performing). By contrast, highly performing therapists only had a 12% drop out rate. Therapist variables that are known to be related to negative outcomes like dropping out include: lack of empathy, negative countertransference, and disagreements with patients about the therapy process. Previous research showed that therapeutic orientation is not related to negative outcomes. Therapists who are perform below average on when it comes to patient dropout might be able to use progress monitoring or some other means of measuring their patients’ outcomes to their advantage. These therapists may require more support, supervision, or training to improve their patients’ outcomes.

Goldberg, S.B., Babins-Wagner, R., Rousmaniere, T., Berzins, S., Hoyt, W.T., Whipple, J.L., Miller, S.D., & Wampold, B.E. (2016). Creating a climate for therapist improvement: A case study of an agency focused on outcomes and deliberate practice. Psychotherapy, 53, 367-375.

There is a lot of evidence that psychotherapy is effective – a result that has been demonstrated in randomized trials and in naturalistic setting. As I have noted numerous times in this Blog, psychotherapy is as effective as medications but without the side effects and with longer lasting results. However, there is room for improvement, especially in the effectiveness of individual therapists. Health care organizations are increasingly interested in quality improvement, which refers to efforts to make changes in practice that will lead to better patient outcomes, better care, and better professional development. One approach to quality improvement in medicine has been through audit and feedback – which involves measuring a clinician’s practice, comparing the clinician’s outcomes to professional standards, and giving the clinician feedback. In psychotherapy, the analogue is routine outcome monitoring in which patient progress is monitored with standardized measures throughout therapy, and therapists receive ongoing feedback on each patient’s progress relative to the average patient with that disorder. We know that therapists tend not to improve in terms of patient outcomes with experience alone, and some authors argue that one of the things that therapists are missing is good quality information about their clients’ progress. What would happen if an agency or organization decided to make it a priority to provide therapists with quality information about client progress? This paper by Goldberg and colleagues is a case study in which an agency deliberately created a culture of quality feedback and professional development to improve therapist expertise, therapist intentional practice, and client outcomes. The case study is of a community mental health agency in Alberta. Over 5,000 clients were seen by 153 therapists over a 7 year period (2008 to 2015) as part of the study. Clients received at least three sessions of therapy (mean = 6.53 sessions, SD = 5.02), and had a range of disorders typically seen in a mental health clinic. Therapists included 49.7% licensed or provisionally licensed professionals at the masters or doctoral level from different professions (e.g., social work, psychology, pastoral counselling), and 50.3% practicum students. Throughout the 7 years of the study, therapists saw an average of 33.52 clients (SD = 26.24). In 2008, the agency required the staff to collect outcome measures of all clients before each session (although patient scores were not tied to staff performance evaluations). This policy change caused a 40% turnover in clinical staff within 4 months (clearly a large minority of therapists did not want to participate in this new clinic directive)! These staff positions were replaced and staffing was stable after that point. In addition to requiring clinicians to provide measures on all patients (although patients could decline to participate), the agency provided monthly clinical consultations with an external consultant as a means of professional development. During these consultation, clinicians were encouraged to bring cases that were not progressing well in order to get feedback on their most challenging patients. Discussions were organized around therapeutic alliance, i.e., clarifying goals and preferences, and ways of facilitating engagement. The overall results showed a significant decline in distress among patients over the course of treatment. Of most interest was that therapists on average showed a significant improvement in their outcomes over time. That is, contrary to research showing that therapists do not improve over time when left to their own devices, therapists in this agency that received feedback and professional education around difficult cases did improve significantly.

Practice Implications

The findings of this study indicate that psychotherapists can improve over time if they receive quality information about client progress, and if they receive professional development that is tied to this information (i.e., concrete suggestions for ways of working with difficult clients). In other words, it is possible for therapist to develop expertise over time under some conditions. A significant challenge in this case study was that a number of therapists left the agency due to the quality improvement efforts. Some therapists are sensitive to or feel threatened by outcome monitoring. However, therapists who remained or who were subsequently hired by the agency showed a reliable increase in their expertise and client outcomes as a result of deliberate intentional practice, quality feedback about client progress, and concrete professional development focused on the therapeutic alliance.

Bar-Kalifa, E., Atzil-Slonim, D., Rafaeli, E., Peri, T., Rubel, J., & Lutz, W. (2016, October 24). Therapist–client agreement in assessments of clients’ functioning. Journal of Consulting and Clinical Psychology. Advance online publication. http://dx.doi.org/10.1037/ccp0000157.

There has been a lot of research in the past decade on progress monitoring (i.e., regularly providing reliable feedback to therapists on client outcomes, the alliance, and client functioning). This research indicates that client outcomes can be enhanced if therapists have ongoing information on how their client or the relationship is progressing. In this innovative research by Bar-Kalifa and colleagues, the authors studied 77 therapists who saw a total of 384 clients. The therapists were experienced at providing cognitive-behavioral therapy. Clients for the most part had a depressive or anxiety disorder and were seen for an average of 36 sessions. Client outcomes were measured pre- and post-treatment. Emotional and psychological functioning during the past week was rated by the client before each session, and the same measure was given to the therapist to rate their client at the end of each session. After therapists made their rating, they were given ongoing feedback (i.e., progress monitoring) about how their clients’ rated their own functioning during the past week. Did clients and therapists agree on level of client functioning, was this agreement stable over time, and was this agreement or disagreement related to client outcomes? The authors used sophisticated statistical modeling to separate the effects of client ratings of their functioning from therapists’ ratings, and to examine the impact of the changing relationship between therapist and client ratings over time on client outcomes. The authors found little difference in the level of client and therapist ratings of client functioning, and they found that therapists tended to be accurate (i.e., congruent with clients) in tracking client functioning over time. More importantly, the ability of therapists to accurately track client functioning from session to session was related to better client outcomes in terms of key symptoms of depression and anxiety.

Practice Implications

The ability of therapists to accurately track client functioning over time was related to better client outcomes. This means that therapists who were aware of their clients’ functioning through feedback methods were better equipped to help their clients. In particular, information about how client functioning was changing from session to session might have allowed therapists to take corrective action for clients who were not doing well from one session to another. This information might have allowed therapists to reconsider a treatment formulation for a particular client, for example. Therapists should be aware of how a client is doing at a particular session, but more importantly therapists should be sensitive to fluctuations in client functioning across sessions. This might be best achieved with ongoing progress monitoring.

Community and Mental Health Team, Health and Social Care Information Centre (2015). Psychological therapies; Annual report on the use of IAPT services: England 2014/15.

There have been calls from mental health professional organizations and by the media to provide publicly funded psychotherapy in Canada. Rates of common mental disorders in Canada are high, such that about 20% of the population will personally experience a mental illness in their lifetime. In 1998, the estimated direct and indirect economic cost of mental illness in Canada was $7.9 billion (all figures are in Canadian dollars). Current estimates of costs to fund a public psychotherapy service in Canada may be about $1 billion to $2.8 billion – which far outweighs the cost. Most outpatient psychotherapy in Canada is provided by professionals in private practice who charge somewhere between $100 and $200 per session, costing Canadians nearly $1 billion per year. Some people are fortunate to have workplace insurance that covers some but not all of the costs, but most people in Canada do not have insurance and so they pay out of pocket or they go untreated. Research shows us that approximately 13 to 18 sessions are needed for 50% of clients to get better with psychotherapy. Which means that even with an insurance plan, many Canadians who need psychotherapy will find it to be a financial burden. Since 2008, the National Health Service in England implemented the Improving Access to Psychotherapies (IAPT) services to provide publicly funded psychotherapy to the population. The psychological treatments provided through IAPT are evidence-based (e.g., CBT, interpersonal psychotherapy, brief dynamic psychotherapy for depression). For mild to moderate problems, individuals get low intensity interventions first (i.e., self help, internet based interventions), followed by more intensive psychotherapy if needed. Treatment outcomes are measured from pre- to post-treatment with valid standardized measures of depression and anxiety. At post-treatment, patients are categorized as reliably deteriorated, not changed, improved, and recovered. The goal of the IAPT is to achieve 50% recovery rates among patients. In their online 2014-15 annual report, the IAPT service reported that it treated over 400,000 patients in that year. 44.8% of patients were rated as reliably recovered – that is over 180,000 mentally ill patients improved and no longer had a mental illness. Reliable improvement was seen in 60.8% of patients – this included recovered patients plus those who still had a disorder but were feeling significantly better than when they started. Recovery was highest for people 65 years and older (57.8%). Rates of recovery were similar for depression (44.6%) and anxiety (47.8%) disorders, and between men and women. Waiting times for treatment was less than 28 days for 66.0% of patients.

Practice Implications

The experience in England with the IAPT is instructive for Canada. The IAPT service provides evidence-based psychological therapies within a publicly funded national health service. The IAPT approached its target of 50% of patients recovering from mental illness, and over 60% of patients were reliably improved. Waiting times were low for most patients. Given the experience in England’s National Health Service, the implementation of a national strategy for psychotherapy appears to be feasible and effective. Will political leaders in Canada be able to see the financial and human value of publicly funded psychotherapy?

Castonguay, L.G., Boswell, J.F., Zack, S., Baker, S., Boutselis, M., Chiswick, N., Damer, D., Hemmelstein, N., Jackson, J., Morford, M., Ragusea, S., Roper, G., Spayd, C., Weiszer, T., Borkovec, T.D., & Grosse Holtforth,, M. (2010). Helpful and hindering events in psychotherapy: A practice research network study. Psychotherapy: Theory, Research, Practice, and Training, 47, 327-344.

There are many reasons why I like this paper, and one reason is that it is a psychotherapy practice research network study (most of the co-authors are independent practice clinicians). This group of clinicians and researchers met on a number of occasions to define the research questions, including: “what do psychotherapists and clients find most and least helpful in a psychotherapy session?”; and “do psychotherapists and clients agree on what was most and least helpful?” The clinicians and researchers also discussed and agreed on the method for collecting and analysing the data. Thirteen independent practice clinicians participated (6 CBT, 4 psychodynamic, and 3 experiental/humanistic). For a period of 18 months, all new clients were invited to participate so that 121 clients with a variety of disorders enrolled in the study. Clients and therapists filled out (on an index card) parts of the Helpful Aspects of Therapy (HAT) measure, which asked them to report, describe, and rate particularly helpful and hindering events from the session they had just completed. For example clients and therapists were asked: “Did anything particularly helpful happen during this session?”; and “Did anything happen during this session which might have been hindering?” When participants answered “Yes” to either of these questions, they were asked to briefly describe the event(s), and then rate them on a scale from 1 to 4 for level of helpfulness or level of hindrance. Both clients and therapists did so at the end of every therapy session. Close to 1500 therapeutic events were recorded by the clients and therapists. The events were then coded and categorized according to type of event by independent raters using an established coding system. Clients rated self-awareness, problem clarification, and problem solution as the most helpful type of events, although self-awareness was significantly the most identified of all helpful events by clients. Therapists rated self-awareness, alliance strengthening, and problem clarification as the most helpful type of events. Therapists identified self-awareness and alliance strengthening significantly more often than any other helpful events. Hindering events were identified much less frequently by clients and therapists. Client identified poor fit (e.g., therapist tried something that didn’t fit the client’s experience) as the most frequent hindering event category. Therapists identified therapist omissions (i.e., failure to provide support or an intervention) as the most frequent hindering event category. Overall, with the exception of self-awareness, therapists and clients did not agree on what were the most helpful or hindering events in therapy.

Practice Implications

Results regarding self awareness indicate that providing clients with opportunities to achieve a clearer sense of their experience (e.g., emotions, behaviors, and perceptions of self) is frequently reported as beneficial by both clients and therapists. The events that therapists most frequently reported as detrimental were those in which they failed to be attuned to their clients’ needs. This may reflect therapists’ concerns with potential alliance ruptures. The overall lack of agreement between therapists and clients on helpful and hindering events raises the question about whether therapists are not aware enough of clients’ experiences, or whether clients are not knowledgeable about what is in fact therapeutic. Perhaps client and therapist ratings of events represent complementary perspectives on what works or does not work in psychotherapy. Regarding participating in research, these independent practice therapists reported that the procedure of writing down helpful and harmful events and reading what their clients wrote after each session had a positive impact on their practice. That is, the process of data collection became immediately relevant to their clinical work.

Author email: lgc3@psu.edu