Blog

Tracey, T.J.G., Wampold, B.E., Lichtenberg, J.W., & Goodyear, R.K. (2014). Expertise in psychotherapy: An elusive goal? American Psychologist, 69, 218-229.

As I have reported many times in this blog, there is substantial evidence for the efficacy of psychotherapy. However, the quality of psychotherapy differs across therapists – that is, some therapists achieve better client outcomes than others. Tracey and colleagues (2014) ask: is it possible to demonstrate expertise in psychotherapy? They define expertise as “increased quality of performance that is gained with additional experience”. Professions that can demonstrate expertise include: astronomers, test pilots, chess masters, mathematicians, and accountants. But several professions may not demonstrate expertise, including: psychiatrists, college admissions officers, court judges, personnel selectors, and psychotherapists. The difference is that the former group has predictable outcomes and has access to quality feedback. In addition, Tracey and colleagues argue that psychotherapy lacks adequate models for how interventions produce benefits. As a result, adherence to treatment protocols (i.e., manuals) is not reliably associated with better patient outcomes. Further, more experienced therapists are not more effective than less experienced therapists. Experienced therapists might have more complete conceptualizations of client problems, but these conceptualizations may not be accurate. Finally, although therapists affect outcomes, client variables (e.g., motivation, severity of symptoms, expectations) likely explain the largest proportion of outcome variance. Tracey and colleagues argue that part of the problem is that psychotherapists do not engage in “deliberate practice”; that is, practice of a specific task (e.g., identifying a rupture in the alliance), receiving specific feedback (e.g., that a rupture was not identified), opportunity for repetition (e.g., to identify a subsequent rupture in the alliance), and opportunity for improvement afforded by error (e.g., better able to identify a future rupture and repairing that rupture). Generally the practice of psychotherapy provides little feedback about the accuracy of past clinical decisions. In other words there is a lack of quality information to help therapists develop into experts. Further, for a whole host of reasons, psychotherapists are notoriously poor at assessing client progress (i.e., like other humans, therapists engage in a number of biased evaluations of their performance). Quality information might be available from progress monitoring (i.e., continuous feedback to therapists about client outcomes), which has been shown to improve client outcomes. However, this may not aid therapists in developing expertise, since progress monitoring provides little information about what therapist behaviors are necessary to improve performance and client outcomes.

Practice Implications

Tracey and colleagues conclude that currently psychotherapy does not provide evidence that it is a profession with expertise. To achieve expertise, therapists need quality information not only about their patients’ outcomes but also about their own average outcomes (i.e. performance) relative to other therapists working with similar clients. And therapists need information on how to manage specific events in psychotherapy. Tracey and colleagues suggest therapists set aside time to generate hypotheses about one’s practice that can be disconfirmed, and then test these hypotheses. For example, if a therapist is experiencing a higher than average number of premature client terminations (which may follow a misunderstanding with the client), the therapist may hypothesize that he or she is not identifying key alliance ruptures. To test this hypothesis, the therapist could repeatedly assess the alliance (with a validated instrument) with some clients, use this information and not clinical judgement alone to identify alliance ruptures (i.e., a week to week severe downward trend in alliance scores), and implement an intervention to repair the alliance with these clients. Do clients with whom a therapist has implemented this procedure drop out at a lower rate? Does this process of deliberately identifying alliance ruptures and repairing them lead to enhanced therapist performance regarding alliance ruptures? This form of deliberate practice (testing disconfirmable hypotheses based on quality information) might lead to greater expertise in identifying alliance ruptures.

Lambert, M. J. (2012). Helping clinicians to use and learn from research-based systems: The OQ-analyst. Psychotherapy, 49(2), 109.

One of the more interesting and clinically relevant trends in psychotherapy research and practice in the past 10 years is the emergence of research on continuous progress monitoring. Continuous progress monitoring occurs when a patient is given a standardized self report measure before a session and the results of patient functioning are fed back to the therapist. (This is distinct from a clinician asking a patient for a verbal account of how he or she is doing this week). The standardized self report assessment is often done repeatedly, sometimes before every session or every fixed number of sessions. Measures, such as the Outcome Questionnaire (OQ) for adults or youths, was specifically designed for this purpose. The OQ assesses symptoms, interpersonal functioning, and life functioning, and clients are identified as improving (i.e., on course), or at risk of deteriorating. Recently, a small meta analysis of 3 to 4 studies representing 454 to 558 clients on the effects of progress monitoring found a moderate relationship between monitoring plus feedback and client outcomes. The method is particularly effective in changing the course of outcomes for patients who are deteriorating. Large research reviews of evidence based treatments in randomized controlled trials show that about 40% to 60% of patients improve or recover from psychotherapy, 30% to 50% may not benefit, and 3% to 14% deteriorate (see my March 2013 blog). These proportions are likely less positive in everyday practice in which clients are not highly screened to meet research inclusion criteria. Unfortunately, clinicians’ views of their own client outcomes are unrealistically positive. In one survey, clinicians in routine practice reported that about 85% of their clients improved or recovered. About 90% of therapists rated themselves in the upper quartile and none rated themselves as below average (50^th percentile). Also there is serious doubt about the ability of clinicians to identify clients during the course of therapy, who ultimately deteriorate. In the paper by Lambert on the use of the Outcome Questionnaire (OQ), he reviewed several studies on continuous progress monitoring in everyday practice. Each therapist was asked to practice as they routinely do with half their usual caseload. With the other half of their caseload clients completed the OQ and the therapist received feedback before every session about patient progress. The feedback did not make a difference for clients who made steady progress (i.e., on track) from week to week. However, continuous progress monitoring did make a difference for the 20% to 30% who showed some sign of deteriorating at some point in treatment. Notifying therapists that these patients were in trouble reduced the rate of deterioration from 20.1% to 5.5%, and monitoring and feedback increased positive outcomes from 22.3% to 55.5%.

Practice Implications

Lambert reported that clinicians in these “practice as usual” studies were initially skeptical but quite surprised at the outcomes related to continuous progress monitoring. Standardized assessments appear to get around the problem of clinician over-estimation of their patients’ positive outcomes. Clinicians were able to more accurately identify clients at risk of deteriorating likely resulting in the therapist doing something different to forestall the negative consequences. Lambert argues that it is in the best interest of at-risk patients to have their symptoms, interpersonal functioning, and life functioning formally monitored throughout treatment. However, clinicians are likely to resist doing so because they believe that they are already highly successful, and even more so than the typical outcomes produced by clinical trials. Formal monitoring of client outcomes has little downside for clinicians (it is inexpensive and requires little training), and it has many upsides for clients, especially those who are at risk for deteriorating.

Author email: michael_lambert@byu.edu

Castonguay, L.G., Boswell, J.F., Zack, S., Baker, S., Boutselis, M., Chiswick, N., Damer, D., Hemmelstein, N., Jackson, J., Morford, M., Ragusea, S., Roper, G., Spayd, C., Weiszer, T., Borkovec, T.D., & Grosse Holtforth,, M. (2010). Helpful and hindering events in psychotherapy: A practice research network study. Psychotherapy: Theory, Research, Practice, and Training, 47, 327-344.

There are many reasons why I like this paper, and one reason is that it is a psychotherapy practice research network study (most of the co-authors are independent practice clinicians). This group of clinicians and researchers met on a number of occasions to define the research questions, including: “what do psychotherapists and clients find most and least helpful in a psychotherapy session?”; and “do psychotherapists and clients agree on what was most and least helpful?” The clinicians and researchers also discussed and agreed on the method for collecting and analysing the data. Thirteen independent practice clinicians participated (6 CBT, 4 psychodynamic, and 3 experiental/humanistic). For a period of 18 months, all new clients were invited to participate so that 121 clients with a variety of disorders enrolled in the study. Clients and therapists filled out (on an index card) parts of the Helpful Aspects of Therapy (HAT) measure, which asked them to report, describe, and rate particularly helpful and hindering events from the session they had just completed. For example clients and therapists were asked: “Did anything particularly helpful happen during this session?”; and “Did anything happen during this session which might have been hindering?” When participants answered “Yes” to either of these questions, they were asked to briefly describe the event(s), and then rate them on a scale from 1 to 4 for level of helpfulness or level of hindrance. Both clients and therapists did so at the end of every therapy session. Close to 1500 therapeutic events were recorded by the clients and therapists. The events were then coded and categorized according to type of event by independent raters using an established coding system. Clients rated self-awareness, problem clarification, and problem solution as the most helpful type of events, although self-awareness was significantly the most identified of all helpful events by clients. Therapists rated self-awareness, alliance strengthening, and problem clarification as the most helpful type of events. Therapists identified self-awareness and alliance strengthening significantly more often than any other helpful events. Hindering events were identified much less frequently by clients and therapists. Client identified poor fit (e.g., therapist tried something that didn’t fit the client’s experience) as the most frequent hindering event category. Therapists identified therapist omissions (i.e., failure to provide support or an intervention) as the most frequent hindering event category. Overall, with the exception of self-awareness, therapists and clients did not agree on what were the most helpful or hindering events in therapy.

Practice Implications

Results regarding self awareness indicate that providing clients with opportunities to achieve a clearer sense of their experience (e.g., emotions, behaviors, and perceptions of self) is frequently reported as beneficial by both clients and therapists. The events that therapists most frequently reported as detrimental were those in which they failed to be attuned to their clients’ needs. This may reflect therapists’ concerns with potential alliance ruptures. The overall lack of agreement between therapists and clients on helpful and hindering events raises the question about whether therapists are not aware enough of clients’ experiences, or whether clients are not knowledgeable about what is in fact therapeutic. Perhaps client and therapist ratings of events represent complementary perspectives on what works or does not work in psychotherapy. Regarding participating in research, these independent practice therapists reported that the procedure of writing down helpful and harmful events and reading what their clients wrote after each session had a positive impact on their practice. That is, the process of data collection became immediately relevant to their clinical work.

Author email: lgc3@psu.edu