Blog

Goldberg, S.B., Rousmaniere, T., Miller, S.D., Whipple, J., Nielsen, S.L., Hoyt, W.T., & Wampold, B.E. (2016). Do psychotherapists improve with time and experience? A longitudinal analysis of outcomes in a clinical setting. Journal of Counseling Psychology, 63, 1-11.

Do therapists get better in providing psychotherapy as they gain more experience? This is a long standing question in psychotherapy, and most studies that compare therapists of different experience levels have not provided encouraging findings. This large longitudinal study in a practice setting by Goldberg and colleagues is unique because they follow therapists over a number of years during their careers. That is, the authors do not focus on outcome differences between therapists with different levels of experience, but rather they see if a therapist improves over time as the therapist accrues experience. Data were collected on 170 therapists and 6,591 patients over 18 years in a large practice in the U.S. Patients were distressed adults who attended an average of 8 sessions (range = 3 to 153) across 13 weeks. Over the 18 years of the study, on average therapists saw 39 patients, saw their first patient of the study after their 5^th year post graduate school, and had been working at the practice for about 5 years. On average patients got better, so that their psychological symptoms declined significantly over the course of treatment (i.e., 50% reliably improved). These rates of improvement are similar to benchmarks set in clinical trials. Contrary to expectations, therapists tended to have slightly poorer patient outcomes as the therapists gained experience. This result remained significant even when patient baseline severity, therapist caseload size, and other factors were controlled. However, more experienced therapists tended to have fewer early unplanned terminations (< 2 sessions) than less experienced therapists.

Practice Implications

This is the first large longitudinal study that followed therapists over several years of their career. Therapists became less effective over time, although the magnitude of the deterioration was very small. At the very least, one can say that patients did not achieve better outcomes as their therapists became more experienced. The authors note that the results of this study are in contrast to a large therapist survey in which most practitioners reported that their skills improved with passing time, and in contrast to another study in which therapists tended to over-estimate their effectiveness and under-recognize failing cases. Ways for therapists to improve their skills and patient outcomes might include: engaging in regular progress monitoring, targeted learning of fundamental therapeutic skills, training with standardized patients, and setting aside time for reflection and clinical consultation.

Miller, D.J., Spengler, E.S., & Spengler, P.M. (2015). A meta-analysis of confidence and judgement accuracy in clinical decision making. Journal of Counseling Psychology, 62, 553-567.

People can make errors in judgements based on decision making rules that are biased. Clinicians also may be prone to making such errors. In their Nobel Prize winning work, Kahneman and Tversky outlined a number of heuristics (i.e., mental shortcuts) that lead to cognitive biases, which in turn affect accuracy of decisions. For example, when making a differential diagnosis clinicians may: rely too heavily on only one piece of information which may be the most available (e.g., “I vividly remember a patient with conversion disorder who had the same history”); or ignore that a particular event (e.g., conversion disorder) is very rare; or seek confirming rather than disconfirming evidence (e.g., the patient has PTSD symptoms that can explain some symptoms). Complicating these biases is the tendency for clinicians to be over-confident. For example, in one study the average psychotherapist rated their performance as better than 80% of their peers, and no therapist rated him or herself in the lower 50^th percentile among peers. In their meta analysis, Miller and colleagues reviewed 36 studies of the relationship between clinician confidence ratings and accuracy of decisions among 1,485 clinicians. The authors were particularly interested in the overconfidence bias, which occurs when individuals report higher confidence in their judgments than is warranted by their actual accuracy. For example, studies have assessed the impact of clinician confidence on clinical accuracy in: detecting random responding on a psychological test, diagnosing a brain disorder verified by medical test using neuropsychological test data, predicting future violence and recidivism in offenders, and patient progress in psychotherapy. Most studies find that clinicians are quite confident in their judgments. But, is this confidence warranted? Miller and colleagues’ meta analysis found a significant but small (r = .15) association between confidence and accuracy. This suggests that clinician confidence is only slightly indicative of decision-making accuracy. The effect was a little larger for more experienced clinicians (r = .25), indicating that more experience and training resulted in somewhat more consistency between a clinician’s confidence and their clinical accuracy. Further, higher confidence leads to poorer accuracy when clinicians have to make repeated decisions without feedback, when feedback is not written, and when an event is rare.

Practice Implications

Clinicians, like everyone else, are sometimes subject to making errors when they only look at confirming evidence, when they rely only on their own memory rather than objective data, and when they are over-confident. Accuracy can be increased when clinicians use decision-making aids that provide quality corrective feedback. Aids to help in decision making might include the use of: objective standardized test data, repeated measurements with feedback to assess patient progress in psychotherapy, and actively looking for disconfirming evidence before making a clinical judgement. As the authors conclude, confidence is not a good substitute for accuracy.

Owen, J., Drinane, J. M., Idigo, K. C., & Valentine, J. C. (2015). Psychotherapist effects in meta-analyses: How accurate are treatment effects? Psychotherapy, 52(3), 321-328.

One of the ongoing debates in the psychotherapy research literature has to do with the relative efficacy of psychotherapies. Is psychotherapy brand A (CBT, for example) more effective than psychotherapy brand B (psychodynamic therapy, for example)? The most common way to test this question is with randomized controlled trials (RCTs), in which clients are randomly assigned to treatment condition (brand A or B). This study design controls for systematic bias in the results that may be caused by differences between clients. But what about therapists? We know for example that therapist effects (i.e., differences between therapists) account for approximately 5% to 10% of client outcomes. Therapist effects are often larger than the effect of the empirically supported treatment that is being offered. Yet it is almost unheard of for therapists to be randomized to treatments, so therapist effects are not controlled in most psychotherapy trials. As a result the effects of the differences between therapists get statistically rolled into the treatment effects. As Owen and colleagues point out, the impact of not controlling for therapist effects is that some differences between treatments in an RCT will appear statistically significant when in fact they are not. One can control for the effect of therapist differences, thus providing a more accurate estimate of treatment effects, but this is rarely done in published RCTs. So, when these RCTs are summarized in a meta analysis, the meta analysis results are also affected by ignoring therapist effects. In their study, Owen colleagues did something very clever. They took data from 17 recent meta analyses of RCTs that found differences between two interventions. These included meta analyses of studies comparing: CBT vs alternative treatments, bona fide treatments vs non-bona fide treatments, culturally adapted treatments vs those that were not adapted, etc. There are many other meta analyses that show no differences between treatments, but the authors wanted to focus specifically on the 17 that did show differences. Owen and colleagues statistically estimated what would happen to the original study findings of significant differences between treatments if therapist effects on patient outcomes were controlled. They controlled for three different sizes of therapist effects that accounted for: 5% (small), 10% (medium), or 20% (large) of patient outcomes. Even small therapist effects (5%) reduced the number of significant differences between treatments from 100% to 80%. When psychotherapist effects were estimated to be medium (10% - which is the best estimate based on research), the number of significant differences between treatments dropped to 65%. For large therapist effects (20%), the number of significant treatment differences was only 35%.

Practice Implications

I have argued previously that the psychotherapist matters. Placing more time and effort in developing good reflective practice based on quality information and developing therapist skills like empathy, progress monitoring, and identifying and repairing alliance ruptures will result in better patient outcomes. As Owen and colleagues note, when reading an RCT that claims to find significant differences between psychotherapies, ask yourself if they took into account the effects of differences between therapists.

Wampold, B.E. & Imel, Z.E. (2015). The great psychotherapy debate (2^nd ed.). New York: Routledge.

The Great Psychotherapy Debate: Starting in April, 2015 I review parts of The Great Psychotherapy Debate (Wampold & Imel, 2015) in the PPRNet Blog. This is the second edition of a landmark, and sometimes controversial, book that surveys the evidence for what makes psychotherapy work. You can view parts of the book in Google Books

In the concluding chapter of their book, Wampold and Imel discuss the evidence and strategies that therapists can use to improve patient outcomes. As indicated in previous PPRNet Blogs, Wampold and Imel concluded that the differences between specific treatment approaches is small. In other words, Wampold and Imel argue that there is no good evidence that one bona fide psychotherapy is more effective than another for most disorders. By “bona fide” treatments, they mean psychotherapy that: provides the client with a plausible theory/explanation of the disorder, delivers a structured intervention based on the plausible theory, and is offered by an effective therapist. The authors also found that contextual factors (e.g., therapeutic alliance, therapist empathy, client expectations) accounted for a sizeable proportion of patient outcomes. A key element in this understanding of effective therapy is the role of the therapist. The authors reviewed various studies and meta analyses that showed that therapists differ widely in their outcomes and in their ability to establish a therapeutic alliance. Unfortunately, therapists tend to be overly-optimistic about their clients’ outcomes. Therapists often do not have quality data on their clients’ progress, and the complexities of the therapeutic work makes it difficult for therapists to keep in mind all aspects of the therapy that is helpful or not helpful to clients. For example, some therapists may be good at establishing an alliance, but they may not be so good at providing a viable treatment structure. Other therapists may be highly empathic with clients who have moderately severe symptoms, but the same therapists may not respond as empathically with more difficult clients. Outcome or process monitoring (i.e., providing therapists with reliable information about the ongoing status of patient symptoms or about the quality of the therapeutic relationship) provides an evidence-based aid in helping therapists to improve their clients’ outcomes.

Practice Implications

Regardless of the type of psychotherapy they use, therapists are responsible for achieving good outcomes for their clients. This includes continually developing therapeutic skills over time. There is some evidence that a reflective attitude towards one’s psychotherapy practice is helpful. Unfortunately, therapists may not be continually improving or reflecting on their practice. This is indicated by research showing that trainees and interns appear to be as competent as experienced clinicians. Therapists need quality information about their clients in order to improve their own practice and clients’ outcomes. But psychotherapy practice is complex, the therapeutic relationship is multifaceted, and clients are variable in their presenting issues and life experiences. All of these make it difficult for any therapist to make accurate decisions in therapy. Progress or process monitoring (i.e., continually measuring outcomes and relationship processes with a psychometrically valid instrument), may be one way for therapists to receive high quality feedback about patient progress in order to improve their psychotherapy practice.

Tritt, A., Kelly, J., & Waller, G. (2015). Patients’ experiences of clinicians’ crying during psychotherapy for eating disorders. Psychotherapy, 52(3), 373-380.

Psychotherapy can be an emotionally intensive experience for both patients and therapists. In a large survey, more than 70% of therapists reported having cried in therapy, and 30% cried during the past month. Therapists who cried almost always saw the experience as positive or neutral (99.2%) for the patient and the therapeutic relationship. Do clients feel the same way about therapists who cry? In this study, Tritt and colleagues surveyed 188 adult patients with an eating disorder who were recently in psychotherapy. Of those, 107 (56.9%) reported that their therapist had cried during therapy. There was no association between frequency of therapist crying and therapist age, patient diagnosis, or type of psychotherapy (i.e., manual-based or not). Therapists who cried a moderate amount were seen by clients as having a positive demeanor (i.e., happy, firm, consistent), whereas therapists who cried more extremely were rated by clients as having a more negative demeanor (i.e., anxious, angry, bored). If therapists who cried were generally perceived by clients to have a positive demeanor, then therapist crying had a positive impact on therapy. That is, clients reported a greater respect for the therapist, greater willingness to express emotions, and higher willingness to undertake therapy in the future. However, if therapists who cried were generally perceived by clients to have a negative demeanor, then therapist crying had a negative impact on therapy. That is, clients were less willing to express emotions in therapy and to undertake therapy in the future. Further, if the therapist who cried was rated as having a negative demeanor, the client experienced more self blame, assumed that there was something wrong in the therapist’s life, and that the therapist and client did not share the same perspective on the client’s life and treatment.

Practice Implications

This small but unique and interesting survey sheds some light on clients’ experiences of therapists who cry during therapy. More than half of clients experienced their therapist crying during therapy. In contrast to surveys of therapists who tend to evaluate therapist crying as exclusively positive or neutral, this survey found that many but not all clients experienced therapist crying as positive. It depends on how the client perceives the therapist as a person. Therapists who are seen by clients as happy, firm, and consistent may assume that patients will experience their crying as a positive indicator of the therapeutic relationship. However, therapists who are seen by clients as anxious, bored, or angry cannot assume that clients will see their tears as being positive for therapy.

Chow, D. L., Miller, S. D., Seidel, J. A., Kane, R. T., Thornton, J. A., & Andrews, W. P. (2015). The role of deliberate practice in the development of highly effective psychotherapists. Psychotherapy, 52(3), 337.

In 2014, Tracey and colleagues caused a stir when they claimed that there was no evidence of expertise in psychotherapy (see my July, 2014 blog). They defined expertise as increased quality of performance that is gained with additional experience – and they concluded that psychotherapy research has not provided evidence that therapist performance improves with experience. The issue is important because differences between therapists account for over 5% of patient outcomes. This seems small, but it is larger than variance in outcomes accounted for by the use of empirically supported treatments (0% - 4%), and almost as large as the variance accounted for by client-rated alliance (5% - 15%). Across a wide variety of professions (e.g., music, medicine, chess, sports), professionals’ engagement in deliberate practice results in improvement and superior performance. However, there is little evidence of this in psychotherapy. In this article by Chow and colleagues, the authors look specifically at “deliberate practice” defined as individualized training activities to improve one’s performance through repetition and refinement. To be effective, deliberate practice has to be focused on achieving specific targets and guided by conscious monitoring of outcomes over a long period of time. The authors collected a sample of 69 therapists who worked across a number of organizations and practice areas, and these therapists provided data related to 4,850 patients. Seventeen of the 69 therapists who treated 1,632 clients also provided data on professional development activities. Therapists were multidisciplinary (i.e., counsellors, psychologists, marital therapists, social workers, psychotherapists) with an average of over 8 years of experience, who worked mainly in private practice or within the national health service in the U.K., and who primarily treated adult patients with depression or anxiety disorders. Patient outcomes were measured repeatedly with a valid standardized scale, and deliberate practice was self reported by therapists using a measure that asked about the frequency and time therapists engaged in 25 activities outside of work aimed at improving therapeutic skills. On average, clients improved by the end of treatment and the effect was large (d = 1.22). As expected therapists differed in their patient outcomes (i.e., some therapists were reliably more effective than others). Therapist demographic variables, theoretical orientation, years of experience, and practice setting were not related to patient outcomes. However, the amount of time in deliberate practice activities was associated with a reduction in client distress. Compared to the less effective therapists (2.62 hrs/wk in deliberate practice), the best performing therapists (7.39 hrs/wk in deliberate practice) spent about 2.81 times more time on deliberate practice. Therapists rated the following deliberate practice activities as the most relevant to their patients’ outcomes: reviewing challenging cases, attending training workshops, reflecting on past sessions, and reflecting on what to do in future sessions.

Practice Implications

Although this is a single study with a relatively small sample of therapists, it is one of those rare studies to assess the effects of therapist deliberate practice on patient outcomes. As is the case with other professions, reviewing one’s performance can play an important role in identifying errors, altering course, and remediating problems. As Tracey and colleagues indicated, therapists need good quality information in order to learn from their errors and make adjustments so that clients can improve. Quality information might be available from progress monitoring (i.e., continuous feedback to therapists about client outcomes), which has been shown to improve client outcomes especially for at-risk cases. Chow and colleagues go further to suggest targeted learning by using standardized clients within training and supervision contexts. Deliberate practice is not only for newer or less experienced therapists, since experienced therapists also vary in their ability to engage and help clients. Highly effective therapists spend more time engaging in activities outside of their practice specifically aimed at improving their performance.