Blog

Delgadillo, J., Deisenhofer, A.-K., Probst, T., Shimokawa, K., Lambert, M. J., & Kleinstäuber, M. (2022). Progress feedback narrows the gap between more and less effective therapists: A therapist effects meta-analysis of clinical trials. Journal of Consulting and Clinical Psychology, 90, 559–567.

Some therapists are more effective than others. This is often referred to as the therapist effect. Somewhere between 1% and 29% of patient outcomes can be attributed to which therapist the patient receives. In general, therapists with high facilitative interpersonal skills, high humility, and an ability to withstand difficulties in practice (i.e., ruptures, burnout) may be more clinically effective. To improve outcomes in therapy, some have suggested using routine outcome monitoring and progress feedback. This involves regularly measuring and tracking patient progress with standardized self-report scales throughout treatment and providing the clinician with this information during therapy. Progress feedback allows the therapist to compare their patient’s progress against norms and against the patient’s own progress in preceding sessions. If the patient is not progressing or is deteriorating, then the therapist is alerted to address the issue. Research indicates that progress feedback makes therapy more effective. Less is known about how progress feedback leads to better outcomes. In this meta-analysis, Delgadillo and colleagues assessed the impact of progress feedback on the therapist effect – that is, does progress feedback improve the outcomes of less effective therapists? The meta-analysis was of six clinical trials with data from 4,549 patients and 131 therapists who were randomly assigned to a progress feedback condition or to a control condition without progress feedback. The variability between therapists (ICC = .011) suggested that 1.1% of the overall variance in patient outcomes was due to therapist effects. However, feedback was associated with a significant reduction in the therapist effect (ICC = .009) by 18.2%. A closer look at the data indicated that progress feedback narrowed the gap between more and less effective therapists, such that patients of less effective therapists benefitted the most from their therapist receiving feedback.

Practice Implications

In this meta-analysis conducted on data from controlled studies, there were few under-performing therapists. However, implementing progress feedback was clinically important to achieve better outcomes among some of these therapists. That is, even a single underperforming therapist could attain relatively poor outcomes with dozens or even hundreds of patients. Who the therapist is matters – and some therapists (and their patients) can benefit from supplementing clinical judgement with reliable feedback about patient progress throughout the course of psychotherapy.

Lutz, W., de Jong, K., Rubel, J.A., & Delgadillo, J. (2021). Measuring, predicting, and tracking change in psychotherapy. In M. Barkham, W. Lutz, and L.G. Castonguay (Eds.) Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change (7^th ed.). Wiley. Chapter 4.

Routine outcome monitoring is also known as progress monitoring and feedback. This involves regularly assessing patients with a psychometrically reliable scale before a therapy session and providing the therapist with feedback on the patient’s progress. The feedback includes how the patient is doing relative to the average patient, and how the patient is doing in this session relative to their own scores in previous sessions. By doing this, therapists can get regular and reliable information about their patient’s progress and be alerted to when the patient is not on track to improve or is getting worse. Decisions about patient improvement on a session-to-session basis are complex because they involve knowing how the patient is doing regarding symptoms, quality of life, and relationship functioning relative to other patients and relative to their own functioning in the past. No wonder therapists often mis-judge when a patient is getting worse. Routine outcome monitoring involves decision-making tools to enhance a clinician’s decisions – like the way a physician uses a blood test or x-ray to enhance their clinical observations. In this part of the chapter, Lutz and colleagues review the research over the past 50 years on outcome monitoring and feedback. The effects of psychotherapy with feedback compared to psychotherapy without feedback ranges from small (g = .07) to medium (d = .40) in size. These effects seem small, but the authors remind us that feedback is a relatively simple clinical tool provided in addition to psychotherapy, and so these positive effects occur are over and above the general effectiveness of psychotherapy. Highest effect sizes are achieved for clients who are not on track (likely to get worse) such that feedback compared to no feedback in these not-on-track patients result in effect sizes ranging from 0.36 to 0.53, indicating a moderate to large effect. Further, when feedback was provided, patient dropout was reduced by 20% compared to when feedback was not provided.

Practice Implications

Although routine outcome monitoring is relatively easy to use, there are barriers to their implementation. Organizational cultures are difficult to change, and resources must be assigned to implement these strategies. Clinicians must have the technology, some training, and funds to purchase the psychometric scales which may be a challenge for some. And attitudinal barriers are a problem if managers or clinicians do not value outcome measurement. Nevertheless, patient reported outcomes with psychometrically valid scales should be central to ensure good patient-centered care. Psychotherapists can benefit from quality information to help their clinical decision making, especially when it comes to identifying patients who might not be benefitting. Such feedback about patients who are at risk of getting worse may help clinicians to adjusting treatment and their interpersonal stances to these patients.

Lambert, M. J., Whipple, J. L., & Kleinstäuber, M. (2018). Collecting and delivering progress feedback: A meta-analysis of routine outcome monitoring. Psychotherapy, 55(4), 520-537.

Somewhere between 5% and 10% of adult clients in clinical trials of psychotherapy get worse, and the numbers are likely higher in regular clinical practice. In addition, some therapists are more effective than others, so that some therapists have few clients who get worse whereas others consistently have high rates of poor client outcomes. Unfortunately, therapists have a difficult time assessing their client outcomes. Many therapists are overly optimistic about their clients’ outcomes, and clinicians frequently do not identify when clients get worse. One likely reason for this erroneous assessment of client outcomes is that typically psychotherapists do not have quality information in order to make accurate decisions and predictions. Assessing client outcomes on a regular basis throughout treatment is a difficult and complicated endeavour, and one that is beyond the capacity of most people. So, like other professionals (pilots, air traffic controllers, engineers) psychotherapists can improve their predictions and decision-making if they have access to quality information about their clients’ functioning. One source of such information for psychotherapists could be from the use of routine outcome monitoring. Routine outcome monitoring involves assessing client mental health functioning with reliable psychometric scales throughout the course of treatment, and feeding this information back to therapists who can use the data to adjust what they are doing if necessary. The two most commonly used outcome monitoring tools are the Outcome Questionnaire-45 (OQ-45) which is part of the OQ Analyst Feedback System, and the Outcome Rating Scale (ORS) and Session Rating Scale (SRS) which are part of the Partners for Change Outcome Management System (PCOMS). In this meta-analysis, Lambert and colleagues assessed the effect of regular outcome monitoring with the OQ-45 and the ORS to improve client outcomes. In 15 studies with almost 8,500 participants, the OQ-45 outperformed treatment as usual but with a small effect (SMD = .14, 95% CI [.08, .21]). However, the positive effect of using the OQ-45 with feedback was larger for the 31.2% of clients who were not doing well in therapy (SMD = .33, 95% CI [.25, .41]). Among those studies that used the OQ standardized feedback system that provides recommendations to therapists, the effects were even larger (SMD = .49, 95% CI [.25, .73]). Similarly, in nine studies with over 2,000 participants, the effects of using the PCOMS system had a small to moderate positive effects on client outcomes (SMD = .40, 95% CI [.29, .51]).

Practice Implications

The research evidence supports the use of routine outcome monitoring with the OQ-45 or the PCOMS to improve client outcomes. Quality information that is fed back to clinicians can compensate for the limited capacity that any clinician has to accurately detect a client that is worsening in psychotherapy. The information provided to therapists with these feedback systems can highlight potential problems in the client and identify strain in the therapeutic alliance. This information can sensitise therapists to at-risk clients and situations, and encourage therapists to adjust their interventions or interpersonal stances accordingly.

Author email: lambert.michaelphd@gmail.com

Norcross, J. C., & Lambert, M. J. (2018). Psychotherapy relationships that work III. Psychotherapy, 55(4), 303-315.

Relationship factors in psychotherapy are some of the most important predictors of patient outcomes. They outweigh factors like the type of therapy provided in determining whether patients get better after psychotherapy. In this second overview article, Norcross and Lambert provide a review of 17 meta-analyses of relationship factors in psychotherapy that contribute to positive outcomes. Like the review of patient factors also found in this blog and E-Newsletter, this article briefly outlines those evidence-based relationship factors that reliably predict patient outcomes in psychotherapy. The therapeutic relationship refers to how the therapist and patient relate to each other, or their interpersonal behaviors. By contrast, techniques or interventions refer to what is done by the therapist. Practice guidelines typically focus on interventions or therapeutic orientation. As the authors argue, what is missing from treatment guidelines are the person of the therapist and the therapeutic relationship – evidence for which is backed up by 5 decades of research. Even in studies of highly structured manualized psychotherapy for a specific disorder in which efforts were made to reduce the effect of individual therapist, up to 18% of outcomes (a moderate to large effect) could be attributed to the person of the therapist. By contrast somewhere between 0% and 10% of outcomes (a small to moderate effect) is attributable to specific treatment methods. So, which therapeutic relationship factors are reliably related to patient outcomes? These include: the therapeutic alliance in individual therapy (306 studies, g = .57) couple therapy (40 studies, g = .62), and adolescent psychotherapy (43 studies, g = .40), collaboration (53 studies, g = .61) and goal consensus (54 studies, g = .49), cohesion in group therapy (55 studies, g = .56), therapist empathy (82 studies, g = .58), collecting and delivering client feedback or progress monitoring (24 studies, g = .14 to .49), managing countertransference (9 studies, g = .84), and repairing therapeutic alliance ruptures (11 studies, g = .62) among others. Over the next few months, I will be reviewing these meta analyses in more detail to discuss how therapists can use this evidence base to improve their patients’ outcomes.

Practice Implications

The research as a whole indicates that therapists should make the creation and cultivation of the therapeutic relationship a primary goal of therapy. Factors such as managing the therapeutic alliance, repairing alliance ruptures, engaging in ongoing progress monitoring, managing countertransference and others should be used to modify treatments and interpersonal stances in order to maximize outcomes. When seeking out professional development and training, practitioners should focus on evidence-based relationship factors (managing the alliance, judicious self disclosure, managing emotional expression, promoting credibility of the treatment, collecting formal feedback, managing countertransference) in addition to focusing on evidence-based treatments.

Murphy, R. & Hutton, P. (2017). Therapist variability, patient reported therapeutic alliance, and clinical outcomes in adolescents undergoing mental health treatment: A systematic review and meta-analysis. The Journal of Child Psychology and Psychiatry, doi:10.1111/jcpp.12767.

The therapeutic alliance refers to the affective bond between therapist and client, and their agreement on the tasks and goals of therapy. The alliance is a well-known predictor of outcomes in adult psychotherapy with a mean alliance-outcome correlation of r = .28. Less is known about the role of the alliance in the treatment of adolescents. Some reviews indicate that the alliance-outcome relationship in children and adolescents is weaker than observed among adults, but these reviews may have been flawed since they included both children and adolescents in the same review, and the number of studies they reviewed was small. A large rigorous systematic review of adolescents’ perceptions of the alliance can provide insight into their experience of psychological treatment and inform routine mental health practice. In their meta analysis, Murphy and Hutton reviewed studies of clinical samples of adolescents between the age of 12 – 19 who received psychological treatment. The authors made sure that the measures of alliance and outcomes were reliable, they excluded studies of those with medical and neurocognitive problems, and included only studies with adolescents (i.e., excluding studies with primarily children). Twenty-seven studies with almost 3,000 participants were included. Main presenting problems of adolescent patients were: substance use, eating disorders, behavioral difficulties, and a range of mood and anxiety disorders. The mean weighted effect size of the alliance-outcome relationship among studies of psychological treatment of adolescents was r = .29 (95% CI: 0.21, 0.37; p < .001) indicating a moderate effect.

Practice Implication

This is the largest meta analysis of the alliance-outcome relationship in the psychological treatment of adolescents with mental health problems. The alliance was moderately associated with outcomes, and so therapeutic alliance may be a reliable predictor of clinical progress in the treatment of adolescents. The findings suggest that those working with adolescents should routinely assess the alliance after each session in order to evaluate if they need to address relational barriers to positive outcomes. For example, if the alliance markedly declines from one session to the next, then clinicians should address potential problems in their relationship with the adolescent client, renegotiate goals, or renegotiate the tasks of therapy.

Hall, G.C.N., Ibarak, A.Y., Huang, E.R., Marti, C.N., & Stice, E. (2016). A meta-analysis of cultural adaptations of psychological interventions. Behavior Therapy.

Cultural adaptation of psychological interventions involves identifying cultural contexts of behaviors and developing constructs of mental health functioning relevant to the cultural context. Most cultural adaptation of psychotherapies involves taking existing treatments originally developed for those of European ancestry and adapting them for another specific cultural group or context. However, a few efforts exist in which new treatments were developed within a particular culture to address culture-specific concerns. Eight dimensions along which interventions could be culturally adapted include: language, people, metaphors, content, concepts, goals, methods, and context. Some researchers have expressed concern that cultural adaptation could distance an intervention from its evidence-base, and reduce its effectiveness. In this meta analysis by Hall and colleagues, the researchers look closely at the effects all culturally adapted treatments and prevention methods. They reviewed 78 studies that included nearly 14,000 participants. All studies included culturally adapted interventions for individuals of non-European ancestry. For example, these included studies that adapted CBT interventions for various disorders (mainly depression and anxiety disorders), or studies that match therapist to client in terms of ethnicity. Only 5% of studies created a new intervention developed within a particular culture, whereas the vast majority of studies adapted an existing treatment initially developed for clients of European ancestry. The average effect size was g = .67 (confidence intervals not reported), indicating that culturally adapted interventions produced better outcomes than comparison conditions.  Culturally adapted interventions were also more likely to result in better outcomes than the same interventions that were not adapted (g = .52). Effect sizes for cultural adaptation in treatment studies (g = .72) were larger than for prevention studies (g = .25), likely because participants in treatment studies had higher levels of initial psychopathology. There was little evidence that matching therapist and client on ethnicity was helpful.

Practice Implications

This meta analysis provides compelling evidence that cultural adaptation of existing treatments can result in more positive outcomes compared to not adapting the same treatment. The effect sizes may even underestimate the true effects of cultural adaptation because the outcome variables like measures of depression were rarely adapted to a specific culture (e.g., depression among Chinese participants may be expressed differently than depression among European participants, and most depression measures were created by and for Europeans).