Blog

Castonguay, L.G., Barkham, M., Youn, S.J., & Page, A. (2021). Practice-based evidence: Findings from routine clinical settings. In Barkham, W. Lutz, and L.G. Castonguay (Eds.) Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change (7^th ed.). Wiley. Chapter 6.

Practice-based evidence refers to research that is conducted as part of routine clinical practice. Often these studies do not impose strict research conditions like randomization of patients, and so they produce findings that are more relevant to psychotherapy as practiced real-world. In studies from practice-research networks, clinicians are often involved in the design and implementation of the study. Our Psychotherapy Practice-Research Network (PPRNet) is an example of a collaboration between researchers and clinicians to produce practice-based evidence. In a large survey, we found that most clinicians regardless of theoretical orientation wanted more research on the therapeutic relationship and on professional development. And so PPRNet developed a research program on training psychotherapists to identify and repair therapeutic alliance ruptures and microaggressions. In this chapter, Castonguay and colleagues review some of the key findings from practice research networks. First, White patients report better outcomes than Black patients in routine care, and these differences were linked back to the clinicians. That is, therapists varied in their effectiveness with racial and ethnic minority patients. Second, patients benefit when clinicians monitor the therapeutic alliance and outcomes on a session-to-session basis using reliable and valid measures. Noticing when a patient’s ratings of the alliance decline from one session to the next, might indicate a problem in the therapeutic relationship. Third, when monitoring the alliance, therapists should also rate the alliance. If a therapist’s ratings of the alliance are higher than their patients, then this may be a sign that the therapist is not paying attention to problems in the alliance. Fourth, developing and maintaining a strong therapeutic alliance likely will improve patient outcomes. Fifth, a reliable and valid assessment of client’s past and current interpersonal difficulties is likely to improve a therapist’s treatment plan for that patient.

Practice Implications

Doing research in clinical practices is not as well controlled as clinical trials research. But practice-based evidence is more relevant to how psychotherapy is done in the real world with real patients. The research is not so clear about why some therapists are more effective with racial and ethnic minority (REM) patients. However, complementary research suggests that some therapists who have a previously high level of multicultural orientation (cultural humility, open to conversations about culture, and cultural comfort) are more effective with REM patients. Also monitoring the therapeutic alliance with a valid scale on a session-to-session basis leads to better outcomes. Such monitoring will alert the therapist to resolve an alliance rupture if a patient’s ratings decrease from one session to the next. If therapists also rate the alliance and find that their scores are higher than their patient’s, then this may alert the therapist to a potential problem. Finally, knowing if a patient has current and past interpersonal problems can inform a therapist to focus on how those problems affect current symptoms and to talk about how those problems manifest themselves in the therapeutic relationship.

Norcross, J.C. & Wampold, B.E. (2018). A new therapy for each patient: Evidence‐based relationships and responsiveness. Journal of Clinical Psychology, Online First, DOI: 10.1002/jclp.22678

Over the next several months, I will review in this blog results of a number of meta-analyses conducted recently on patient factors and relationship factors in psychotherapy. These factors provide evidence-based guidance to psychotherapists on how best to relate to and adapt to clients so that psychotherapy is more effective. This introductory article by Norcross and Wampold is an overview of the nine meta analyses related to transdiagnostic client factors to which therapists can adapt their interpersonal stances and treatment. The goal is to enhance treatment effectiveness by therapists tailoring therapy to individual client characteristics that are related to outcomes. Decades of research indicate that client transdiagnostic characteristics have more influence on outcomes than the particular treatment method, and likely more influence than the particular client diagnosis. The research indicates that giving the identical treatment to every client without adaptation to client characteristics is not an effective approach to providing psychotherapy. These meta analyses of client factors indicate that therapists should select different interventions and relational stances according to the client and the context. What are these client characteristics and therapist adaptations that are reliably related to outcomes? The client factors most strongly related to outcomes include therapist adaptations to: client culture/race/ethnicity (99 studies, g = .50); client preferences for type of therapy (51 studies, g = .28), client religion/spirituality (97 studies, g = .13 to .43), client stage of change (76 studies, g = .41), client reactance/resistance level (13 studies, g = .78), client coping style (32 studies, g = .53), and client attachment style (32 studies, g = .35). Over the next months, I will be reviewing in more detail these meta analyses of client factors and the practice implications of each so that therapists can use this evidence-base to help them to adapt to particular client characteristics.

Practice Implications

Practitioners will find that fitting the therapy to clients’ culture, stage of change, religion/spirituality, reactance/resistance, coping style, and attachment style will improve treatment outcomes. Doing so will have a greater impact on outcomes than the particular type of therapy provided or adapting treatment to the particular client diagnosis. The results of this large body of evidence suggests that therapists should no longer ask: “what is my theoretical orientation” but rather they should ask: “what relationship, adaptation, and approach will be most effective with this particular client”.

Norcross, J. C., & Lambert, M. J. (2018). Psychotherapy relationships that work III. Psychotherapy, 55(4), 303-315.

Relationship factors in psychotherapy are some of the most important predictors of patient outcomes. They outweigh factors like the type of therapy provided in determining whether patients get better after psychotherapy. In this second overview article, Norcross and Lambert provide a review of 17 meta-analyses of relationship factors in psychotherapy that contribute to positive outcomes. Like the review of patient factors also found in this blog and E-Newsletter, this article briefly outlines those evidence-based relationship factors that reliably predict patient outcomes in psychotherapy. The therapeutic relationship refers to how the therapist and patient relate to each other, or their interpersonal behaviors. By contrast, techniques or interventions refer to what is done by the therapist. Practice guidelines typically focus on interventions or therapeutic orientation. As the authors argue, what is missing from treatment guidelines are the person of the therapist and the therapeutic relationship – evidence for which is backed up by 5 decades of research. Even in studies of highly structured manualized psychotherapy for a specific disorder in which efforts were made to reduce the effect of individual therapist, up to 18% of outcomes (a moderate to large effect) could be attributed to the person of the therapist. By contrast somewhere between 0% and 10% of outcomes (a small to moderate effect) is attributable to specific treatment methods. So, which therapeutic relationship factors are reliably related to patient outcomes? These include: the therapeutic alliance in individual therapy (306 studies, g = .57) couple therapy (40 studies, g = .62), and adolescent psychotherapy (43 studies, g = .40), collaboration (53 studies, g = .61) and goal consensus (54 studies, g = .49), cohesion in group therapy (55 studies, g = .56), therapist empathy (82 studies, g = .58), collecting and delivering client feedback or progress monitoring (24 studies, g = .14 to .49), managing countertransference (9 studies, g = .84), and repairing therapeutic alliance ruptures (11 studies, g = .62) among others. Over the next few months, I will be reviewing these meta analyses in more detail to discuss how therapists can use this evidence base to improve their patients’ outcomes.

Practice Implications

The research as a whole indicates that therapists should make the creation and cultivation of the therapeutic relationship a primary goal of therapy. Factors such as managing the therapeutic alliance, repairing alliance ruptures, engaging in ongoing progress monitoring, managing countertransference and others should be used to modify treatments and interpersonal stances in order to maximize outcomes. When seeking out professional development and training, practitioners should focus on evidence-based relationship factors (managing the alliance, judicious self disclosure, managing emotional expression, promoting credibility of the treatment, collecting formal feedback, managing countertransference) in addition to focusing on evidence-based treatments.

Levy, K.N., Kivity, Y., Johnson, B.N., & Gooch, C.V. (2018). Adult attachment as a predictor and moderator of psychotherapy outcome: A meta‐analysis. Journal of Clinical Psychology. Online first publication, DOI: 10.1002/jclp.22685.

Adult attachment refers to characteristic ways people manage their emotions and relationship styles. Securely attached individuals adaptively and flexibly experience emotions and they are able to give and receive love and support to others. Insecure attachment can be sub-categorized as avoidant or anxious attachment. Those who are anxiously attached tend to up-regulate their feelings so that they may feel easily overwhelmed, and they tend to be preoccupied with relationship loss. Those with avoidant attachment styles tend to down-regulate their emotions so that they have difficulty experiencing or expressing feelings, and they might dismiss the importance of relationships as a means of protecting themselves. John Bowlby, the founder of attachment theory, argued that psychotherapy had the potential to serve as a secure base from which individuals might explore themselves and relationships. He also described the therapist as a temporary attachment figure with which the patient might develop an emotional bond to promote change and for a corrective experience. In this meta-analysis, Levy and colleagues looked at whether attachment dimensions can change in psychotherapy and whether they can predict improvement in patient symptoms pre- to post-therapy. (A note on meta analysis. It is a method of systematically reviewing a research literature, combining the effect sizes in that literature, and summarizing these effects. Because meta analyses usually contain many studies, their results are much more reliable than the results of any single study, and so they provide the most solid basis for making practice recommendations). In this meta analysis, Levy and colleagues included 36 studies, totaling 3,158 clients. Higher client attachment security (or lower attachment insecurity) at the start of therapy was associated with better outcomes by post-treatment (r = 0.17, p < 0.001, 95% CI = [0.13, 0.22], k = 32). Also, greater improvement in attachment security (change in attachment security from pre- to post-treatment) predicted better outcomes (r = 0.16, p < 0.001, 95% CI = [0.07, 0.25], k = 15). When looked at separately, higher levels of either attachment anxiety or attachment avoidance were associated with poorer outcomes, and change in either type of attachment insecurity was associated with better outcomes. These effects appeared to be consistent regardless of the type of therapy (non-interpersonal vs interpersonal therapies).

Practice Implications

Although attachment insecurity is associated with poorer outcomes, change in attachment insecurity is possible with psychotherapy and this change is associated with better symptom outcomes. Therapists should expect longer and more challenging treatment with patients who are anxiously attached. Anxiously attached individuals may appear engaged early in therapy, but they are quick to anger, feel rejected, and become overwhelmed. Such individuals may benefit from help to contain their emotional experiences by repeating the treatment frame and increasing structure. They may also benefit from interpersonally-oriented therapy focused on reducing their preoccupation with relationship loss. Avoidantly attached individuals may appear aloof, but they may be easily overwhelmed by demands for closeness. Therapists may have to carefully balance the amount of interpersonal space or demands in treatment with these clients so that they remain in therapy.

Reese, R. J., Duncan, B. L., Bohanske, R. T., Owen, J. J., & Minami, T. (2014). Benchmarking outcomes in a public behavioral health setting: Feedback as a quality improvement strategy. Journal of Consulting and Clinical Psychology, 82(4), 731-742.

Psychotherapy has demonstrated its efficacy in randomized controlled trials. But do these findings in highly controlled studies translate to everyday practice in publicly funded agencies that treat low income clients? Previous research in the US showed that outcomes of treatment-as-usual in public behavioural health agencies are generally not positive, so that only 20 to 35% of clients reliably improved. One approach to improving outcomes is to transport specific evidence-based treatments into practice settings. For example, research on applying CBT for panic and depression in a publicly funded agency resulted in similar outcomes to those achieved in randomized controlled trials. However, an alternative strategy of improving outcomes is to use continuous outcome monitoring, which involves repeated (weekly) measurement of client outcomes with reliable scales, and feedback to therapists on the client’s status relative to previous sessions and relative to other similar clients. Research has demonstrated that this strategy improves client outcomes and reduces the number of clients who deteriorate. In this study, Reese and colleagues examined the outcomes of a large public behavioural health service in the U.S. that treats low-income individuals. The service implemented repeated outcome monitoring of clients with feedback to therapists. Over 5,000 clients mainly with depression, mood, and anxiety disorders were treated by 84 therapists who were licensed at the masters degree or higher. The clients completed the Outcome Rating Scale (a measure of symptom outcome) prior to each session, and the Session Rating Scale (a measure of the therapeutic alliance) after each session. Therapists received two days of training on how to use these measures and on the continuous feedback they were provided in order to improve their treatment of clients and their outcomes. Outcomes from this public behavioural health service were compared to previous large studies in publicly funded settings that implemented specific evidence-based treatments. The findings were similar, with about 42% showing reliable pre- to post-treatment improvement. The results of implementing continuous outcome monitoring with feedback for depressive symptoms were also large and positive (d = 1.34). These effects were similar to benchmarks established in randomized controlled trials of specific psychotherapies.

Practice Implications

Continuous outcome feedback enables therapists to identify clients who are not benefiting
from a given treatment, so that clinicians may collaboratively design different interventions or change their interpersonal stances. The inclusion of outcome monitoring and feedback in this publicly funded psychotherapy system, resulted in outcomes that were: better than what is often seen in such public service settings, equivalent to those public systems that implemented specific evidence based treatments, and similar to those reported in highly controlled randomized trials. The authors concluded that adding routine outcome monitoring and feedback is a viable alternative to transporting specific evidence based treatments to publicly funded psychotherapy programs. The measures used in this study are available free for individuals to use at: betteroutcomesnow.com.

Author email: jeff.reese@uky.edu

Miller, S. D., Hubble, M. A., Chow, D., & Seidel, J. (2015). Beyond measures and monitoring: Realizing the potential of feedback-informed treatment. Psychotherapy, 52(4), 449-457.

Progress monitoring is the process of repeatedly assessing client functioning with validated measures and providing feedback to therapists. The feedback is designed to identify problems with the therapeutic relationship or with client deterioration by comparing client progress to similar clients. This allows therapists to change what they are doing, renegotiate aspects of therapy, or directly address the issues. Research is clear that progress monitoring significantly increases the proportion of clients who improve, reduces drop outs by a third, shortens the length of therapy, and reduces costs. Yet the research also indicates that only 12% of psychologists are using progress monitoring in their practice. If progress monitoring is so useful, then why aren`t more therapists using it? In this review, Miller and colleagues discuss some of the barriers and problems with using or adopting progress monitoring in clinical practice. They describe that even in the most favorable circumstances, it takes about two decades for new treatments to be integrated into routine care. Another issue is that recent surveys indicate that only about 33% of psychologists and 66% of training directors are aware of progress monitoring. Even for those who are aware, a common barrier might be cost and time to implement the procedures. Despite the brevity and low cost of the tools, like the PCOMS, they all place an additional burden on clinicians’ busy schedule. There is also the issue of staff turnover. As staff come and go, organizations may lose those who lead, train, and support the use of progress monitoring. Probably the biggest barrier is skepticism on the part of clinicians who might see the tools as too superficial, or who might be concerned that repeated measurement may somehow negatively affect the therapeutic relationship. However, research indicates that clients generally report positive experiences – they like being a more integral part of the assessment process, and they appreciate the ability to track their own progress. Finally, whereas clinicians may use progress monitoring to improve clinical decision-making, administrators may see it as a means of conducting performance reviews.

Practice Implications

In most health care fields, it can take 20 years for an innovation to make it into routine practice. That might be the case for progress monitoring. More clinicians need to know about it, be trained in its use, and see for themselves that the information is valid, of high quality, and that it can supplement their work in identifying clients who are not doing well. In particular, progress monitoring may be a means of enhancing the therapeutic alliance as it provides therapists and clients a vehicle to discuss how the therapy is going, what needs focus, and what to do if things go awry. Organizations need to treat progress monitoring as a means of helping therapists to improve their skills, and not as a means of auditing performance. Therapists need quality information upon which to make sound clinical decisions, and progress monitoring is one way of receiving this information.