Blog

Castonguay, L.G., Barkham, M., Youn, S.J., & Page, A. (2021). Practice-based evidence: Findings from routine clinical settings. In Barkham, W. Lutz, and L.G. Castonguay (Eds.) Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change (7^th ed.). Wiley. Chapter 6.

Practice-based evidence refers to research that is conducted as part of routine clinical practice. Often these studies do not impose strict research conditions like randomization of patients, and so they produce findings that are more relevant to psychotherapy as practiced real-world. In studies from practice-research networks, clinicians are often involved in the design and implementation of the study. Our Psychotherapy Practice-Research Network (PPRNet) is an example of a collaboration between researchers and clinicians to produce practice-based evidence. In a large survey, we found that most clinicians regardless of theoretical orientation wanted more research on the therapeutic relationship and on professional development. And so PPRNet developed a research program on training psychotherapists to identify and repair therapeutic alliance ruptures and microaggressions. In this chapter, Castonguay and colleagues review some of the key findings from practice research networks. First, White patients report better outcomes than Black patients in routine care, and these differences were linked back to the clinicians. That is, therapists varied in their effectiveness with racial and ethnic minority patients. Second, patients benefit when clinicians monitor the therapeutic alliance and outcomes on a session-to-session basis using reliable and valid measures. Noticing when a patient’s ratings of the alliance decline from one session to the next, might indicate a problem in the therapeutic relationship. Third, when monitoring the alliance, therapists should also rate the alliance. If a therapist’s ratings of the alliance are higher than their patients, then this may be a sign that the therapist is not paying attention to problems in the alliance. Fourth, developing and maintaining a strong therapeutic alliance likely will improve patient outcomes. Fifth, a reliable and valid assessment of client’s past and current interpersonal difficulties is likely to improve a therapist’s treatment plan for that patient.

Practice Implications

Doing research in clinical practices is not as well controlled as clinical trials research. But practice-based evidence is more relevant to how psychotherapy is done in the real world with real patients. The research is not so clear about why some therapists are more effective with racial and ethnic minority (REM) patients. However, complementary research suggests that some therapists who have a previously high level of multicultural orientation (cultural humility, open to conversations about culture, and cultural comfort) are more effective with REM patients. Also monitoring the therapeutic alliance with a valid scale on a session-to-session basis leads to better outcomes. Such monitoring will alert the therapist to resolve an alliance rupture if a patient’s ratings decrease from one session to the next. If therapists also rate the alliance and find that their scores are higher than their patient’s, then this may alert the therapist to a potential problem. Finally, knowing if a patient has current and past interpersonal problems can inform a therapist to focus on how those problems affect current symptoms and to talk about how those problems manifest themselves in the therapeutic relationship.

Miller, S. D., Hubble, M. A., Chow, D., & Seidel, J. (2015). Beyond measures and monitoring: Realizing the potential of feedback-informed treatment. Psychotherapy, 52(4), 449-457.

Progress monitoring is the process of repeatedly assessing client functioning with validated measures and providing feedback to therapists. The feedback is designed to identify problems with the therapeutic relationship or with client deterioration by comparing client progress to similar clients. This allows therapists to change what they are doing, renegotiate aspects of therapy, or directly address the issues. Research is clear that progress monitoring significantly increases the proportion of clients who improve, reduces drop outs by a third, shortens the length of therapy, and reduces costs. Yet the research also indicates that only 12% of psychologists are using progress monitoring in their practice. If progress monitoring is so useful, then why aren`t more therapists using it? In this review, Miller and colleagues discuss some of the barriers and problems with using or adopting progress monitoring in clinical practice. They describe that even in the most favorable circumstances, it takes about two decades for new treatments to be integrated into routine care. Another issue is that recent surveys indicate that only about 33% of psychologists and 66% of training directors are aware of progress monitoring. Even for those who are aware, a common barrier might be cost and time to implement the procedures. Despite the brevity and low cost of the tools, like the PCOMS, they all place an additional burden on clinicians’ busy schedule. There is also the issue of staff turnover. As staff come and go, organizations may lose those who lead, train, and support the use of progress monitoring. Probably the biggest barrier is skepticism on the part of clinicians who might see the tools as too superficial, or who might be concerned that repeated measurement may somehow negatively affect the therapeutic relationship. However, research indicates that clients generally report positive experiences – they like being a more integral part of the assessment process, and they appreciate the ability to track their own progress. Finally, whereas clinicians may use progress monitoring to improve clinical decision-making, administrators may see it as a means of conducting performance reviews.

Practice Implications

In most health care fields, it can take 20 years for an innovation to make it into routine practice. That might be the case for progress monitoring. More clinicians need to know about it, be trained in its use, and see for themselves that the information is valid, of high quality, and that it can supplement their work in identifying clients who are not doing well. In particular, progress monitoring may be a means of enhancing the therapeutic alliance as it provides therapists and clients a vehicle to discuss how the therapy is going, what needs focus, and what to do if things go awry. Organizations need to treat progress monitoring as a means of helping therapists to improve their skills, and not as a means of auditing performance. Therapists need quality information upon which to make sound clinical decisions, and progress monitoring is one way of receiving this information.

The Great Psychotherapy Debate: Since in April, 2015 I review parts of The Great Psychotherapy Debate (Wampold & Imel, 2015) in the PPRNet Blog. This is the second edition of a landmark, and sometimes controversial, book that surveys the evidence for what makes psychotherapy work. You can view parts of the book in Google Books.

In this chapter, Wampold and Imel contrast the Medical Model to a Contextual Model of psychotherapy. The Medical Model argues that there is a biological explanation for a disorder and that the basis for treatment is to address the biological system causing the disorder. Understanding the cause of the disorder (e.g., excess stomach acid) leads to an explanation of the mechanism of the disorder and of change (e.g., reduce stomach acid), which in turn leads to specific interventions (e.g., administer an antacid). Key to this model is specificity: that is, a specific disorder can be explained by a specific mechanism, and a specific treatment based on this explanation will result in alleviation of the disorder. In psychotherapy, for example, one could argue that PTSD symptoms are caused by maladaptive avoidance of traumatic memories, which can be successfully treated by repeated exposure to the traumatic memories to reduce symptoms. Exposure is the specific intervention indicated by the purported mechanism or cause of PTSD. By contrast, Wampold and Imel discuss a Contextual Model of psychotherapy. This model combines the elements of common factors in psychotherapy (e.g., the bond, real relationship, therapeutic alliance) with specific factors of interventions. The Contextual Model indicates that people are fundamentally social animals that require relationships with others to survive and to heal. That is, the initial therapeutic bond between client and therapist is the basis of psychotherapeutic practice. There are three elements to the Contextual Model. (1) The real relationship – which is based on genuineness [openness and honesty] and therapist empathy. Both genuineness and empathy are related to client outcomes. (2) Expectations – which, like the placebo effect, increases one’s readiness to benefit from treatment, is related to greater hope of improvement, and belief in the treatment. The placebo effect is associated with improvements in a pharmacological treatments of depression, and client expectations are related to psychotherapy outcomes. (3) Specific ingredients – as indicated in the Medical Model refers to psychotherapeutic interventions based on a psychological theory of a disorder. But unlike in the Medical Model, the Contextual Model sees the key element of specific ingredients as the agreement between client and therapist on the explanation for the disorder and on the treatment. In other words, a therapeutic alliance in part depends on clients and therapists agreeing on the specific tasks and goals of therapy.

Practice Implications

Patient outcomes are enhanced by a positive therapeutic bond, genuineness and congruence in the way a therapist interacts with a client, and a therapeutic alliance in which clients and therapists agree on tasks and goals of therapy. Typically, these occur within a context in which therapists use some specific techniques of therapy to which he or she feels an allegiance. It is also important that the client agree with the explanation for their disorder provided by this specific treatment model and that the client expects that this treatment will provide them with relief. Although common and specific factors of psychotherapy have been seen as having entirely separate roles in client outcomes, Wampold and Imel argue that within the Contextual Model, they are intimately tied to each other.

The Great Psychotherapy Debate: Since in April, 2015 I review parts of The Great Psychotherapy Debate (Wampold & Imel, 2015) in the PPRNet Blog. This is the second edition of a landmark, and sometimes controversial, book that surveys the evidence for what makes psychotherapy work. You can view parts of the book in Google Books.

Wampold, B.E. & Imel, Z.E. (2015). The great psychotherapy debate: The evidence for what makes psychotherapy work (2^nd edition). New York: Routledge.

Why We Should Care About Allegiance Effects in Psychotherapy Research

Allegiance in psychotherapy refers to the degree to which a researcher or therapist believes that the therapy they are studying or delivering is effective. Clients have an expectation that therapists have an explanation for their disorder and that the therapy used to address that explanation will lead to improvements. On their part, psychotherapists choose a therapeutic approach that is consistent with their understanding of psychological distress. Wampold and Imel argue that therapist allegiance is a common factor across therapies that contributes to good patient outcomes. Although allegiance is an important therapeutic factor, it complicates the conduct of psychotherapy research. In a trial comparing two treatments, for example, researchers and therapists tend to be affiliated with one of the treatments and so they believe in the effectiveness of their treatment. They often do not feel the same way about the comparison treatment, or they may desire that their preferred treatment be more effective than the comparison. In medication trials, this allegiance effect can be controlled by a double blind placebo controlled design in which both therapist and patient are not aware of who is receiving which active medication, or who is receiving a placebo. It is impossible to blind therapists in psychotherapy trials – therapists have to know what treatment they are providing. When doing a meta analytic review of psychotherapy trials, it is possible and relatively easy to identify the allegiance of the researchers in a particular study by looking at their past publications, and by reading what they say about the therapies they are comparing. Often, the developer of a treatment manual is a co-author of the trial. Wampold and Imel review several meta analyses that assess the allegiance effects. In three meta analyses from published in 1980, 1999, and 2013 the correlation between ratings of researcher allegiance and effects of psychotherapy on patient outcomes ranged from moderate to large (r = .26 to r = .85). One interesting meta analysis illustrates the magnitude of this effect. The reviewers looked at 69 studies on self statement monitoring (SSM), a type of cognitive therapy developed by Meichenbaum. The average effect of SSM compared to controls in all studies was d = .53 to d = .74, which is moderate. However, effect sizes found in the studies co-authored by Meichenbaum were nearly twice as large, d = d = 1.23. Being a co-investigator in a study of a therapy that one develops, apparently doubles the effect of the treatment on patient outcomes.

Practice Implications

Therapist allegiance to a treatment is important to the effectiveness of the treatment in that therapist allegiance increases the therapist’s confidence in the treatment’s effectiveness and increases a patient’s expectation of getting better. However, when interpreting psychotherapy trials, especially those that pit one type of therapy against another, it is important to keep in mind the researchers’ allegiance. It is rare to see trials that compare two interventions in which the research team is made of up proponents of the two interventions. However such trials are important and necessary.

Castonguay, L., Barkham, M., Lutz, W., & McAleavey, A. (2013). Practice-oriented research: Approaches and applications. In M.E. Lambert (Ed.), Bergin and Garfield’s Handbook of Psychotherapy and Behavior Change, 6th Edition (pp. 85-133). New York: Wiley.

In this chapter of the Handbook, Castonguay and colleagues (2013) review research methods and results associated with practice research networks (PRN). There is substantial evidence to show that psychotherapists often are not influenced by research findings when they prepare their case formulations and conduct interventions. As a result, clients may not be benefiting fully from nearly 60 years of research in psychotherapy methods and processes. There may be several explanations for this divide. Clinicians may perceive psychotherapy research, especially the emphasis on empirically supported treatments, as limited in its clinical relevance. Researchers may pay limited attention to concerns of clinicians when developing research strategies and treatment manuals. The end result is that clinicians feel disenfranchised from the research field, and therefore unaffected by the findings. Clinicians may pay more attention to psychotherapy research if they were more involved and “owned” the research and findings. One solution is to develop PRN based on a partnership of practitioners and researchers in which they collaborate on all aspects of a study; i.e., generation of ideas, implementation, and publication. Castonguay and colleagues (2013) report on the research generated by several PRNs in the U.S. The American Psychiatric Institute for Research and Education’s PRN (APIRE-PRN) conducted several studies including: one study that found that compared to White patients, African Americans were less likely to be prescribed second generation antipsychotic medications, which are considered to be the treatment of choice by psychiatrists; and a second study that reported that presence of a personality disorder, low Global Assessment of Functioning scores, and seeing a psychiatrist at a discounted fee was associated with treatment non-compliance. The Pennsylvania Psychological Association PRN (PPA-PRN) conducted several studies, including one study that found that better patient outcomes were associated with: higher expectancy for change among clients, lower client interpersonal problems, greater number of therapy sessions, and lower therapist case load. A second PPA-PRN study that I reported in my August 2013 Blog found that therapist efforts to foster client awareness of their emotions, thoughts, and behaviors were perceived as particularly helpful by both clients and therapists. Finally, the National Drug Abuse Treatment Clinical Trials Network (CTN) conducted several studies with the intent of bringing drug abuse researchers into the real world and creating opportunities for clinicians to participate in research. This network completed over 50 trials. For example, in one trial, researchers found evidence for better retention, treatment engagement, and family functioning for brief strategic family therapy compared to treatment as usual.

Practice Implications

A qualitative study reported by Castonguay and colleagues (2010) indicated that clinician involvement in PRN research fostered new learning as well as a sense of community with other professionals with shared goals. Therapists also indicated that their clients perceived their research participation as intrinsically meaningful and an opportunity to contribute to scientific knowledge. However, participating in a PRN had its challenges as well. Clinicians had to remember detailed procedures, at times practitioners had to depart from their clinical routine, and clinicians had to find time to complete questionnaires and other procedures. Castonguay and colleagues (2010) recommended that practice based research procedures in a PRN remain simple and clear, that clinicians have to have ready access to research staff for consultations, and that incentives have to be built in for clients and clinicians to participate. Studies in which research goals and clinical goals are indistinguishable are most likely to succeed.