Blog

Muir, H. J., Coyne, A. E., Morrison, N. R., Boswell, J. F., & Constantino, M. J. (2019). Ethical implications of routine outcomes monitoring for patients, psychotherapists, and mental health care systems. Psychotherapy, 56(4), 459–469.

On average, psychotherapy is effective for a variety of disorders, however about two thirds of patients in regular clinical practice do not improve or recover. Even more problematic is the 5% to 10% of patients who get worse during psychotherapy. Research has repeatedly demonstrated that psychotherapists are not capable of identifying patients who get worse, and therapists regularly over-estimate their effectiveness. This is not surprising because the information therapists need to make these determinations about patients is complex. Therapists, like most humans, have a difficult time integrating complex information to make nuanced decisions about how to act. Therapists need help in the form of reliable and accurate information to make complex clinical decisions. One approach to addressing these concerns is to use routine outcome monitoring (ROM), which involves assessing patient progress by reliable means as patients progress through therapy. ROM also involves feeding that information back to therapists on a regular basis so that they can make the best determinations. In this narrative review, Muir and colleagues discuss the ethical implications of the use of ROM given the research support. The research indicates that ROM feedback leads better outcomes for patients than treatment as usual, and most importantly ROM feedback to therapists reduces patient deterioration in half. Qualitative research also finds that patients generally view ROM feedback as valuable, and patients prefer to use it. Given these findings, implementing ROM represents one way to help therapists to meet ethical aspirations of beneficence, and to fulfill ethical obligations of forestalling harm. The research also indicates that therapists differ in their outcomes. One large study in clinical practices indicated that above average therapists were two times more effective than other clinicians. Individual psychotherapists may not be aware of their effectiveness and so do not know if they need more training or need to focus on certain types of patients for whom they are particularly effective. Ethically, ROM may be one means by which therapists can assess the boundaries of and areas competence, and those areas for which they may need more training and supervision.

Practice Implications

ROM may be a means for psychotherapists to practice ethically by knowing their patients better and by knowing themselves better. Basing one’s perception of one’s own competence fulfills the ethical requirement to ground clinical practice within the scientific knowledge of the field. Consistent and reliable information may allow therapists to know their own general effectiveness, and also to know for which patients they are more or less effective. This might result in therapists focusing their practices on those patient problems for which they are effective, or to seek further training and supervision in those areas in which they can improve.

Cuijpers, P. (2020) Measuring success in the treatment of depression: What is most important to patients? Expert Review of Neurotherapeutics, 20, 123-125.

There is lots of evidence now that psychotherapies of various types are efficacious for the treatment of depression. Psychotherapy trials focus largely on depressive symptoms, and define major depression according to psychiatric diagnostic manuals. However, the diagnosis of major depression, for example, is not a unitary construct. That is, it is simply a collection of symptoms and signs that are purported to make up a category of disorder. In fact, people with major depression are quite varied on a whole range of things, like severity, coping style, motivation, attachment style, personality, and extent of comorbidity with other diagnoses. This means that many psychotherapy studies may be focusing on patient outcomes (i.e., reduction of depressive symptoms) that may or may not be important to patients. In this paper, Cuijpers reviews the literature on what patients want from psychotherapy. He found that while symptom reduction was important to patients with depressive disorders, it was not the only outcome they wanted from psychotherapy. Patients also want to have a more fulfilling lives, to return to productive work, to solve conflicts with close loved ones, to learn to live with a chronic disability or disease, to learn to handle the effects of trauma, and other quality of life issues. Fortunately, some studies do report the effects of psychotherapy on quality of life, social functioning, anxiety, hopelessness, and interpersonal problems. However, even these studies treat such outcomes as if they were uniformly important to all patients in the study. Very few studies take a personalized approach to patient outcomes, in which the outcomes of interest are those determined by each patient specific to their own circumstances and wishes.

Practice Implications

Psychotherapists who practice from an evidence-informed perspective often try to measure outcomes in their own practices using reliable measurements. However, many of these measurements may be too general for any specific patient, or they may represent outcomes that do not align with what the individual patient wants. Practicing clinicians who assess outcomes in their own practices, may want to consider supplementing standard symptom outcome measures with more personalized assessments for patients.

Solstad, S.M., Castonguay, L.G., & Moltu, C. (2018). Patients’ experiences with routine outcome monitoring and clinical feedback systems: A systematic review and synthesis of qualitative empirical literature. Psychotherapy Research. doi=10.1080/10503307.2017.1326645.

Routine outcome monitoring or progress monitoring involves assessing client outcomes or the therapeutic alliance on a weekly basis in psychotherapy, and then giving feedback to the therapist about how the client is doing relative to the previous week and relative to similar clients. Research on progress monitoring indicates that it improves outcomes and it reduces by half the number of clients who might get worse. Despite its benefits, many therapists are not aware of progress monitoring or are reluctant to use the procedure. Some have expressed concerns that progress monitoring could interfere with the therapeutic relationship. However, very few studies have asked clients about their experiences of progress monitoring. In this synthesis of qualitative studies, Solstad and colleagues reviewed 16 studies in which clients were interviewed about their experiences of progress monitoring. The authors used a procedure in which they identified common themes across the studies and categorized client statements within those themes (e.g., thematic analysis). The authors were interested in identifying what were the hindering and helpful processes in clients’ experiences of their therapists’ use of progress monitoring. Four main themes emerged from the research. First, some clients voiced suspicion of how the progress monitoring data was going to be used and why the procedure was implemented. That is, clients sometimes felt that filling out questionnaires weekly was mainly a bureaucratic exercise, or possibly a means to justify reducing services. Second, some clients felt the questionnaires were not flexible enough to capture the complexity of mental health and of client concerns. The questionnaires often focused on symptoms, but clients were also interested in the therapeutic relationship, family, and social functioning. Third, some clients wanted to be more fully informed about the rationale for progress monitoring so that they could feel more empowered to define their own outcomes and treatment plans. Fourth, some clients found progress monitoring to help them to see graphically their own progress, to become more engaged in treatment planning, and to participate in collaborative and reflective discussions with their therapist.

Practice Implications

If psychotherapists choose to use progress monitoring in their practices, they should make sure that clients know what the data will be used for and that the exercise is not just a bureaucratic process. The practice of outcome or progress monitoring can be used to stimulate reflection not only in the therapist but also in the client. Reviewing the client data together might enhance conversations about therapy, the therapeutic relationship, and help to establish realistic goals for therapy. Therapists might consider not only measuring symptom progress repeatedly, but also measuring the working alliance on a regular basis.

Duncan, B. L., & Reese, R. J. (2015). The Partners for Change Outcome Management System (PCOMS): Revisiting the client’s frame of reference. Psychotherapy, 52(4), 391-401.

Generally, psychotherapy is effective for a wide variety of disorders, but regardless, many clients do not benefit. Further, the research shows that some therapists are more effective than others, but therapists tend to grossly over-estimate their effectiveness. In one large survey, therapists reported that their outcomes were better than 75% of their peers, no therapist rated themselves as below average, and therapists tended to over-estimate their effectiveness and under-estimate client deterioration. One way to evaluate patient outcomes and processes is to engage in progress monitoring and feedback. This involves repeated brief assessments of client outcomes followed by real-time feedback to therapists to gauge client progress and signal potential problems. Several such systems exist including the Outcome Questionnaire-45.2 and the Partners for Change Outcome Management System (PCOMS). The PCOMS is made up of the Outcome Rating Scale (ORS) and the Session Rating Scale (SRS). The ORS measures distress in 3 atheoretical domains (personal, family, social) not based on diagnosis. The SRS is a measure of therapeutic alliance. Both the ORS and SRS are very short 4-item scales that can be administered before (ORS) and after (SRS) each session of therapy. In this paper, Duncan and Reese review the research supporting the use of the PCOMS. A meta analysis found that clients whose therapists received feedback with the PCOMS were 3.5 times more likely to experience reliable change and had less than half the chance of experiencing deterioration. Five randomized controlled trials demonstrated the advantage of the PCOMS over treatment as usual, including by reducing drop outs and achieving reliable change in fewer sessions.

Practice Implications

A lot of research has demonstrated that most therapists over-estimate their effectiveness and that many are not able to identify clients who are getting worse. It is time for therapists to acknowledge this positive bias of their effectiveness and their need for quality information in order to make good clinical decisions. Progress monitoring and feedback systems are one means by which therapists can receive quality information. The repeated use of the PCOMS for example, can help to identify when clients begin to deteriorate and/or when problems emerge with the therapeutic alliance. Being able to identify these issues early may allow therapists to act quickly to avert client deterioration or drop out.

Miller, S. D., Hubble, M. A., Chow, D., & Seidel, J. (2015). Beyond measures and monitoring: Realizing the potential of feedback-informed treatment. Psychotherapy, 52(4), 449-457.

Progress monitoring is the process of repeatedly assessing client functioning with validated measures and providing feedback to therapists. The feedback is designed to identify problems with the therapeutic relationship or with client deterioration by comparing client progress to similar clients. This allows therapists to change what they are doing, renegotiate aspects of therapy, or directly address the issues. Research is clear that progress monitoring significantly increases the proportion of clients who improve, reduces drop outs by a third, shortens the length of therapy, and reduces costs. Yet the research also indicates that only 12% of psychologists are using progress monitoring in their practice. If progress monitoring is so useful, then why aren`t more therapists using it? In this review, Miller and colleagues discuss some of the barriers and problems with using or adopting progress monitoring in clinical practice. They describe that even in the most favorable circumstances, it takes about two decades for new treatments to be integrated into routine care. Another issue is that recent surveys indicate that only about 33% of psychologists and 66% of training directors are aware of progress monitoring. Even for those who are aware, a common barrier might be cost and time to implement the procedures. Despite the brevity and low cost of the tools, like the PCOMS, they all place an additional burden on clinicians’ busy schedule. There is also the issue of staff turnover. As staff come and go, organizations may lose those who lead, train, and support the use of progress monitoring. Probably the biggest barrier is skepticism on the part of clinicians who might see the tools as too superficial, or who might be concerned that repeated measurement may somehow negatively affect the therapeutic relationship. However, research indicates that clients generally report positive experiences – they like being a more integral part of the assessment process, and they appreciate the ability to track their own progress. Finally, whereas clinicians may use progress monitoring to improve clinical decision-making, administrators may see it as a means of conducting performance reviews.

Practice Implications

In most health care fields, it can take 20 years for an innovation to make it into routine practice. That might be the case for progress monitoring. More clinicians need to know about it, be trained in its use, and see for themselves that the information is valid, of high quality, and that it can supplement their work in identifying clients who are not doing well. In particular, progress monitoring may be a means of enhancing the therapeutic alliance as it provides therapists and clients a vehicle to discuss how the therapy is going, what needs focus, and what to do if things go awry. Organizations need to treat progress monitoring as a means of helping therapists to improve their skills, and not as a means of auditing performance. Therapists need quality information upon which to make sound clinical decisions, and progress monitoring is one way of receiving this information.

Murphy, R. & Hutton, P. (2017). Therapist variability, patient reported therapeutic alliance, and clinical outcomes in adolescents undergoing mental health treatment: A systematic review and meta-analysis. The Journal of Child Psychology and Psychiatry, doi:10.1111/jcpp.12767.

The therapeutic alliance refers to the affective bond between therapist and client, and their agreement on the tasks and goals of therapy. The alliance is a well-known predictor of outcomes in adult psychotherapy with a mean alliance-outcome correlation of r = .28. Less is known about the role of the alliance in the treatment of adolescents. Some reviews indicate that the alliance-outcome relationship in children and adolescents is weaker than observed among adults, but these reviews may have been flawed since they included both children and adolescents in the same review, and the number of studies they reviewed was small. A large rigorous systematic review of adolescents’ perceptions of the alliance can provide insight into their experience of psychological treatment and inform routine mental health practice. In their meta analysis, Murphy and Hutton reviewed studies of clinical samples of adolescents between the age of 12 – 19 who received psychological treatment. The authors made sure that the measures of alliance and outcomes were reliable, they excluded studies of those with medical and neurocognitive problems, and included only studies with adolescents (i.e., excluding studies with primarily children). Twenty-seven studies with almost 3,000 participants were included. Main presenting problems of adolescent patients were: substance use, eating disorders, behavioral difficulties, and a range of mood and anxiety disorders. The mean weighted effect size of the alliance-outcome relationship among studies of psychological treatment of adolescents was r = .29 (95% CI: 0.21, 0.37; p < .001) indicating a moderate effect.

Practice Implication

This is the largest meta analysis of the alliance-outcome relationship in the psychological treatment of adolescents with mental health problems. The alliance was moderately associated with outcomes, and so therapeutic alliance may be a reliable predictor of clinical progress in the treatment of adolescents. The findings suggest that those working with adolescents should routinely assess the alliance after each session in order to evaluate if they need to address relational barriers to positive outcomes. For example, if the alliance markedly declines from one session to the next, then clinicians should address potential problems in their relationship with the adolescent client, renegotiate goals, or renegotiate the tasks of therapy.