Blog

Truijens, F., Zühlke‐van Hulzen, L., & Vanheule, S. (2018). To manualize, or not to manualize: Is that still the question? A systematic review of empirical evidence for manual superiority in psychological treatment. Journal of Clinical Psychology. Advance online publication. 

In 2010 Webb and colleagues published a meta-analysis in which they showed that the association between adherence to a psychotherapy manual and treatment outcome was close to zero. The same was true for therapist competence in delivering the manualized psychotherapy – almost no relationship to client outcome. Psychotherapy manuals typically specify the theoretical basis for an intervention, the number and sequencing of treatment sessions, the content and objective of sessions, and the procedures of each session. National institutes in the US and the UK have promoted manuals as a means to define what is evidence-based psychotherapy. By doing so these institutes assume that psychotherapy that is manualized is more effective that non-manualized treatment. However, detractors have argued that: (1) strict adherence to manuals may reduce therapists’ ability to individualize treatment to client needs and characteristics; (2) manuals are often designed for single disorders but clients tend to have many comorbid conditions; and (3) it is impossible for clinicians to gain competence in all different manuals for the various client conditions they may encounter. In this systematic review, Truijens and colleagues ask: does the use of manuals increase therapy effectiveness? To answer this question they conducted three different systematic reviews. First, they reviewed six studies that directly compared manualized versus non-manualized versions of a psychotherapy within the same study. One study showed manuals were superior, three showed no difference, and two studies showed that non-manualized therapies were more effective. Second, they reviewed eight meta-analyses that compared the pre- to post-treatment effect sizes of manualized therapies and of non-manualized therapies versus no-treatment control conditions. Three meta-analyses concluded that manualized therapies were superior, four meta-analyses did not find differences, and one observed non-manualized treatments to be superior. Third, the authors reviewed 15 additional studies to those reviewed by Webb and colleagues in their original meta-analysis. Overall, Truijens found similar results that support the conclusion that the level of adherence to psychotherapy manuals is not substantially related to better treatment outcomes.

Practice Implications

Although treatment manuals may be helpful for training purposes and to ensure validity in psychotherapy research, there is actually little consistent evidence that adhering to a manual results in better client outcomes. Some have argued that rigid adherence to a treatment manual can be harmful to clients. Therapists may need to take a flexible stance when applying research-supported therapeutic principles and interventions. Such a stance adjusts therapy to take into account client characteristics like level of resistance, coping style, attachment style, and others. Truly evidence-informed approaches incorporate what we know about client characteristics, therapeutic relationship factors, and therapist factors to promote positive outcomes in psychotherapy clients.

Solstad, S.M., Castonguay, L.G., & Moltu, C. (2018). Patients’ experiences with routine outcome monitoring and clinical feedback systems: A systematic review and synthesis of qualitative empirical literature. Psychotherapy Research. doi=10.1080/10503307.2017.1326645.

Routine outcome monitoring or progress monitoring involves assessing client outcomes or the therapeutic alliance on a weekly basis in psychotherapy, and then giving feedback to the therapist about how the client is doing relative to the previous week and relative to similar clients. Research on progress monitoring indicates that it improves outcomes and it reduces by half the number of clients who might get worse. Despite its benefits, many therapists are not aware of progress monitoring or are reluctant to use the procedure. Some have expressed concerns that progress monitoring could interfere with the therapeutic relationship. However, very few studies have asked clients about their experiences of progress monitoring. In this synthesis of qualitative studies, Solstad and colleagues reviewed 16 studies in which clients were interviewed about their experiences of progress monitoring. The authors used a procedure in which they identified common themes across the studies and categorized client statements within those themes (e.g., thematic analysis). The authors were interested in identifying what were the hindering and helpful processes in clients’ experiences of their therapists’ use of progress monitoring. Four main themes emerged from the research. First, some clients voiced suspicion of how the progress monitoring data was going to be used and why the procedure was implemented. That is, clients sometimes felt that filling out questionnaires weekly was mainly a bureaucratic exercise, or possibly a means to justify reducing services. Second, some clients felt the questionnaires were not flexible enough to capture the complexity of mental health and of client concerns. The questionnaires often focused on symptoms, but clients were also interested in the therapeutic relationship, family, and social functioning. Third, some clients wanted to be more fully informed about the rationale for progress monitoring so that they could feel more empowered to define their own outcomes and treatment plans. Fourth, some clients found progress monitoring to help them to see graphically their own progress, to become more engaged in treatment planning, and to participate in collaborative and reflective discussions with their therapist.

Practice Implications

If psychotherapists choose to use progress monitoring in their practices, they should make sure that clients know what the data will be used for and that the exercise is not just a bureaucratic process. The practice of outcome or progress monitoring can be used to stimulate reflection not only in the therapist but also in the client. Reviewing the client data together might enhance conversations about therapy, the therapeutic relationship, and help to establish realistic goals for therapy. Therapists might consider not only measuring symptom progress repeatedly, but also measuring the working alliance on a regular basis.

Jennissen, S., Huber, J., Ehrenthal, J.C., Schauenburg, H., & Dinger, U. (2018). Association between insight and outcome of psychotherapy: Systematic review and meta-analysis. The American Journal of Psychiatry. Published Online: https://doi.org/10.1176/appi.ajp.2018.17080847

For many authors, one of the purported mechanisms of change in psychotherapy is insight. In fact, the utility of insight for clients with mental health problems was first proposed over 120 years ago by Freud and Breuer. Briefly, insight refers to higher levels of self-understanding that might result in fewer negative automatic reactions to stress and other challenges, more positive emotions, and greater flexibility in cognitive and interpersonal functioning. Although insight is a key factor in some psychodynamic models, it also plays a role in other forms of psychotherapy. Experiential psychotherapy emphasises gaining a new perspective through experiencing, and for CBT insight relates to becoming more aware of automatic thoughts. Jennissen and colleagues defined insight as patients understanding: the relationship between past and present experiences, their typical relationship patterns, and the associations between interpersonal challenges, emotional experiences, and psychological symptoms. In this study, Jennissen and colleagues conducted a systematic review and meta analysis of the insight-outcome relationship, that is the relationship between client self-understanding and symptom reduction. They reviewed studies of adults seeking psychological treatment including individual or group therapy. The predictor variable was an empirical measure of insight assessed during treatment but prior to when final outcomes were evaluated. The outcome was some reliable and empirical measure related to symptom improvement, pre- to post- treatment. The review turned up 22 studies that included over 1100 patients mostly with anxiety or depressive disorders who attended a median of 20 sessions of therapy. The overall effect size of the association between insight and outcome was r = 0.31 (95% CI=0.22–0.40, p < 0.05), which represents a medium effect. Moderator analyses found no effect of type of therapy or diagnosis on this mean effect size, though the power of these analyses was low.

Practice Implications

The magnitude of the association between insight and outcome is similar to the effects of other therapeutic factors such as the therapeutic alliance. When gaining insight, patients may achieve a greater self-understanding, which allows them to reduce distorted perceptions of themselves, and better integrate unpleasant experiences into their conscious life. Symptoms may be improved by self-understanding because of the greater sense of control and master that it provides, and by the new solutions and adaptive ways of living that become available to clients.

Author email: Simone.Jennissen@med.uni-heidelberg.de

Duncan, B. L., & Reese, R. J. (2015). The Partners for Change Outcome Management System (PCOMS): Revisiting the client’s frame of reference. Psychotherapy, 52(4), 391-401.

Generally, psychotherapy is effective for a wide variety of disorders, but regardless, many clients do not benefit. Further, the research shows that some therapists are more effective than others, but therapists tend to grossly over-estimate their effectiveness. In one large survey, therapists reported that their outcomes were better than 75% of their peers, no therapist rated themselves as below average, and therapists tended to over-estimate their effectiveness and under-estimate client deterioration. One way to evaluate patient outcomes and processes is to engage in progress monitoring and feedback. This involves repeated brief assessments of client outcomes followed by real-time feedback to therapists to gauge client progress and signal potential problems. Several such systems exist including the Outcome Questionnaire-45.2 and the Partners for Change Outcome Management System (PCOMS). The PCOMS is made up of the Outcome Rating Scale (ORS) and the Session Rating Scale (SRS). The ORS measures distress in 3 atheoretical domains (personal, family, social) not based on diagnosis. The SRS is a measure of therapeutic alliance. Both the ORS and SRS are very short 4-item scales that can be administered before (ORS) and after (SRS) each session of therapy. In this paper, Duncan and Reese review the research supporting the use of the PCOMS. A meta analysis found that clients whose therapists received feedback with the PCOMS were 3.5 times more likely to experience reliable change and had less than half the chance of experiencing deterioration. Five randomized controlled trials demonstrated the advantage of the PCOMS over treatment as usual, including by reducing drop outs and achieving reliable change in fewer sessions.

Practice Implications

A lot of research has demonstrated that most therapists over-estimate their effectiveness and that many are not able to identify clients who are getting worse. It is time for therapists to acknowledge this positive bias of their effectiveness and their need for quality information in order to make good clinical decisions. Progress monitoring and feedback systems are one means by which therapists can receive quality information. The repeated use of the PCOMS for example, can help to identify when clients begin to deteriorate and/or when problems emerge with the therapeutic alliance. Being able to identify these issues early may allow therapists to act quickly to avert client deterioration or drop out.

Miller, S. D., Hubble, M. A., Chow, D., & Seidel, J. (2015). Beyond measures and monitoring: Realizing the potential of feedback-informed treatment. Psychotherapy, 52(4), 449-457.

Progress monitoring is the process of repeatedly assessing client functioning with validated measures and providing feedback to therapists. The feedback is designed to identify problems with the therapeutic relationship or with client deterioration by comparing client progress to similar clients. This allows therapists to change what they are doing, renegotiate aspects of therapy, or directly address the issues. Research is clear that progress monitoring significantly increases the proportion of clients who improve, reduces drop outs by a third, shortens the length of therapy, and reduces costs. Yet the research also indicates that only 12% of psychologists are using progress monitoring in their practice. If progress monitoring is so useful, then why aren`t more therapists using it? In this review, Miller and colleagues discuss some of the barriers and problems with using or adopting progress monitoring in clinical practice. They describe that even in the most favorable circumstances, it takes about two decades for new treatments to be integrated into routine care. Another issue is that recent surveys indicate that only about 33% of psychologists and 66% of training directors are aware of progress monitoring. Even for those who are aware, a common barrier might be cost and time to implement the procedures. Despite the brevity and low cost of the tools, like the PCOMS, they all place an additional burden on clinicians’ busy schedule. There is also the issue of staff turnover. As staff come and go, organizations may lose those who lead, train, and support the use of progress monitoring. Probably the biggest barrier is skepticism on the part of clinicians who might see the tools as too superficial, or who might be concerned that repeated measurement may somehow negatively affect the therapeutic relationship. However, research indicates that clients generally report positive experiences – they like being a more integral part of the assessment process, and they appreciate the ability to track their own progress. Finally, whereas clinicians may use progress monitoring to improve clinical decision-making, administrators may see it as a means of conducting performance reviews.

Practice Implications

In most health care fields, it can take 20 years for an innovation to make it into routine practice. That might be the case for progress monitoring. More clinicians need to know about it, be trained in its use, and see for themselves that the information is valid, of high quality, and that it can supplement their work in identifying clients who are not doing well. In particular, progress monitoring may be a means of enhancing the therapeutic alliance as it provides therapists and clients a vehicle to discuss how the therapy is going, what needs focus, and what to do if things go awry. Organizations need to treat progress monitoring as a means of helping therapists to improve their skills, and not as a means of auditing performance. Therapists need quality information upon which to make sound clinical decisions, and progress monitoring is one way of receiving this information.

Podina, I. R., & Fodor, L. A. (2018). Critical review and meta-analysis of multicomponent behavioral e-health interventions for weight loss. Health Psychology, 37(6), 501-515.

Over 35% of Americans are overweight or obese, and this poses significant health-related challenges. Obesity likely contributes to heart disease, Type II diabetes, and some forms of cancer. Also, obesity is often co-morbid with mental health conditions including depression and binge-eating disorder. Practice guidelines list multicomponent behavioural interventions as state of the art treatment for weight loss. These include dietary counselling, increased physical activity, and behavioural methods to support behaviour change. However, such interventions often require direct in-person contact with a health or mental health professional, which can be expensive and create a barrier to accessing treatment for some. An option to increase access is to deliver the multicomponent behavioural intervention by internet or by another electronic format such as DVD. In this meta analysis, Podina and Fodor reviewed 47 randomized controlled studies representing over 1500 participants in which e-health interventions for weight loss in overweight or obese individuals were tested against in-person treatment or a control condition (no treatment or treatment as usual). E-health interventions were more effective than control conditions for weight loss outcomes at post-treatment, g = 0.34 (95% CI [0.24 to 0.44]). Similar results were found at follow-up. However, e-health interventions were significantly less effective than active in-person treatments, g = -0.31 (95% CI [-0.43 to -0.20]) for weight loss in overweight or obese individuals.

Practice Implications

E-health interventions (mostly internet delivered treatment) of multicomponent behavioral treatment for weight loss was more effective than no treatment or treatment as usual. However, e-health was significantly less effective than traditional face to face behavioral interventions to help people reduce their body weight. The authors raised concerns about the use of e-health interventions for weight loss as the first line treatment as the effects were small and the approach was less effective than in-person interventions.