Imel, Z. E., Pace, B. T., Soma, C. S., Tanana, M., Hirsch, T., Gibson, J., Georgiou, P., Narayanan, S., & Atkins, D. C. (2019). Design feasibility of an automated, machine-learning based feedback system for motivational interviewing. Psychotherapy, 56(2), 318–328.
I do not mean to conjure up the image of a dystopian future, but I could not resist the pithy title for this blog. Ideally, psychotherapists in training or those who seek professional development would receive high quality accurate feedback about their behavior (e.g., about interpersonal skills, empathy, vocal tone, body language) and competence (e.g., regarding specific interventions) in real time. This would allow psychotherapists and trainees can make fine-tuned adjustments to their behaviors and interventions that match or complement the specific patient with which they are working. But, given the current technology, this is impossible. Instead psychotherapy training and feedback to practicing clinicians is slow, cumbersome, and imprecise. Current supervision and consultation practices rely on giving feedback based on the clinician’s verbal case report or, at best, based on viewing video recordings. There are systems that provide feedback on patient outcomes that may alert psychotherapists to something going amiss in for the patient. But such feedback occurs post-session, is based on patient self-report, and does not inform immediate in-session therapist behaviors. In this study, Imel and colleagues evaluated an initial proof of concept of an automated feedback system that generated quality metrics about specific therapist interventions and about therapist skills like empathy. They used computer technology based on natural language processing to take conversational data from video of psychotherapy sessions in order to answer questions like: “what did the therapist and patient talk about during the session?”, “how empathic was the therapist?”, and “how often did the therapist use reflections versus closed questions in the session?” The authors developed a machine learning tool to transcribe, code, and rapidly generate feedback to 21 experienced and novice therapists who recorded a 10-minute session with a standardized patient (a standardized patient is an actor who loosely follows a script). The machine learning technology was accurate at defining or coding a “closed question” by a therapist (e.g., a question with a yes/no answer; inter rater agreement with a human coder ICC = .80), but not as accurate at defining or coding a therapist empathic statement (inter rater agreement with a human coder ICC = .23). The system provided immediate feedback the therapists about their behaviors during the session using graphics and text (fidelity to specific interventions, counseling style, empathy, percent open/closed questions, percent reflections). All therapists rated the tool as “easy to use”, 86% strongly agreed that the feedback was representative of their performance, 90% agreed that if the tool was available, they would use it in their clinical practice.
Typically, professional consultation or supervision involves a consultant giving the therapist feedback based on imprecise descriptions of events in a therapy session that occurred at some point in the recent past. This method of training and consultation in psychotherapy has not changed much in the past 60 years. One key drawback of current methods of training and consultation is that they do not make use of real-time feedback to help therapist adjust behaviors to the specific patient or context. It is possible that in the near future with rapid advances in artificial intelligence and machine learning a therapist will be able to finish a session with a patient and receive an immediate feedback report about the previous hour. The feedback might include metrics on empathy, the percent of questions vs reflections, competence in specific interventions, among other personalize ratings. This future might also have novice trainees receive immediate real-time in-session feedback about behaviors of interest that need to be adjusted, or for which more training is necessary. For some, this might be a vision of a dystopian future, for others it may represent a way forward in which therapists achieve more refined skills and better patient outcomes.