Introduction
Training and continuous learning are foundational to operations within the 988 Suicide and Crisis Lifeline. New counselors are required to complete roleplays during onboarding, and all active counselors must participate in annual professional development. However, despite this strong emphasis on training, evaluating the effectiveness of different approaches has remained a challenge. A key barrier has been the limited availability of data: centers have the capacity to manually review only a small share of interactions, making it difficult to assess training impact at scale. Over the past 18 months, Volunteers of America Western Washington (VOAWW) – a leading 988 center in Washington State – has significantly enhanced their training and expanded quality assurance. The organization incorporated ReflexAI’s Prepare platform with realistic, AI-powered simulations and instant feedback. The organization also deployed ReflexAI’s Assure tool for quality assurance to understand insights on 100% of interactions. This brief examines the effects of increased simulation practice and feedback on crisis counselor performance, both overall and in critical areas of suicide risk screening and empathic communication.
Methods
The population included 174 crisis center counselors, all of whom completed at least 10 crisis calls during the period of investigation. 51 counselors (29%) completed a high number of simulations during onboarding and continuous learning, which was defined as at least eight. A further 32 counselors (19%) completed at least four simulations, which was defined as moderate. 91 counselors (52%) completed under four simulations, which was defined as low/none.
These counselors also completed 41,993 interactions supporting individuals in crisis over the time period of investigation, all of which were assessed for performance across a range of dimensions. Each dimension (e.g., suicide risk screening, empathetic communication) was evaluated for fidelity to clinical protocols at the high, medium, or low level using tailored AI models. Historically, only a small share of interactions would have been assessed by manual review from other team members, which can also lead to many intercoder consistency challenges.
Importantly, all simulation participants received feedback on their performance within each simulation, which was delivered immediately after completion. This feedback was provided in alignment with adult learning best practices that foster both the skills and confidence to engage in this work. As shared by Levi Van Dyke, Chief Behavioral Health Officer at VOAWW, “It’s essential to our mission that every counselor can understand their specific strengths and opportunities at every stage of their journey with VOAWW.”
Findings
Crisis counselors who utilized a high number of simulations with instant feedback met organizational standards on significantly more of their interactions. There is a discernible improvement between counselors who engaged with a high number of simulations compared with a moderate amount, and further improvement when comparing moderate engagement with low/no engagement.
Counselors who participated in a high number of simulations received the highest overall performance assessment in 17% more conversations than individuals with limited simulation and instant feedback experience. On the particular dimension of risk assessment, counselors who engaged in a high number of simulations met the highest performance standards in 18% more conversations. On the particular dimension of empathetic communication, counselors who engaged in a high number of simulations met the highest performance standard in 16% more conversations.
Discussion
The work of crisis counselors is challenging and multifaceted, requiring mastery of protocols such as a clinically-validated risk assessment, as well as items traditionally considered “soft skills” such as empathetic communication. Thousands of responders support millions of contacts every year, which is made possible by the tools and teams at hundreds of centers. Every counselor deserves access to learning tools that provide effective practice and feedback. And every caller deserves a counselor who is prepared to engage at the highest level.
This analysis shows the impact of one innovation and demonstrates the potential for efficient and impactful innovation in the years ahead. Variance is expected as requirements and capabilities change over time, but these findings show that utilizing realistic simulations and immediate feedback can improve interaction performance. As shared by Erica Olson, Training Coordinator at VOAWW, “It’s gratifying to show the impact of new tools on the skills of our counselors because they are truly on the front lines of supporting individuals in crisis across our state and the entire country.”
Additional resources
Ericsson, K. A. (2008).
Deliberate practice and acquisition of expert performance: A general overview.
https://onlinelibrary.wiley.com/doi/10.1111/j.1553-2712.2008.00227.x
Gould, M. S., Lake, A. M., Port, M. S., Kleinman, M., Hoyte-Badu, A. M., Rodriguez, C. L., Chowdhury, S. J., Galfalvy, H., & Goldstein, A. (2025).
National Suicide Prevention Lifeline (now 988 Suicide and Crisis Lifeline): Evaluation of crisis call outcomes for suicidal callers. Suicide & Life-Threatening Behavior, 55(3), e70020.
https://doi.org/10.1111/sltb.70020
Louie, R., Orney, I. H., Pacheco, J. P., Shah, R. J., Brunskill, E., & Yang, D. (2025).
Can LLM-simulated practice and feedback upskill human counselors? A randomized study with 90+ novice counselors.
https://arxiv.org/abs/2505.02428
Richard, O., Jollant, F., Billon, G., Attoe, C., Vodovar, D., & Piot, M. A. (2023).
Simulation training in suicide risk assessment and intervention: A systematic review and meta-analysis. Medical Education Online, 28(1), 2199469.
https://doi.org/10.1080/10872981.2023.2199469
Saunders, A., Vega, M., Ianelli, H., Cross, S., & Attoe, C. (2021).
Evaluating the impact of simulation-based mental health training on self-efficacy: A retrospective data analysis. International Journal of Healthcare Simulation.
https://www.ijohs.com/article/doi/10.54531/XRRK9799
Steenstra, I., Nouraei, F., & Bickmore, T. W. (2025).
Scaffolding empathy: Training counselors with simulated patients and utterance-level performance visualizations.
https://arxiv.org/abs/2502.18673
Suicide Prevention Resource Center. (2024).
Simulation in suicide-specific clinical training: Guidelines for programs.
https://sprc.org/online-library/simulation-toolkit
Williams, B., Reddy, P., Marshall, S., Beovich, B., & McKarney, L. (2017).
Simulation and mental health outcomes: A scoping review. Advances in Simulation, 2, 2.
