“At ReflexAI, we are on a mission to improve training for peer-to-peer crisis support among veterans with our revolutionary AI-powered model, HomeTeam. Appen is an essential partner to us in this process. They appreciate the sensitive nature of our work and provide expert support for fine-tuning our model to accurately replicate how a conversation about mental health and crisis would go. Our partnership with Appen enabled us to achieve 93% positive user feedback.”
– Glenn Herzberg, Director of Product Marketing at ReflexAI
Mental health support for U.S. veterans is essential, with nearly 20 veteran suicides occurring daily. To address this critical need, ReflexAI developed HomeTeam, an AI-driven platform that helps crisis counselors and loved ones prepare for conversations about veteran mental health and suicide prevention. To achieve this, ReflexAI needed high-quality training data to build a realistic and empathetic model capable of handling sensitive communication around mental health. This case study explores the development, refinement, and execution of a successful AI-powered mental health platform.
As leaders at The Trevor Project, a nonprofit focused on suicide prevention for LGBTQ+ youth, Sam Dorison and John Callery-Coyne identified inefficiencies in traditional crisis counselor training, which relies on repetitive and time-consuming role plays. To streamline the process without compromising the quality of training, they developed AI software using OpenAI’s GPT-2 to provide realistic training simulations. This innovation enabled The Trevor Project to increase training capacity from 60 to 600 call center agents per month.
Building on this success, Dorison and Callery-Coyne cofounded ReflexAI. With backing from organizations like the U.S. Department of Veterans Affairs and Google, ReflexAI launched HomeTeam, an AI platform designed to help crisis counselors and loved ones of veterans practice sensitive conversations about mental health and suicide prevention in a safe, low-risk environment. Today, ReflexAI has successfully delivered high quality training to thousands and continues to offer HomeTeam for free to all veterans, encouraging them to support each other with mental health challenges.
HomeTeam integrates educational modules on key mental health topics with AI-powered roleplay simulations for practical training. ReflexAI focused on delivering a tool that emphasizes responsible AI deployment, backed by user research and feedback from the veteran community. To meet their goals, ReflexAI partnered with Appen to gather high-quality training data and fine-tune their AI model for realistic, empathetic conversations while prioritizing responsible AI development and human feedback.
Fine-tuning an AI model for mental health use cases is crucial because the subject matter requires a high degree of empathy, sensitivity, and nuance. A general model may lack the context-specific understanding needed to address complex emotional situations, recognize signs of distress, or provide appropriate support. By fine-tuning with appropriate data and human evaluation, the AI can be trained to handle the specific language, challenges, and ethical considerations associated with mental health, ensuring it delivers accurate, compassionate, and safe responses tailored to individual needs.
To prepare HomeTeam for sensitive mental health conversations, ReflexAI and Appen had to first overcome the following challenges:
To build an AI model that simulates realistic conversations around mental health and suicide prevention, ReflexAI needed high-quality training data that reflected the sensitive nature of veteran experiences while incorporating proven conversational strategies and counseling techniques. In partnership with Appen, they first established specific data requirements such as written quality, subject matter expertise, and volume to adapt to the ever-evolving nature of these requisites.
Veterans come from diverse backgrounds and face varying mental health challenges. ReflexAI and Appen worked together to collect representative data that captured a range of veteran experiences and communication styles, making the AI model more empathetic and adaptable.
Given the sensitive nature of mental health conversations, ReflexAI prioritized ethical AI development. Appen played a critical role in ensuring that the training data was ethically sourced and representative of diverse veteran experiences. To ensure high ethical standards, Appen captured data from a diverse group of contributors across the US, paying contributors fairly and providing support for those working with sensitive topics like mental health and suicide. ReflexAI and Appen’s shared commitment to responsible AI ensured the tool could handle sensitive situations, such as suicide prevention.
Recognizing that traditional data collection methods were insufficient, ReflexAI and Appen adopted a two-pronged approach to overcome these challenges:
Appen assembled a diverse group of highly qualified human contributors with subject matter expertise across mental health, education, technology, and veteran's services to generate training data. Crowd diversity is crucial for AI development because it ensures the model learns from a wide range of perspectives and experiences, resulting in more inclusive and accurate models capable of handling diverse real-world scenarios. This diverse crowd ensured that HomeTeam accurately captured the nuances of mental health conversations.
Key statistics include:
ReflexAI and Appen utilized this diverse crowd to complete various tasks throughout the model training and fine-tuning life cycle. Key tasks included creating synthetic transcripts, running simulations, and annotating and evaluating the model’s performance before deployment. Incorporating human expertise at multiple stages of the model training lifecycle enabled ReflexAI to execute the most empathetic, sensitive and realistic model possible. The iterative process ensured that the AI simulations accurately reflected real-world conversations and respected ethical boundaries.
By addressing the challenges of data collection and ethical considerations, HomeTeam now provides a comprehensive training tool that allows users to engage in realistic roleplay scenarios, enhancing their ability to support veterans facing mental health crises. Key outcomes include:
The development of ReflexAI's HomeTeam platform marks a significant advancement in veteran mental health support. By utilizing AI roleplay simulations, ReflexAI enables counselors and loved ones to practice difficult conversations in a controlled and safe environment. Partnering with Appen ensured that the training data used to develop the model was high-quality and ethically sourced, enabling ReflexAI to create an effective and empathetic tool. As AI mental health companies continue to innovate, HomeTeam serves as a model for how technology can be responsibly deployed to support the unique needs of veterans and their communities.
The partnership with Appen ensured that the training data used to develop the AI model was both high-quality and ethically sourced, As AI continues to play a pivotal role in mental health support, HomeTeam stands as a model for how technology can be responsibly used to address the unique needs of veterans and their support networks.
