Why AI Mental Health Chatbots Fail When It Matters Most: The Hidden Vulnerabilities Stress-Testing Reveals - An Interview with Shirali and Arul Nigam of Circuit Breaker Labs
Shirali and Arul Nigam of Circuit Breaker Labs on why AI mental health chatbots fail, how stress-testing exposes their hidden vulnerabilities, and what therapists need to know.
Curt and Katie talk with Shirali and Arul Nigam, the sibling co-founders of Circuit Breaker Labs, about what therapists tend to get wrong about AI, why the safety infrastructure behind many mental health chatbots is weaker than it looks, and how their team stress-tests these tools to find dangerous failures before real users ever encounter them.
Generative AI is probabilistic, so the same prompt can return a safe answer one moment and a harmful one the next. Shirali and Arul explain how guardrails get bypassed by a misspelled word, a teenager's slang, or the hundredth message in a long conversation, why mental health chatbots tend to fail in the moments that matter most, and what stress-testing hundreds of thousands of simulated conversations actually reveals about model safety.
The conversation closes on what clinicians can do now, why clinical insight is the missing ingredient in AI safety, and why third-party validation is becoming the standard regulators and developers expect. Used well, AI can be a supplement to care or a gateway to a human therapist, but it is not a replacement, and getting there safely starts with building clinical insight in from the foundation.
In this episode, we discuss:
- Why people usually turn to AI in place of no care, not in place of a therapist
- Why generative AI's unpredictability, not a single bad answer, is the real safety problem
- How a misspelling, slang, or a long conversation can slip past chatbot guardrails
- Why AI mental health chatbots tend to fail in the highest-risk moments
- What stress-testing hundreds of thousands of conversations reveals about model safety
- Why clinical insight is the missing ingredient, and what clinicians can do now
Timestamps:
0:00 - Introduction
1:38 - Meet Shirali and Arul Nigam and Circuit Breaker Labs
3:37 - What therapists get wrong about AI
5:28 - Deterministic versus generative AI, and why the risk scales
8:26 - The safety problem in AI mental health: trust, training data, and agreeableness
11:29 - Guardrails, lifeguard models, and the 988 problem
17:30 - Deploying clinical insight at scale and building safety in from the start
21:09 - How stress-testing works: context pollution and adversarial simulation
27:11 - What the stress tests reveal: variance, typos, and bypassed guardrails
30:59 - Regulation, credential hallucination, and third-party validation
36:58 - What clinicians can do, and the missing clinical insight
40:20 - Where to find Circuit Breaker Labs
Guest Bios:
Shirali and Arul Nigam are siblings and the co-founders of Circuit Breaker Labs, which autonomously pressure-tests the AI systems that interact with people to find hidden mental health vulnerabilities before they reach real users. Shirali brings expertise in neuroscience, translational research, and clinical work, with experience at the Howard Hughes Medical Institute's Janelia Research Campus, NIH NINDS, Harvard's Wyss Institute, Johns Hopkins, and Children's National. She holds a BS in Biomedical Engineering from The George Washington University and an MBA from The Wharton School, University of Pennsylvania. Arul has conducted technical and policy research on ethical AI, with a focus on bias and fairness, at Georgetown University and Thomas Jefferson High School for Science and Technology, and holds a BSBA in Operations and Analytics from Georgetown University. Learn more at circuitbreakerlabs.ai.
Full show notes and transcript: mtsgpodcast.com
Join the Modern Therapist Community
Patreon: https://www.patreon.com/c/mtsgpodcast
Facebook Group: https://www.facebook.com/groups/therapyreimagined
Modern Therapist's Survival Guide Creative Credits
Voice Over by DW McCann: https://www.facebook.com/McCannDW/
Music by Crystal Grooms Mangano: https://groomsymusic.com/
