When Mohammad Iqbal Nouyed, a postdoctoral fellow at West Virginia University, started analyzing what AI could teach medical students about psychiatry, he helped uncover something unexpected: the technology was surprisingly good at creating realistic patient case studies, but it still needs human doctors watching over its shoulder.
The study, conducted at WVU's School of Medicine in Morgantown, put ChatGPT-5 Pro to the test. Researchers asked the AI to generate clinical vignettes—brief scenarios describing a patient's symptoms, history, and behavior—for training future psychiatrists. The results were encouraging but came with a clear warning sign.
"With AI, we are connected no matter what—that's the reality," said Dr. Wanhong Zheng, a professor in WVU's Department of Behavioral Medicine and Psychiatry who co-led the study. "Students and residents can read about these new mental health concerns of people using AI in medical journals and social media, but they don't typically see them in real clinical settings. We designed this study to cover major psychiatric problems known to be related to AI chatbot use."
The AI successfully created training cases covering schizophrenia spectrum disorder, anxiety disorders, mood disorders, major depression, and psychosis. Gangqing "Michael" Hu, an associate professor who co-led the project, explained why this matters: doctors need to understand when a chatbot might actually make a patient's condition worse rather than better.
"A chatbot can be warm and agreeable, but that can be risky if it keeps validating a patient's unusual beliefs," Hu said. "If a chatbot reinforces a patient's unusual thinking instead of gently questioning it, that could make symptoms harder to treat."
The research team—also including Dr. Dilip Chandran and Dr. Daniel Elswick, both board-certified psychiatrists—evaluated the AI-generated cases and found they contained strong diagnostic details. However, the team emphasized that human oversight must remain central to any medical training that uses AI materials. Their findings were published in the journal npj Digital Medicine.
The researchers aren't suggesting AI replace traditional teaching methods. Instead, they see it as a supplement—giving students exposure to scenarios they might not encounter often during their training. But for now, every AI-generated case must pass through an experienced psychiatrist's review before reaching a medical student.