When patients stop taking their medications, the reasons often remain hidden in doctors' notes—until now. Researchers at the University of Tartu in Estonia have demonstrated that artificial intelligence can read clinical notes and reveal why people discontinue treatments like statins or antidiabetic medications, with accuracy rates between 93% and 96%. The breakthrough, published in the Journal of Medical Internet Research, could help healthcare systems understand treatment patterns in ways that were previously too time-consuming to pursue.
The team, led by junior research fellow Hendrik Šuvalov of the Health Informatics Research Group, combined prescription data from a 10% representative sample of the Estonian population spanning 2012 to 2019 with doctors' free-text clinical notes. Prescription records alone only show when a medication stopped being purchased—never why. "The reason is often written in the doctor's notes instead," Šuvalov explained. "Until now, this information could only be used to a very limited extent because manually reviewing medical records is extremely time-consuming."
The researchers used the Llama 3.1-70B language model running on a secure local university server to scan clinical notes for references to treatment discontinuation. Medical experts then verified the results. The AI proved remarkably adept: accuracy reached 93% to 98% for identifying discontinuation phrases, and 95% to 96% for categorizing the underlying reasons.
The findings reveal something that could reshape how clinicians approach patient care. Adverse reactions were the most common reason for stopping medication—accounting for approximately 70% of documented statin discontinuations and nearly 45% of antidiabetic medication discontinuations. The researchers also found that stopping antidiabetic medications was more frequently linked to insufficient treatment effect and contraindications compared with statins.
What makes this study valuable extends beyond the specific findings. Šuvalov notes that health data typically focuses on diagnoses, lab results, and prescriptions, but misses the richer story unfolding in clinical notes. "Patients' experiences, side effects, and reasons for changing treatment often make their way into doctors' notes, but remain unused in conventional data analyses," he said. By converting unstructured clinical text into analyzable data, the methodology opens doors for researchers to better understand real-world treatment pathways and support evidence-based health policy decisions. In short, the quiet conversations between doctors and patients about why a treatment didn't work can now inform how future patients are cared for.