Claude 3.5 Sonnet checked a research ethics application in about seven minutes—and gave fully correct answers to 87.9% of its checks in a University of Bristol study. The result points to a possible new helper for a process that can involve reading hundreds of pages, but the study also found that AI tools differ sharply in what they can do.

Before a study can begin, its application must be reviewed against standards and requirements. Some checks involve comparing details across several documents, a time-consuming task for research approvals staff. The Bristol team, working with the UK Health Research Authority (HRA), tested whether large language models—AI programs that work with text—could help with this routine work.

The researchers compared Claude 3.5 Sonnet, ChatGPT-4o and Microsoft Copilot on checks against the HRA’s published quality standards. Claude had the highest rate of fully correct answers, at 87.9%. ChatGPT scored 66.5%, while Copilot scored 62.8%. The time needed to assess an application also varied: Claude averaged just over seven minutes, ChatGPT 13 minutes and Copilot 23 minutes.

The gap appeared in the models’ ability to flag problems, too. Claude identified an average of eight errors in each application. ChatGPT found 2.3, and Copilot 1.3. Copilot also failed to answer data protection questions in most applications. Across all three systems, problems with logical reasoning were the most common source of inaccurate answers. Claude produced fewer hallucinations—answers that are wrong or not supported by the information—than the other models.

The findings, published in Scientific Reports on Nov. 16, suggest AI could help identify missing details or departures from written standards before an application reaches an ethics committee. That could reduce some routine checking and give staff more time for complex issues. The study did not show that AI can decide whether research is ethically acceptable. That judgment, the researchers stressed, belongs to people, not a checklist or an automated system.

Lead author Dr. Panos Maghsoudlou, an NIHR academic clinical lecturer in ophthalmology at the University of Bristol, said the results show both the promise of AI-assisted checks and the risk of using its answers without verification. Dr. Jonathan Fennelly-Barnwell, HRA director of approvals and a co-author, said the research was intended to build evidence about how AI might support approval processes. He said routine automated checks could free staff to focus on more complex regulatory questions and might help shorten review timelines.

For now, the results describe a possible support tool, not a replacement for researchers, approvals staff or ethics committees. The large differences between the three models also underline why any use would need careful checking. Further work can help establish where automated reviews are reliable and how they can fit into a process where human judgment remains central.