When 291 Americans encountered suspicious headlines on a fake social media app, they reacted to fact-checking labels in a way that challenges our assumptions about machines versus people. Researchers at Penn State discovered something counterintuitive: users trust AI and human fact-checkers equally—but they value them for opposite reasons.
The stakes for this research couldn't be higher. Social media is drowning in misinformation, and the volume far exceeds what human fact-checkers can reasonably handle. Understanding how people evaluate competing fact-checking systems matters because it shapes how we'll design tools to protect users from false information at scale.
The study, published in Media Psychology, tested how 291 U.S. participants responded to news headlines labeled as fact-checked by either an AI system or human experts. The researchers behind it—led by S. Shyam Sundar, Evan Pugh University Professor at Penn State, and Mengqi Liao, now an assistant professor at the University of Georgia—wanted to understand something deeper than just "which do people trust more?" They found that question itself misses the point.
The findings reveal a striking division of perceived labor. Users trust AI more when it spots what researchers call "low-level linguistic features"—the telltale red flags in wording and phrasing that signal a post isn't credible. AI feels like a linguistic scanner, catching the surface-level signs of manipulation. Humans, by contrast, earned trust for corroborating evidence across multiple sources—the harder, slower work of piecing together information from different places to verify claims.
What's particularly revealing is why people held these views. Participants assumed AI systems were objective and accurate, yet simultaneously distrusted them for lacking human judgment. These competing perceptions—one positive, one negative—essentially canceled each other out, explaining why trust levels came out roughly equal between the two approaches. "There's a very clear distinction," Sundar said, "that AI is considered good at low-level linguistic features, like identifying telltale signs that something is not credible. Humans are seen as being better at corroborating evidence from multiple sources."
The research team tested three types of explanations for why a post was marked false. Evidence-based explanations provided references to contradictory information. Feature-based explanations flagged suspicious wording. A "black box" option offered no explanation at all. Users significantly preferred systems that explained their reasoning—any explanation beat the black box approach. This insight carries practical weight: the mere act of showing users how a system reaches a decision helps calibrate their trust and prevents blind reliance on automated verdicts.
Perhaps most intriguingly, Sundar suggested the future may not be either-or at all. "The ideal situation would be a human-AI collaboration," he said, "but it's not always possible for humans to intervene and check for evidence from multiple sources. So, we are going to have to, at some level, completely automate this whole fact-checking business."
The research points toward a more honest approach to digital trust. Rather than asking whether people should trust AI or humans more, designers should help users understand what each does well. AI excels at speed and pattern-matching. Humans bring contextual judgment. The most effective fact-checking tools won't hide this division of labor—they'll make it transparent, helping people move beyond naive stereotypes about machines and toward a clearer picture of what the technology can actually do.