Incoming college students are picking up AI tools without picking up the instruction manual. A new study from researchers at James Madison University and the Office of the Provost found that while American freshmen are already actively using generative AI, their understanding of how these tools actually work—and how to evaluate them critically—remains surprisingly shallow.
The research team, led by Jaime Miller, Stuart Miller, and Rachel Whitman Rotch, began gathering data in summer 2025 using the Generative AI Literacy Assessment Test (GLAT) to measure incoming students' baseline AI literacy. Their findings suggest that higher education has been operating on a dangerous assumption: that today's digitally native students automatically understand the technology reshaping their world. The data tells a different story.
"What we found is that incoming students are already using AI tools—but their underlying understanding of how those tools work, and how to evaluate them critically, is still very limited," said Stuart Miller, assistant director of Academic Data Acquisitions and Reporting. The implications are significant. If students are deploying these systems without foundational knowledge or ethical guardrails, colleges aren't just failing to develop skilled users—they're potentially enabling uncritical adoption of powerful technology.
The researchers' core recommendation challenges the targeted intervention approach many institutions favor. Rather than designing specialized courses for certain student populations, the study argues that colleges need institution-wide, early AI literacy education available to all incoming students. This curriculum should prioritize three interconnected skills: understanding how generative AI systems function, developing metacognitive awareness (thinking about your own thinking), and cultivating ethical reasoning about when and how to use these tools.
The timing of this research feels urgent. AI has moved from hypothetical future concern to immediate classroom reality. Students are using these tools to draft essays, solve problem sets, and gather research—often without the conceptual framework to understand what they're delegating to a machine or the ethical considerations that should guide those decisions. A baseline assessment like the GLAT provides something institutions have lacked: concrete data about where students actually stand.
The research team's work has already gained traction beyond academic circles. They presented their findings at the 2026 Association for Psychological Science Annual Convention in Barcelona, Spain, where their poster—titled "AI Everywhere, Literacy Nowhere: The Need for Generative AI Education"—drew notable attention from fellow researchers and educators grappling with identical challenges at their own institutions.
The success of this partnership between James Madison University and the Office of the Provost has opened doors for deeper collaboration. Two additional joint studies are scheduled for later this summer and early fall, with plans to track how students' AI literacy develops throughout their college experience and to refine how institutions measure AI-related ethical reasoning and metacognition.
For institutions ready to act, the message is clear: the window for proactive, intentional AI education is now. Students will keep using these tools regardless. The question is whether colleges will equip them to do so thoughtfully.