A student in Tennessee stares at a math problem, gets it wrong, and the screen doesn't hand over the answer. Instead, a friendly AI named NUMI walks them through the mistake, step by step, then makes them show they've got it right three times in a row. That small act of patience may be a big deal for how kids learn math — and for who gets to learn it at all.
The experiment comes from researchers at the University of Toronto Mississauga, led by economics professor Philip Oreopoulos, who teamed up with math and computer science professor Michael Liut to build NUMI. In a rigorous study involving more than 6,000 Tennessee middle schoolers, they wanted to answer a hot and mostly unanswered question: does AI actually help kids learn, or just help them finish faster?
"Students want to get through their math assignments quickly and easily," Oreopoulos says. "Easier equals less effort and less learning." The trap with most AI tools is that they spit out answers instantly, which feels great in the moment but skips the actual thinking. NUMI was designed to do the opposite — to coach students through their errors rather than solve the problem for them, turning mistakes into what Oreopoulos calls "productive learning moments."
The results were encouraging. Students using NUMI scored three points higher on tests covering both practiced and unpracticed problems than students who got conventional computerized instruction. That might sound small, but it matters because the point was to prove that a well-designed AI tutor can out-teach a computer that just drills answers. Oreopoulos calls it "objective evidence to show that a well-designed AI tutor is more effective than instruction without AI" — a first in a field where almost no research like this had been done.
Why does this matter beyond one classroom? Because great human tutors are expensive and rare, and they mostly flow to families who need them least. A well-built AI tutor, Oreopoulos points out, could scale quality tutoring to far more students for a much lower cost — potentially transforming outcomes for kids from all kinds of backgrounds.
The researchers aren't claiming victory yet. They're careful to say a good human tutor is usually better, and that the real question is whether a well-designed AI tutor beats nothing at all when no human is available. That's a low bar that could change a lot of lives.
Next, Oreopoulos wants to test a voice-to-voice NUMI that talks the way a great teacher talks — patient, playful, a little funny. The more the tutor feels like a person, the more kids may actually want to learn. As schools everywhere debate whether AI helps or harms, this experiment offers a hopeful answer: it depends on how you build it. Build it to teach, and it just might.