When sediment clouds swallow an underwater robot whole, MIT engineer Amy Phung and her colleagues used to wait—sometimes for hours—until the marine dust settled enough for cameras to see again. Now, they don't have to. Phung and senior scientist Richard Camilli at the Woods Hole Oceanographic Institution have developed Sonar-MASt3R, a breakthrough system that fuses sonar's acoustic precision with optical cameras' visual clarity to navigate and map murky underwater environments in real time, even when visibility is near zero.
For decades, underwater robotics teams have faced a stubborn choice: use optical cameras for detailed imagery in clear water, or rely on sonar to sense through murk but sacrifice visual detail. Sonar sensors emit acoustic waves and measure how they bounce back, revealing the exact shape, distance, and depth of objects—but the resulting maps look like fuzzy outlines. Optical cameras, meanwhile, capture stunning visual detail only in relatively clear, well-lit conditions. A handful of research groups have attempted to merge these two approaches—what scientists call opti-acoustic fusion—but none had achieved real-time, high-resolution 3D mapping of underwater environments in truly turbid conditions until now.
Phung, a graduate student in MIT's Department of Aeronautics and Astronautics, and Camilli built their system on top of MASt3R, an image-matching algorithm developed by French researchers. MASt3R analyzes visual images to estimate the relative depth of each pixel, generating 3D maps in real time. The catch: it works in relative measurements only. "It will say 'this pixel is five units closer than this pixel,' but it can't say whether that's 5 meters or 5 feet," Phung explained. Sonar solves that problem. By providing absolute measurements of distance and scale from acoustic reflections, sonar acts as a calibrating anchor, allowing Sonar-MASt3R to generate precise, scaled 3D maps.
The team tested their system in tank experiments where they controlled water visibility, deliberately kicking up clouds of sediment to simulate real-world conditions. Even in the densest sediment clouds, Sonar-MASt3R mapped the tank's environment and visualized centimeter-scale details of objects submerged within it. The achievement opens doors that have long been closed: safe navigation through the surf zones where unexploded underwater mines rest, deep-sea construction and maintenance work, scientific exploration of the seafloor, and underwater recovery operations.
Camilli notes that mines set in turbid surf zones present particular hazards. "The ability to get rid of those safely is best done by robotics," he says. "But a lot of these explosives are set in surf zone environments where visibility adds to the challenge of doing this safely. That's one of many applications that our technique can be used for." Phung presented the research at the IEEE International Conference on Robotics and Automation (ICRA) this week, signaling the arrival of a new capability in an old field.
The achievement echoes nature's own solutions—Phung describes the method as pairing a dolphin's echolocation with a sea turtle's close-range vision. Now, underwater vehicles no longer need to choose between seeing far or seeing clear. They can do both, pushing robotics into environments where humans simply cannot go, even when the water is too dark and clouded to trust either technology alone.