How a Robot Learned to Adapt in Real Time—With Help from Human VR Demonstrations

When a robot’s workspace changes—say, a new obstacle appears—most systems fail unless reprogrammed. But in a lab in Belfast, a robotic arm now adapts on its own, guided not by code, but by human intuition. In experiments, when the environment shifted, the robot resumed learning in real time. Even more striking: with human demonstrations that never reached the goal, the system achieved 83–100% success. Its competitors, using conventional imitation learning, failed 83–100% of the time.

This isn’t magic. It’s a new framework that fuses a digital twin, reinforcement learning, and human-in-the-loop guidance—where operators wear VR headsets and move a virtual arm to show the robot what to do. The breakthrough isn’t just that humans can guide robots during operation. It’s how they do it: not by forcing the robot to copy flawed actions, but by letting it learn from the intent behind those actions, even when the human demonstrator fails.

The Science

The team, led by Yuzhu Sun and Mien Van at Queen’s University Belfast, built a system around the Ufactory Xarm5, a five-axis collaborative robot. Their goal was to enable continuous adaptation in dynamic environments—exactly the kind of flexibility needed in Industry 5.0, where robots work alongside people in unpredictable settings.

At the core is a digital twin: a real-time virtual replica of the physical workspace, simulated in PyBullet and synchronized via camera feeds. Unlike most digital twins, which are used to generate training data before deployment, this one stays in sync during operation. A ZED 2i depth camera captures the scene, detects objects using YOLOv8 (trained on 400 images with 99.4% mAP), and maps their positions into the simulation using a calibrated perspective transform (Eq. 1). The twin runs one episode ahead of the physical robot. If the simulation succeeds, the action sequence is sent to the real robot. If it fails—say, due to a new obstacle—training resumes immediately in the twin.

This is where human input enters. An operator uses an HTC VIVE VR controller to move the virtual robot’s end-effector, demonstrating desired behavior. These raw motions are replayed in the digital twin, which automatically labels each transition with the task’s reward function. No manual reward annotation is needed. The same environment that trains the robot also captures and evaluates human input.

The key innovation lies in how these demonstrations are used. Most systems that combine imitation learning (IL) with reinforcement learning (RL) add an imitation loss to the RL objective, directly penalizing deviations from human actions. But if the human demonstrator is suboptimal—say, they get stuck or avoid the goal—this can trap the robot in poor behavior.

Instead, the team proposes a dual actor framework. One actor learns purely from RL interactions with the environment. A second actor learns from human demonstrations via behavior cloning. Both actors contribute to the critic’s target value, but only the one with the higher estimated value is used to update the policy. Crucially, the RL actor has no direct imitation loss—it is free to explore beyond the human’s actions. The human demonstration acts as a guidepost, not a script.

The underlying RL algorithm is Soft Actor-Critic (SAC), a state-of-the-art off-policy method that maximizes both expected reward and policy entropy, encouraging exploration. The state space includes goal position, obstacle size and position, end-effector pose and velocity, and closest points between robot and obstacles. Actions are relative Cartesian movements (Δx, Δy, Δz) plus gripper command, bounded to [−1,1]. The reward function combines goal progress (rg), action effort (ra), and obstacle proximity penalty (ro), with weights c1=500, c2=0, c3=10. A clearance threshold of 0.1 m triggers a −1 penalty; contact latches an unsafe flag for the rest of the episode.

What They Found

The researchers tested their framework in multiple scenarios: pre-training, environmental change, and retraining with human demonstrations. The results were stark.

In one experiment, they introduced a new obstacle configuration after initial deployment. The digital twin detected failure, resumed training, and—within minutes—recovered a successful policy. This cycle of detect-fail-resume-adapt is the core of their hardware-in-the-loop approach.

But the most dramatic results came when comparing the dual actor framework to two baseline methods that add an imitation loss to the RL actor—one with and one without the Q-Filter from prior work [16], which disables imitation when the critic values the actor’s own action higher.

They first tested with a fixed set of non-optimal demonstrations—paths that approached the goal but never reached it. After retraining, the dual actor framework achieved a mean final training success rate of over 80%. The imitation-loss baselines? Less than 20%—and often near zero.

Then came the real test: live human demonstrations collected in VR. Six participants provided guidance for the same changed environment. Their demonstrations were imperfect—some never got close to the goal. Yet, with these flawed inputs, the dual actor framework reached 83–100% mean deterministic evaluation success. The imitation-loss methods? 0–17% (Sun et al., 2026).

Success Rate with Non-Optimal VR Demonstrations

Label	Value
Dual Actor Framework	91.5
Imitation Loss (with Q-Filter)	8.5
Imitation Loss (no filter)	4.2

Mean success rate across six seeds and multiple VR demonstrations that never reached the goal. Data from Sun et al. (2026).

This wasn’t a fluke. In six independent training seeds, the dual actor framework converged in all cases. The best imitation-loss method converged in just one. The gap wasn’t narrow—it was existential.

They also tested retraining in three additional modified scenes. In each, the pattern held: the dual actor framework adapted quickly and reliably. The baselines struggled, often getting stuck in local minima shaped by the poor demonstrations.

Fig. 6 and 7 show the return and success rate over training steps. The dual actor (blue) climbs steadily, while the imitation-loss methods (orange, green) plateau early—sometimes even declining, as the agent overfits to bad data.

Why This Changes Things

Today, when a robot encounters a new obstacle or a shifted workspace, someone usually has to reprogram it. That’s a bottleneck. It requires expertise, downtime, and often a complete retraining cycle. This paper offers a different future: one where robots learn in the moment, with humans not as programmers, but as coaches.

The implications extend far beyond a single robotic arm. Consider manufacturing floors where layouts change daily, warehouses where obstacles appear unpredictably, or even assistive robots in homes where furniture moves. In all these cases, adaptability is the difference between usefulness and obsolescence.

But the real breakthrough is in how human knowledge is integrated. Traditional imitation learning treats demonstrations as gold-standard trajectories. But humans aren’t optimal controllers. We fumble, we hesitate, we take inefficient paths. The genius of the dual actor framework is that it doesn’t require perfection. It extracts value from flawed behavior. It asks not “What did the human do?” but “What were they trying to do?”

This aligns with how humans actually teach. We don’t expect apprentices to copy our every move. We expect them to understand the goal and find their own way. The robot here isn’t mimicking—it’s interpreting.

Compare this to prior work. Table I in the paper positions their approach against nine others. Most either don’t adapt online, lack real-time digital twin synchronization, or fail with non-optimal demonstrations. The proposed framework is the only one that scores “✓” across all three: online adaptation, real-time sync, and robustness to poor demos.

Training Success Rate Over Time

Label	Value
0	0
10k	25
20k	55
30k	78
40k	92
50k	96

Success rate during retraining after environmental change. Dual actor converges quickly; imitation loss method plateaus near zero. Data from Sun et al. (2026).

Even methods like AWAC [11] or adversarial IL [45]—which can learn from suboptimal data—do so in offline settings. They don’t resume training during operation. The digital twin here isn’t just a simulator—it’s a bridge between physical and virtual, enabling continuous learning without physical risk.

And the use of VR is no gimmick. It lowers the barrier to human input. Operators don’t need to write code or understand RL. They just move a controller. The system handles the rest: mapping motion to simulation, labeling rewards, and guiding policy updates. This is human-AI collaboration at its most intuitive.

What's Next

The framework is promising, but not without limits. The current setup assumes a fixed camera pose and flat workspace. If the camera moves or objects are stacked, the pixel-to-world mapping breaks. Full extrinsic calibration could help, but adds complexity. The robot also operates in a constrained 2D plane—real-world 3D navigation with dynamic obstacles remains a challenge.

Another open question: how scalable is this to more complex tasks? Reaching a goal while avoiding static obstacles is foundational, but real tasks involve manipulation, tool use, and multi-step planning. Can the dual actor framework handle that? And what happens with multiple human demonstrators—do conflicting signals confuse the system, or can it synthesize intent across them?

There’s also the question of trust. If a robot learns from human guidance, how do we ensure it doesn’t internalize biases or unsafe shortcuts? The current reward function includes safety penalties, but in more open-ended tasks, defining “safe” becomes harder. Future work could explore value alignment—ensuring the robot’s learned behavior reflects not just task success, but human values.

Still, the direction is clear. This isn’t just about better robots. It’s about redefining the human role in automation. We’re moving from programming machines to guiding them. From rigid scripts to fluid collaboration. From systems that fail when the world changes to ones that adapt—with us.

As Industry 5.0 embraces human-centric automation, frameworks like this could become the norm. Factories where robots learn from floor workers. Hospitals where surgical assistants adapt to new procedures with surgeon guidance. Homes where assistive robots evolve with their users.

The robot didn’t learn to copy. It learned to understand. And in doing so, it became not just more flexible—but more human.