Robots learn obstacle avoidance from human demos, but miss hidden surfaces
A robot learns obstacle avoidance from scene data and demos, but head-mounted depth can miss hidden surfaces.
- Publication
- arXiv
- Stage
- Preprint
- What we read
- Summary of the paper
- Authors
- Jason Jingzhou Liu, Tony Tao, Deepak Pathak, Ruslan Salakhutdinov
- Universities and research institutions
- Carnegie Mellon University, ∗Equal contribution
Institution metadata: Original paper ↗
What they did and found
The robot learned from demonstrations recorded with a head-mounted camera. It builds a 3D map from depth data, then retargets human motions to avoid obstacles. The system also masks views that don’t match well, and real-time adjustments helped some tasks.
Why it matters
This suggests robots can learn safe moves in cluttered spaces from shared sensor data and demos, but sensing gaps and weak force feedback stay risks.
Offline scene data help shape the robot’s posture to avoid obstacles while following demonstrations. It relies on one depth view and can miss hidden surfaces.
On-policy corrections add real-world tweaks to the robot’s motions, showing that small human-guided adjustments can improve performance over time without changing the whole task.
What remains uncertain
Delay-calibrated action labels are approximate and cannot capture exact contact forces. The offline data uses a single head-mounted depth camera, which may miss occluded or moving surfaces.
Original sources · 1
- Pixel-UMI: Learning Environment-Aware Whole-Body Manipulation from Sensorized Human Demonstrations ↗arXiv · 2026-11-02
Check the original paper for its authors, methods, version and access terms.
What is your take?
Ask a question, add useful context or share a different perspective. Keep the conversation respectful and grounded.