Policy Distillation Techniques in Robotics Research

Can a single neural network policy generalize over poses, objects, obstacles, backgrounds, scene arrangements, in-hand objects, and start/goal states? Introducing Neural MP: A generalist policy for solving motion planning tasks in the real world 🤖 Quickly and dynamically moving around and in-between obstacles (motion planning) is a crucial skill for robots to manipulate the world around us. Traditional methods (sampling, optimization or search) can be slow and/or require strong assumptions to deploy in the real world. Instead of solving each new motion planning problem from scratch, we distill knowledge across millions of problems into a generalist neural network policy.  Our Approach: 1) large-scale procedural scene generation 2) multi-modal sequence modeling 3) test-time optimization for safe deployment Data Generation involves: 1) Sampling programmatic assets (shelves, microwaves, cubbys, etc.) 2) Adding in realistic objects from Objaverse 3) Generating data at scale using a motion planner expert (AIT*) - 1M demos! We distill all of this data into a single, generalist policy Neural policies can hallucinate just like ChatGPT - this might not be safe to deploy! Our solution: Using the robot SDF, optimize for paths that have the least intersection of the robot with the scene. This technique improves deployment time success rate by 30-50%! Across 64 real-world motion planning problems, Neural MP drastically outperforms prior work, beating out SOTA sampling-based planners by 23%, trajectory optimizers by 17% and learning-based planners by 79%, achieving an overall success rate of 95.83% Neural MP extends directly to unstructured, in-the-wild scenes! From defrosting meat in the freezer and doing the dishes to tidying the cabinet and drying the plates, Neural MP does it all! Neural MP generalizes gracefully to OOD scenarios as well. The sword in the first video is double the size of any in-hand object in the training set! Meanwhile the model has never seen anything like the bookcase during training time, but it's still able to safely and accurately place books inside it. Since, we train a closed-loop policy, Neural MP can perform dynamic obstacle avoidance as well! First, Jim tries to attack the robot with a sword, but it has excellent dodging skills. Then, he adds obstacles dynamically while the robot moves and it’s still able to safely reach its goal. This work is the culmination of a year-long effort at Carnegie Mellon University with co-lead Jiahui(Jim) Yang as well as Russell Mendonca, Youssef Khaky, Russ Salakhutdinov, and Deepak Pathak The model and hardware deployment code is open-sourced and on Huggingface!  Run Neural MP on your robot today, check out the following: Web: https://lnkd.in/emGhSV8k Paper: https://lnkd.in/eGUmaXKh Code: https://lnkd.in/e6QehB7R News: https://lnkd.in/enFWRvft

What if robots could react, not just plan? A good read: https://lnkd.in/gEGSp_5U This paper proposes a Deep Reactive policy (DRP), a visuo-motor neural motion policy designed for generating reactive motions in diverse dynamic environments, operating directly on point cloud sensory input. Why does it matter? Most motion planners in robotics are either: Global optimizers: great at finding the perfect path, but they are way too slow and brittle in dynamic settings. Reactive controllers: quick on their feet, but they often get tunnel vision and crash in cluttered spaces. DRP claims to bridge the gap. And what makes it different? 1. IMPACT (transformer core): pretrained on 10 million generated expert trajectories across diverse simulation scenarios. 2. Student–teacher fine-tuning: fixes collision errors by distilling knowledge from a privileged controller (Geometric Fabrics) into a vision-based policy. 3. DCP-RMP (reactive layer): basically a reflex system that adjusts goals on the fly when obstacles move unexpectedly. Results are interesting for real-world evaluation: Static environments: Success Rate: DRP 90% | NeuralMP 30% | cuRobo-Voxels 60% Goal Blocking: Success Rate: DRP 100% | NeuralMP 6.67% | cuRobo-Voxels 3.33% Goal Blocking: Success Rate: DRP 92.86% | NeuralMP 0% | cuRobo-Voxels 0% Dynamic Goal Blocking: Success Rate: DRP 93.33% | NeuralMP 0% | cuRobo-Voxels 0% Floating Dynamic Obstacle: Success Rate: DRP 70% | NeuralMP 0% | cuRobo-Voxels 0% What stands out from the results is how well DRP handles dynamic uncertainty, the very scenarios where most planners collapse. NeuralMP, which relies on test-time optimization, simply can’t keep up with real-time changes, dropping to 0 in tasks like goal blocking and dynamic obstacles. Even cuRobo, despite being state-of-the-art in static planning, struggles once goals shift or obstacles move. DRP’s strength seems to come from its hybrid design: the transformer policy (IMPACT) gives it global context learned from millions of trajectories, while the reactive DCP-RMP layer gives it the kind of “reflexes” you normally don’t see in learned systems. The fact that it maintains 90% success even in cluttered or obstructed real-world environments suggests it isn’t just memorizing scenarios; it has genuinely learned a transferable strategy. That being said, the dependence on high-quality point clouds is a bottleneck. In noisy or occluded sensing conditions, performance may degrade. Also, results are currently limited to a single robot platform (Franka Panda). So this paper is less about replacing classical planning and more about rethinking the balance between experience and reflex. 

Deep Reactive Policy: Learning Reactive Manipulator Motion Planning for Dynamic Environments Arxiv: https://lnkd.in/gCBpvpX2 Project: How can robot arms plan collision-free motions in cluttered, dynamic, partially observable environments—like kitchens, factories, or warehouses? Deep Reactive Policy (DRP) introduces a visuo-motor neural policy trained on 10M expert trajectories, enhanced with student–teacher finetuning and a reactive goal-proposal module (DCP-RMP). The result: a motion policy that outperforms classical planners and prior neural methods in both simulation and real-world tests . 🔁 At a Glance 💡 Goal: Achieve safe, collision-free manipulator motion in dynamic, cluttered, and partially observable environments. ⚙️ Approach: IMPACT Policy (Transformer): Pretrained on 10M cuRobo trajectories with point-cloud input. Student–Teacher Finetuning: Distills local obstacle avoidance from Geometric Fabrics into IMPACT. DCP-RMP Module: Locally reactive layer that adapts goals in real time when dynamic obstacles appear. 📈 Impact (Key Metrics) 🧪 Simulation (DRPBench): 84.6–86% SR on Static + Suddenly Appearing Obstacle tasks. 75.5% SR on Floating Dynamic Obstacles vs. 32% (IMPACT alone). 65.3% SR on Dynamic Goal Blocking vs. 3% (cuRobo) . 🤖 Real-world (Franka Panda + Intel RealSense): 90–100% SR on static + appearing obstacle tasks. 70% SR on dynamic obstacle avoidance vs. 0–30% for baselines. 93% SR on Dynamic Goal Blocking (drawer + human operator test) . 🔬 Experiments 🧪 Tasks: Static clutter, sudden obstacle appearance, floating obstacles, goal blocking, dynamic goal blocking. 🦾 Robot: Franka Panda manipulator + multi-camera point cloud setup. 📐 Input: Point clouds fused with joint goals → IMPACT → refined by DCP-RMP. 🛠 How to Implement 1️⃣ Pretrain IMPACT on large-scale motion datasets. 2️⃣ Finetune via student–teacher loop (Geometric Fabrics as privileged teacher). 3️⃣ Deploy with DCP-RMP for local dynamic obstacle reactivity. 📦 Deployment Benefits ✅ Combines global planning awareness with local dynamic reactivity. ✅ Runs at 300 Hz (real-time feasible). ✅ Robust to partial observability and noisy depth perception. ✅ Generalizes from simulation to real-world tasks. 📣 Takeaway DRP bridges motion planning and reactive control. By fusing transformer-based policies, iterative finetuning, and reactive RMP modules, it sets a new benchmark for safe, adaptable manipulator planning in the wild

NVIDIA AI Releases HOVER: A Breakthrough AI for Versatile Humanoid Control in Robotics Researchers from NVIDIA, Carnegie Mellon University, UC Berkeley, UT Austin, and UC San Diego introduced HOVER, a unified neural controller aimed at enhancing humanoid robot capabilities. This research proposes a multi-mode policy distillation framework, integrating different control strategies into one cohesive policy, thereby making a notable advancement in humanoid robotics. The researchers formulate humanoid control as a goal-conditioned reinforcement learning task where the policy is trained to track real-time human motion. The state includes the robot’s proprioception and a unified target goal state. Using these inputs, they define a reward function for policy optimization. The actions represent target joint positions that are fed into a PD controller. The system employs Proximal Policy Optimization (PPO) to maximize cumulative discounted rewards, essentially training the humanoid to follow target commands at each timestep..... Read full article here: https://lnkd.in/gqji8w8U Paper: https://pxl.to/ds6aqqk8 GitHub Page: https://pxl.to/ds6aqqk8 NVIDIA NVIDIA AI

From object pickups to agile turns, Opt2Skill turns dynamic whole-body trajectories into deployable humanoid skills. [⚡Join 2500+ Robotics enthusiasts – https://lnkd.in/dYxB9iCh] A team from Georgia Institute of Technology - Fukang Liu, Zhaoyuan Gu, Yilin Cai, Ziyi Zhou, Hyunyoung Jung, Jaehwi Jang, Shijie Zhao, Sehoon Ha, Yue Chen, Danfei Xu†, and Ye Zhao† (†Equal Senior Authors) Introduce Opt2Skill, a learning-based framework that distils dynamically feasible whole-body trajectories into generalizable loco-manipulation skills for humanoids. Opt2Skill starts with motion trajectories optimised in physics simulation for specific tasks and converts them into a reusable skill policy via reinforcement learning. The trained controller enables humanoids to handle complex tasks like navigating obstacles while carrying objects, reaching and walking with balance, and reacting to contact forces, all with high physical realism. This bridges the gap between task-level planning and low-level control, making human-like coordination a learned, reusable skill. If humanoids can now distil and reuse whole-body strategies, what task libraries should we teach them next? Paper: https://lnkd.in/e534wGdb Project Page: https://lnkd.in/eK4YExn8 #HumanoidRobotics #LocoManipulation #ReinforcementLearning #TrajectoryOptimization