Nvidia is helping humanoid robots learn through Apple Vision Pro instruction

Robotics have been a foundational element of Nvidia’s stratospheric growth in recent years. When the chip maker announced GR00T in March of last year, it was heralded as a watershed moment from humanoid robotics. Most of the category’s biggest names were available from launch, including, 1X Technologies, Agility Robotics, Apptronik, Boston Dynamics, Figure AI, Fourier Intelligence, Sanctuary AI, and Unitree Robotics.

At CES on Monday, CEO Jensen Huang unveiled another essential aspect of GR00T: Blueprint. The new modality is based around imitation learning, a fundamental aspect of teaching robots a new skill. It’s doubly important in the world of humanoids, whose designs are based on the one doing the teaching.

Imitation learning is what it sounds like. A person performs an action and the robot follows suit. It’s an effective method of educating systems that are designed to automate existing tasks, like those in factories and warehouse, where the first several rounds of humanoids are being deployed.

Teleoperation has a fundamental role to play in all of this. It can be used to teach robots remotely, instantly digitizing one’s actions through an approximation of a real-life environment.

With GR00T Blueprint, users can create these actions by using Apple’s Vision Pro. The process is captured as a digital twin, which the robot can then repeatedly execute in simulation.