NVIDIA has unveiled a range of technologies aimed at advancing humanoid robot development, including the launch of the Isaac GR00T N1, the first open, customisable foundation model designed for humanoid reasoning and skills.
NVIDIA's latest offerings include simulation frameworks and development materials such as the NVIDIA Isaac GR00T Blueprint for synthetic data generation, alongside Newton, an open-source physics engine created in partnership with Google DeepMind and Disney Research, specifically for robotic applications.
The GR00T N1 model, which is now available, is the first in a series of models that NVIDIA plans to release to robotics developers globally. NVIDIA anticipates this will aid industries facing significant labour shortages, projected at over 50 million individuals worldwide. "The age of generalist robotics is here," said Jensen Huang, Founder and CEO of NVIDIA. "With NVIDIA Isaac GR00T N1 and new data-generation and robot-learning frameworks, robotics developers everywhere will open the next frontier in the age of AI."
The GR00T N1 model operates on a dual-system architecture, reflecting the cognitive processes of humans. "System 1" serves a reflexive, rapid-response function akin to human intuition, while "System 2" is designated for careful, deliberative decision-making. System 2 utilises a vision language model to interpret environmental cues and directives, with System 1 executing these plans through precise robotic motions, informed by both human demonstration data and synthetic data from the NVIDIA Omniverse platform.
This model is designed to undertake a range of tasks, from basic functions such as grasping and object movement to more complex, multi-step tasks. These capabilities are anticipated to be valuable in sectors such as material handling, packaging, and inspection. Developers and researchers have the option to further refine GR00T N1 using real or synthetic data tailored to specific robots or tasks.
In a presentation, Jensen Huang illustrated 1X's humanoid robot executing domestic cleaning tasks autonomously, using a post-trained policy built on the GR00T N1 model. "The future of humanoids is about adaptability and learning," stated Bernt Børnich, CEO of 1X Technologies. "NVIDIA's GR00T N1 model provides a major breakthrough for robot reasoning and skills. With a minimal amount of post-training data, we were able to fully deploy on NEO Gamma — furthering our mission of creating robots that are not tools, but companions that can assist humans in meaningful, immeasurable ways."
Several other humanoid developers, such as Agility Robotics, Boston Dynamics, Mentee Robotics, and NEURA Robotics, have received early access to the GR00T N1 model.
NVIDIA has also teamed up with Google DeepMind and Disney Research to work on the Newton physics engine, aimed at enhancing robots' ability to manage intricate tasks with improved precision. This engine, built on the NVIDIA Warp framework, is designed for robot learning and will function with simulation frameworks like Google DeepMind's MuJoCo and NVIDIA Isaac Lab. Future plans include its integration with Disney's physics engine.
Google DeepMind and NVIDIA are working together on MuJoCo-Warp, a project expected to accelerate robotic machine learning by over 70 times. This will be accessible via Google DeepMind's MJX library and Newton. Disney Research intends to utilise Newton for its robotic character platform, contributing to the next generation of interactive entertainment robots. "The BDX droids are just the beginning," said Kyle Laughlin, Senior Vice President at Walt Disney Imagineering Research & Development. "We're committed to bringing more characters to life in ways the world hasn't seen before, and this collaboration with Disney Research, NVIDIA and Google DeepMind is a key part of that vision."
Additional collaboration between NVIDIA, Disney Research, and Intrinsic will focus on developing OpenUSD pipelines for robotics data workflows.
NVIDIA has addressed the need for extensive datasets in robot development with the GR00T N1 blueprint for synthetic data generation. Using NVIDIA Cosmos Transfer models, this blueprint facilitates the creation of vast amounts of synthetic motion data from minimal human demonstrations. This approach has already allowed NVIDIA to generate substantial synthetic data in a limited timeframe, bolstering GR00T N1's capabilities by a reported 40% compared to reliance on real data alone. NVIDIA is also making the GR00T N1 dataset available as part of a larger open-source physical AI dataset.
NVIDIA has made the GR00T N1 training data and task evaluation scenarios available for download, alongside the interactive demo of the NVIDIA Isaac GR00T Blueprint for synthetic data generation. Furthermore, the Newton physics engine is expected to be released later this year.
