Training humanoid robots to perform a variety of actions consistently is challenging and time-consuming. Nvidia researchers have developed a general-purpose artificial intelligence-powered SONIC controller that can learn from large and diverse human motion data. The controller converts commands from virtual reality, pre-recorded videos or text prompts into smooth, coordinated movements of the robot.
Image source: NVIDIA
“Humanoid robots must have the same coordination and adaptability as humans, but current systems use separate controllers to train different skills, making them difficult to scale.” – said Yuke Zhu, principal researcher at Nvidia. — “We wanted to see if a well-trained controller could provide the basis for multiple movement options involving the entire body.”
Zhu and his colleagues set out to create a universal controller for controlling robot motion that could learn from large and diverse human motion data. Additionally, they try to ensure that the controller is able to generalize the knowledge gained during training and apply it to perform new actions and actions. According to Zhu, “This approach could bring humanoid robots closer to creating general motor intelligence suitable for a variety of tasks, eliminating the need for engineers to develop separate controllers for each specific operation.”.
The SONIC controller developed by the team is a basic model for controlling the movement of humanoid robots. It acts as the robot’s “locomotion system,” translating high-level commands into smooth, coordinated movement of the entire body. The model was trained on over 100 million frames of human motion, allowing it to learn a wide range of motion patterns, not just a single task.
Developers tested SONIC in an entity simulator using a variety of motion and object manipulation scenarios, including actions not included in the training set. The same controller can handle different types of input, whether it’s VR remote control, video-based motion or commands from a multi-modal model, without the need for retraining.
The researchers tested the controller on a humanoid robot, assessing the accuracy of reproduction of learned and new, previously unseen reference movements. The SONIC system was found to provide a wide range of natural and sustainable movements throughout the body, relying on a single model rather than separate control strategies for each specific movement. Combined with add-ons, the controller can control the robot remotely and convert text commands into specific actions.
“In the future, this approach could accelerate the development of humanoid robots for manufacturing, warehouse and logistics tasks, and other fields that require a variety of physical actions. More broadly, SONIC serves as a universal motor control framework that can interact with higher-level artificial intelligence models and translate logical reasoning into reliable physical actions.”” Zhu concluded.
For now, Zhu and his colleagues plan to continue developing and improving their model. In particular, they hope to improve the system’s ability to sense its surroundings to reduce the risk of collisions and accidents, and improve the robot’s ability to navigate dynamic environments or rough terrain. In the future, there are plans to integrate SONIC with the Nvidia Isaac GR00T platform, which will allow humanoid robots to combine advanced logical thinking with universal control of whole-body movements.
If you find an error, select it with your mouse and press CTRL+ENTER.
