December 25, 2022 at 2:45 p.m. by Maximilian Hohm – Nvidia had its own CEO’s Avator, Toy Jensen, sing a version of Jingle Bells in a Christmas video. Several modern AI technologies were used to prevent him from ever having to sing himself. Read more about this below.
Nvidia has shown creativity for the holiday season, uploading a celebratory video of its founder and CEO Jen-Hsun Huang singing jingle bells. But not the CEO himself, but his animated avatar “Toy Jensen” sings. The Christmas carol is interpreted in a more modern way and accompanied by TJ snapping his fingers. But the video isn’t just a goodie from the manufacturer that’s supposed to make people smile around Christmas, it also impressively shows how far the technology has come.
Toy Jensen is a creation of Nvidia’s Omniverse Avatar Cloud Engine (ACE). According to Nvidia, the vocal insert only took a few hours to render with the help of the ACE and shows realistic animations on the face in combination with body movements and the vocal insert. The latter is also an Nvidia development. Although Huang’s voice can be heard in the clip, he did not sing the song himself. It was sung by professional singers and its pitch and voice were adapted to it.
However, this is not a pure AI achievement, but the work of Jochem van der Saag, a composer and producer. However, Omniverse Audio2Face and Audio2Gesture are AI-controlled and have delivered a corresponding performance based on the singing, which lets TJ do well. The clip, on the other hand, was prompted by a performance by Jen-Hsun Huang in Taipei in November, where he was involved in the live performance of Lady Gaga’s “Always Remember Us This Way” at an e-sports event. However, he says he’s too shy to sing himself, so his engineers had to use his avatar to do it.
Quelle: Toms Hardware