NVIDIA recreats Jensen Huang’s virtual avatar to speak at GTC 2021

This spring, as part of the opening of the GTC 2021 conference, NVIDIA CEO Jen-Hsun Huang held an hour and a half press conference in his kitchen, talking about the company’s new products. But now, during the SIGGRAPH 2021 conference, NVIDIA revealed that Huang’s April press conference was not entirely real. The interior of the kitchen, Jensen Huang, his jacket, gestures, facial expressions, hair – all this was artificially recreated using computer graphics. In any case, during a small fragment of the presentation.

Just at the computer graphics conference SIGGRAPH 2021, Nvidia revealed through a documentary that April’s press conference was a “fake” one. In the screen, Jensen Huang’s kitchen, his leather clothes, even his expressions, movements, hair… are all synthetic#NVIDIA #SIGGRAPH pic.twitter.com/fyrb8wgCYa

– Jacobson (@jakobsonradical) August 12, 2021

To recreate a virtual copy of Huang, the NVIDIA Omniverse platform was used, designed for collaboration in a virtual environment and physically correct simulation in real time. It includes tools for creating animations.

Initially, experts digitized the head of the company using a variety of digital SLR cameras. A digital copy of Huang’s kitchen was also created. Then a 3D model of the NVIDIA head was created. In the future, it was necessary to train the artificial intelligence system, which made it possible to create a realistic imitation of Huang’s gestures and facial expressions. Then the virtual Huang was added to the image of the virtual kitchen. In the end, a fairly realistic video of the speech of the head of the company turned out, in which the virtual avatar was indistinguishable from a real person.

“To create a virtual Jensen, the teams performed a full face and body scan to create a 3D model, then trained the AI ​​to mimic his gestures and expressions, and used AI magic to make his clone realistic,” NVIDIA wrote on its blog. “Digital Jensen was then transferred to a copy of his kitchen, which was deconstructed to show the holodeck on the Omniverse, which surprised the audience and made them question which part of the performance was real and which was recreated.”

It should be noted that the virtual Jensen Huang was present in the frame only for a short time – only 14 seconds in the time interval from 1:02:41 to 1:02:55. The real Huang conducted most of the press conference on his own.

NVIDIA has also posted a video detailing how their team made a presentation at GTC 2021. The video also includes a snippet on the creation of Jensen Huang’s virtual and his virtual kitchen.

Source: vice