Last year, Nvidia and its technology partners twice demonstrated the impressive Avatar Cloud Engine (ACE), which aims to breathe more life into NPCs in games. A new tech demo continues at the GTC/GDC.
Covert Protocol
The ACE now includes new animation and voice functions that are intended to enable even more natural conversations and emotional expressions. The new cloud APIs for Automatic Speech Recognition (ASR), Text-to-Speech (TTS), Neural Machine Translation (NMT), and Audio2Face (A2F) enable developers to easily implement intelligent avatars and scale across applications.
In this field, Nvidia is working with Inworld, among others, and the result is a new technology demo called Covert Protocol. It relies on Unreal Engine 5, with Inworld's SDK integrating MetaHuman from Epic, as well as Nvidia ACE technologies such as ASR and A2F. The demo source code will soon be opened to other developers. In the tech demo, for which there is a video, the ray tracing backgrounds are of course preferably rendered by RTX graphics cards.
Recommended editorial contentHere you will find external content from [PLATTFORM]. To protect your personal data, external integrations will only be displayed if you confirm this by clicking on “Load all external content”:Load all external contentI agree that external content will be displayed to me. This means that personal data is transmitted to third-party platforms. Read more about our privacy policy.
In the demo, the player takes on the role of a private detective who talks to other characters and completes tasks based on the conversations. The interactions with the NPCs happen in real time and are an important part of the gaming experience. Each “run” in the Covert Protocol tech demo is intended to be unique and produce different results. The level of AI-driven interactivity and freedom of action is intended to open up new possibilities “for emergent gameplay, where players must think quickly and adapt their strategies in real time to master the intricacies of the game world.” Role-playing games in particular could benefit from this.
Recommended editorial contentHere you will find external content from [PLATTFORM]. To protect your personal data, external integrations will only be displayed if you confirm this by clicking on “Load all external content”:Load all external contentI agree that external content will be displayed to me. This means that personal data is transmitted to third-party platforms. Read more about our privacy policy.
Lip sync and facial animations
In addition to Covert Protocol, Nvidia also offers two other examples for ACE or Audio2Face. World of Jade Dynasty from Perfect World Games is an upcoming MMORPG based on Unreal Engine 5. A video shows Nvidia's offline rendered Audio2Face technology, which generates precise lip synchronization in both Chinese and English and is intended to enable script writers , easily add new voiced content to World of Jade Dynasty.
Recommended editorial contentHere you will find external content from [PLATTFORM]. To protect your personal data, external integrations will only be displayed if you confirm this by clicking on “Load all external content”:Load all external contentI agree that external content will be displayed to me. This means that personal data is transmitted to third-party platforms. Read more about our privacy policy.
RealityArts Studio and Toplitz Productions' Unawake uses Audio2Face to create characters' facial animations during gameplay and in high-resolution cinematics. So far, the action role-playing game based on Unreal Engine 5 is scheduled for the second quarter of 2024. Nvidia confirmed DLSS 3 with frame generation for release.
What do you think of the technology shown for lifelike characters? Use the comment function and let us know your opinion. To comment you must be logged in to PCGH.de or the Extreme forum. If you don't have an account yet, you could consider registering, which has many advantages. Please note the applicable forum rules when commenting.