New Nvidia ACE plugins for Unreal Engine 5 simplify the creation of AI digital humans

At Unreal Fest Seattle 2024, Nvidia announced the release of new plugins designed to allow developers to more easily and efficiently create and deploy of AI-powered MetaHuman characters directly in Unreal Engine 5 for game design on Windows PCs.

The graphics card maker has launched on-device plugins for Nvidia ACE in Unreal Engine 5. There’s also a new Audio2Face-3D Plugin for Autodesk Maya, plus a new Unreal Engine 5 Renderer Microservice with Pixel Streaming.

There are three new Unreal Engine 5 ACE plugins: Audio2Face-3D Plugin, which enables AI-powered lip sync and facial animation directly in Unreal Engine 5, along with Nemotron-Mini 4B Instruct Plugin, which provides response generation for interactive character dialogues and the Retrieval Augmented Generation (RAG) Plugin, which supplies contextual information to enhance character interactions.

And now Unreal Engine 5 Renderer Microservice with Pixel Streaming, which uses Epic’s Unreal Pixel Streaming technology, supports the ACE Animation Graph microservice and Linux OS in early access, allowing developers to stream high-fidelity MetaHuman characters to any device via Web Real-Time Communication (WebRTC).

the new tool aims to simplify the integration of AI-powered digital humans into games and applications and optimise microservices for low latency and minimal memory usage on Windows PCs. It also enhances scalability for cloud-based deployment of digital humans and provides customisable tools and source code for tailored development needs. Developers will be able to apply for early access to download it.

For Autodesk Maya, the new Audio2Face-3D Plugin facilitates high-quality, audio-driven facial animations, provides source code for customisation and integration with other digital content creation tools and streamlines the workflow between Maya and Unreal Engine 5. The Maya ACE plugin is available to download on GitHub.

Get the Creative Bloq Newsletter

Daily design news, reviews, how-tos and more, as picked by the editors.

Thank you for reading 5 articles this month* Join now for unlimited access

Enjoy your first month for just £1 / $1 / €1

Already have an account ? Sign in here

*Read 5 free articles per month without a subscription

Joe is a regular freelance journalist and editor at Creative Bloq. He writes news, features and buying guides and keeps track of the best equipment and software for creatives, from video editing programs to monitors and accessories. A veteran news writer and photographer, he now works as a project manager at the London and Buenos Aires-based design, production and branding agency Hermana Creatives. There he manages a team of designers, photographers and video editors who specialise in producing visual content and design assets for the hospitality sector. He also dances Argentine tango.