Nvidia teases video game characters we can chat with at Gamescom 2024

At Gamescom 2024 in Cologne, Nvidia has today showcased what it sees as the future of gaming. Mecha BREAK will be the first game to use Nvidia ACE and digital human technologies to bring digital humans to life.

While use use of AI image generators like Adobe Firefly and Flux has exploded for the creation of 2D images, Nvidia ACE uses generative AI to allow players to interact with in-game characters. The initial glimpse of the showcase at Gamescom shows a player engage in conversational interactions in Mecha BREAK, promising a more immersive gaming experience.

We've already had a taste of some of Nvidia ACE's capabilities ourselves, but the example above is Nvidia's first showcase of ACE-powered game interactions within a game. Developed by Amazing Seasun Games, the upcoming Mecha BREAK is a multiplayer mech game that allows players to choose from diverse mechs, customise appearances and battle war machines.

The ACE showcase demo allows players to interact with game characters using natural language. Players can ask for advice on objectives, the ideal mech for a task and get their mech’s paint job updated for "maximum battlefield bling". Nvidia Audio2Face-3D NIM and Whisper, OpenAI’s automatic speech recognition model, provide facial animation and speech recognition running on device, while Elevenlabs powers the character voice through the cloud.

Nvidia ACE is a suite of AI-driven digital human technologies. It includes Nvidia's first digital human technology on-device small language model, Nemotron-4 4B Instruct, which is designed for role-playing, with retrieval-augmented generation and function-calling capabilities, allowing game characters to respond to gamers, more intuitively understand instructions and perform more accurate and relevant actions.

The game publisher and developer Perfect World Games is also making use of Nvidia ACE and digital human technology in Legends, now enhanced with AI-powered vision capabilities that allow characters to see gamers and identify people and objects in the real world through a computer’s camera.

Mecha BREAK will launch in 2025, players can sign up for future playtests on the game’s Steam page.

Get the Creative Bloq Newsletter

Daily design news, reviews, how-tos and more, as picked by the editors.

Also at Gamescom, Nvidia has announced that 20 new games are adding DLSS and RTX technologies. Upcoming titles with ray-tracing, DLSS 3 and Reflex include Star Wars Outlaws, which will give GeForce RTX players more frames and low latency (see our Star Wars Outlaws hands-on review. Dragon Age: The Veilguard is launching with Ray Tracing, DLSS 3 and Reflex, and Dune: Awakening is launching with DLSS 3 and Reflex.

Thank you for reading 5 articles this month* Join now for unlimited access

Enjoy your first month for just £1 / $1 / €1

Already have an account ? Sign in here

*Read 5 free articles per month without a subscription

Joe is a regular freelance journalist and editor at Creative Bloq. He writes news, features and buying guides and keeps track of the best equipment and software for creatives, from video editing programs to monitors and accessories. A veteran news writer and photographer, he now works as a project manager at the London and Buenos Aires-based design, production and branding agency Hermana Creatives. There he manages a team of designers, photographers and video editors who specialise in producing visual content and design assets for the hospitality sector. He also dances Argentine tango.