NVIDIA launches RTX-based AI for live streamers

NVIDIA is giving live stream enthusiasts and software developers the tools to create a world full of augmented reality effects, style filters and virtual greenscreens.

Announced at TwitchCon, NVIDIA released details of the RTX Broadcast Engine, which is a new set of software development kits that use AI capabilities to transform live streams.

The engine draws on GeForce RTX GPUs and AI processors called Tensor Cores to create effects without the need for specialised equipment.

It's big business: According to NVIDIA, more than 750 million people across the world will tune in to watch other people play video games.

The RTX Broadcast Engine will work across Twitch Studio, Discord, and Elgato, as well as XSplit, Huya, Douya and Streamlabs.

"The new RTX Broadcast Engine is an exciting advancement that will allow developers in our app store to create powerful new tools for streamers with NVIDIA RTX GPUs," says Streamlabs CEO Ali Moiz.

"We're thrilled to continue working with NVIDIA as they introduce new features to the Streamlabs developer community, and look forward to implementing this new technology.

NVIDIA is working with live stream app OBS to integrate RTX Greenscreen.

"NVIDIA has been at the top of my list when it comes to streaming and recording equipment. I'm continually impressed with what they're doing. And their technology is impressive with RTX features like RTX Greenscreen," comments OBS author Hugh Bailey.

NVIDIA RTX Broadcast Engine features at a glance:

• RTX Greenscreen, to deliver real-time background removal of a webcam feed, so only your face and body show up on the livestream. The RTX Greenscreen AI model understands which part of an image is human and which is background, so gamers get the benefits of a greenscreen without needing to buy one.
• RTX AR, which can detect faces, track facial features such as eyes and mouth, and even model the surface of a face, enabling real-time augmented reality effects using a standard web camera. Developers can use it to create fun, engaging AR effects, such as overlaying 3D content on a face or allowing a person to control 3D characters with their face.
• RTX Style Filters, which use an AI technique called style transfer to transform the look and feel of a webcam feed based on the style of another image. With the press of a hotkey, you can style your video feed with your favourite painting or game art.

Developers need to apply for early access through NVIDIA's website.