NVIDIA releases Audio2Face AI as open-source

NVIDIA has open-sourced its Audio2Face AI tool that generates lifelike facial animation from voice input, giving developers new ways to create realistic 3D avatars.

Nvidia open-sourcing Audio2Face, their tech that turns speech into real-time facial animations. This isn’t just a fun demo — it could reshape how we build digital characters, avatars, and even video content with true expressiveness. The ones who lead won’t just use the tools — they’ll guide them with vision, responsibility, and thoughtful strategy and alignment. https://lnkd.in/gqy6-XDS

Cool stuff. NVIDIA just open sourced Audio2Face, a generative AI tool that converts speech into realistic facial animation and lip-sync—in real-time. 3D developers can now: Animate characters directly from audio, with emotion and accuracy: https://lnkd.in/gh2pm_-H

Nvidia is letting anyone use its AI voice animation tech - Nvidia is open sourcing Audio2Face, its AI tool that generates facial animations for 3D avatars based on audio input. https://lnkd.in/g_y7im6X via The Verge #NVIDIA #AI #Audio2Face

NVIDIA has made its AI-powered Audio2Face tool open-source, allowing developers to create realistic facial animations for 3D avatars using audio input. . . Click here to read the full story: https://lnkd.in/dmGB6wsH Now follow our WhatsApp Channel, and never miss out on daily tech brew https://lnkd.in/dYa9pvpx #TheMainstream #AIAvatar #FacialAnimation #VoiceDrivenAI #GenerativeTech #3DAnimation

🚀 NVIDIA recently made a big move: Audio2Face is now open source. This could change how we build avatars, virtual presenters, and game characters, making advanced animation tools accessible to anyone. ➡️ Watch the video to see why this is a game-changer for creators. 👉 Tap follow for no-fluff AI news & insights from MIT’s Gen AI Global founder Oscar Sanchez 🧠 #ArtificialIntelligence #NVIDIA #Audio2Face #AIAnimation #GenAIGlobal

Playing with the edges of what’s possible with generative AI 🤖 Just read about Nvidia open-sourcing Audio2Face, which turns speech into realistic facial animation + emotions. Why this matters: • Brings life like avatars and digital characters within reach of more creators. ✨ • Opens new possibilities in gaming, VR/AR, virtual customer service, storytelling.👩🏽💻 • But also raises questions: how do we make sure emotional cues are accurate & ethical?🤔 Check out the article on their blog: ⤵️ https://lnkd.in/d-yiVWdv

Here is another Unity URP/SRP Post Processing effect. This is a World Freeze effect animation, where along 10 seconds a scene starts to get frozen totally. It is really interesting how it is made because in Post Processing effects as a Renderer Feature we don't have access to informations about meshes and 3D world positions of each object on the scene. We could use traditional surface shaders, but we would need to add/replace materials in each object in the scene and that is very inefficient. So how it is made? ❄️ World Position Reconstruction: For each pixel on the screen we take its depth and we calculate pixel representation in 3D space. ❄️ Normals Reconstruction: From depth of neighbours we calculate normal vector. Thanks to that, we know in which direction each surface "looks at". ❄️ Triplanar Mapping: Ice texture is applied not in screen space, but in world space. Each pixel gets uv coordinate calculated from each own world space (X/Y/Z axis projection). Thanks to that, when we move our camera, an ice pattern stands in its place and simulates like it would be sticked to world objects. There is a disadvantage of that solution. It only works on objects that are static and are not changing its position dynamically, because a texture won't be sticked to the moving object. But it is a world freeze effect and we can assume, that frozen objects won't move anymore. Despite that if we still want some moving objects to be at least partly frozen (like I did with a player character) we still can achieve that and make another Renderer Feature that works as a mask and it excludes any objects we want from the World Freeze effect by layers. When our player is excluded from the World Freeze effect, we apply to the player traditional surface shader with the same texture, but with less final strength (For the player to not be completely frozen). So it is a combination of 3 shader effects: World Freeze Post Processing effect Renderer Feature, Exclude Mask Renderer Feature, Single Object Surface Shader. Shaders also need to be additive, we don't want to replace a material, but enhance an object and add another overlay for the effect to be more realistic. Of course we can add later additional effects like particle system blizzard or snow for the feature to be even more realistic, but this solution is a really nice start. What do You think? 🙂

Experimenting with new generative workflows in search of a low-fi, 90's, PlayStation 1 look. The underlying level layout only took about 25 minutes to block in using Adobe Substance 3D Modeler in #VR. I dropped that layout into #UnrealEngine to add lighting and a 1st person controller rig. To quickly flesh out the environment and start testing looks, I used krea.ai to train a custom model which helped me generate a library of textures and 2d assets that were stylistically consistent. Then I used Tripo AI to generate 3d models based on some of that 2d output, which helped me populate the space in record time. Lastly, some camera and post effects visually glued the whole thing together. I want to incorporate #AI tools in a way that lets me be more efficient without losing creative control. Simply prompting and hoping for the best is not going to cut it for most professionals. This whole virtual environment only took about a day and a half to put together which makes this combination of tools feel really promising.