AI and Machine Learning Applications in Visual Effects

🧠✨ Blending AI with VFX (02) – Real-world Applications in Flame + ComfyUI + SynthEyes: Extracting 3D Geometry from Image How many times in VFX do we wish we had access to the actual geometry of a scene element… and there’s no one to ask? no 3D team, no scan, no model, just the plate. Well, that’s changing, check some quick examples I made. We can now combine AI-based img to 3D extraction in ComfyUI, camera tracking with Syntheyes from Boris FX , and compositing in Autodesk Media & Entertainment Flame. The result? A smarter, faster, more creative pipeline that makes a huge difference when we deal with product replacement, lighting, texture projection or particle interactions. Modeling a complex object is time-consuming, but now we can generate a 3D geometry from a single image using AI tools. This not only speeds up tasks like screen replacement, glow projection, and interactive FX... it also makes things possible that might’ve been skipped due to time constraints. 🔧 The process: 1️⃣ The shot is tracked in SynthEyes, a classic that’s now embracing AI beautifully: with its own machine learning matte generator, there's no need to roto for matchmoving. You can focus entirely on the track — as it should be. 2️⃣ Grab a carefully selected still from the shot and generate a 3D model using AI-based geometry estimation. I’m currently using the Hunyuan workflow, and after testing, around 80% of the meshes were either good enough or straight-up amazing. The sheer amount of usable geometry you can generate in a single day is… dazing. 3️⃣ Mesh and tracking data are imported into Flame, where compositing begins, now with actual geometry positioned in 3D space using the same camera from the shot. This gives you the power to relight, project textures, and drive particle systems with total control of the scene. ⏱️ Time saved is real, especially when the subject isn’t a cube — but a custom product, a car, or even a human body. And the best part? Quality improves too. Faster and better? That’s a rare win-win in post. 🛠️ What can we do with this geometry? 🔹 Realistic product relighting or replacement 🔹 Texture and reflection mapping 🔹 Particle interaction 🔹 Scene reconstruction for set extensions 🔹 Projection-based cleanup & paintwork 🔹 Custom glows, shadows & light wraps that stick The pipeline is smoother and faster than you'd expect. The results are production-ready. And the creative potential? Only growing... Hunyuan shows amazing understanding of shape and occlusion but there’s always a next gen coming: Amodal3R (code soon to come) looks very smart at predicting occluded elements... and this just keeps evolving. Just another reminder: AI isn’t a buzzword — it’s a tool. When paired with the right software and real VFX experience, it becomes a true force multiplier. More to come! 😉 🐵🚀🪐 #VFX #AI #Compositing #AutodeskFlame #SynthEyes #ImageTo3D #Relighting #VirtualProduction #3DGeometry #VFXWorkflow #Hunyuan #Nuke #ComfyUI

Are AI Tools Good Enough for Real-World Applications? As an early adopter of new technologies, I've consistently integrated AI into my projects. I'm excited to share some recent outcomes where I've brought AI-driven 3D generations into my VFX workflow. 🎨 Starting with Innovation: I kicked off the project by using OpenAI DALL-E 3 to generate innovative 2D graphics to replace the contents of a bus ad. The AI's ability to interpret and visualize creative prompts offered me a selection of visuals to choose from. 📦 AI Texturing: The same AI-generated images were transformed into a realistic package design, textured over 3D models which I crafted in Blender. This integration showcases how AI can enhance the visual touches in a project. 🐻 Text to 3D: For the central piece of the VFX, I created a 3D gummy bear using Genie from Luma AI purely from a text description. Although the initial texture wasn't perfect, I tweaked it in Blender to achieve a gelatin-like look, ensuring the 3D form was just right. 🎬 Animation and Composition: Adding bones and animating the scene in Blender, followed by compositing everything in After Effects, allowed me to focus more on the technical details and composition, which are crucial in VFX projects. 🤖 The AI Advantage: The integration of AI tools into VFX not only reduces the time needed to create complex visuals but also broadens the creative possibilities, especially for those with technical skills but without an artistic background. This doesn't replace the creator, but instead provides them with more powerful tools to enhance the concept. Have you tested AI tools for your creative processes? Let me know which tools you’ve you have found to work best and adopted into your workflow.

I have said for at least a year that generative AI will heavily impact movie and TV production because the output of those industries is moving pictures and text, exactly what generative AI produces. This article suggests this impact might be happening faster than I thought. Generative AI “has already rendered dubbing and translation work nearly obsolete. Visual effects artists, perennially on the bleeding edge of new #technology for #Hollywood, are already working with #machinelearning and some generative #AI, particularly for pre-production visualizations and workflows. “From an artist’s perspective, we’re all trying to get ahead of the game and play with open source tools that are available,” said a cinematographer and director whose credits include The Mandalorian, Black Adam and Fallout. Jim Geduldick, a VFX supervisor and cinematographer has used AI to de-age its stars Tom Hanks and Robin Wright in an upcoming film (picture in post). He says: “Everybody’s using it. Everybody’s playing with it.” Geduldick “sees current generative AI models as more “assistive” than truly imitative of human art. “Are they implementing generative models that are going to speed up both the business and the #creative side of what we’re doing? Yes,” he said, “making pre-visualization cheaper and more efficient, streamlining tedious processes, shaping storyboard design.” “But I think that there is no generative model out there today that doesn’t get touched by artistic hands to get it to the next level. That is for the foreseeable future.” Geduldick is also a critic of #tech companies and their “lofty rhetoric, and soulless AI content produced for content’s sake.” He says they have “repeatedly shown in their public-facing interviews or marketing that there’s a disconnect [in] understanding what creatives actually do,” referring to OpenAI’s comment that generative AI might eliminate some creative jobs, “but maybe they shouldn’t have been there in the first place”. He says: “Film-making is a collaborative thing. You are hiring loads of talented #artists, technicians, craftspeople to come together and create this vision that the writers, director, showrunners and producers have thought up.” Other film-makers are using “generative AI as an anonymization device,” in documentaries, “to map actors’ faces over real subjects who faced harrowing violence.” Others do this for “the victims of nonconsensual, deepfake pornography.” For industries that aren’t selling moving pictures or text, the impact of generative AI will be much smaller. Although it will be used for making video ads, generative AI will probably not have a big impact on the productivity of companies manufacturing products, constructing buildings, mining minerals, raising or catching fish, producing or distributing energy, or even offering healthcare. #technology #innovation #startups #artificialintelligence https://lnkd.in/gqMA8zbB

The Future of Makeup & FX Testing Has Arrived** 💄 Imagine being able to test *dozens* of makeup, hair, and special effects looks — instantly — without booking a full team, setting up a studio, or spending thousands on materials. Thanks to new AI-driven visualization tools, studios and creatives can now explore character looks *on the fly.* Whether you’re a director, producer, or concept artist, you can quickly mock up: ✨ Creature transformations ✨ Hairstyles ✨ Aging or injury effects ✨ Entire new character concepts Before a single brush touches an actor’s face, you can visualize, refine, and align on the creative direction — saving time, budget, and countless hours of back-and-forth. Even better, these tools help **directors and producers clearly guide their vision** to the hair, makeup, and special effects teams. With a shared visual starting point, everyone can collaborate faster, more creatively, and with total confidence in the look they’re chasing. It’s not about replacing artists — it’s about empowering them with a smarter way to experiment, iterate, and bring ideas to life. Once the vision is locked, the artistry of the makeup and FX teams can shine brighter than ever. The imagination-to-screen gap just got a whole lot smaller. 🚀 #FilmProduction #VFX #MakeupDesign #InnovationInFilm #AIDrivenCreativity #StudioTools #CharacterDesign #Filmmaking #AI #deepfake #genai

5 ways we leverage AI in animation production to increase creativity and speed up our process: 👨🚀 Omniverse and Audio2Face by NVIDIA: These tools allow us to quickly create facial expressions on 3D characters, such as MetaHumans, based on audio recordings. This brings characters to life in a matter of minutes. 👨🚀 Move AI: By analyzing movement in live-action footage, Move AI can convert it into motion capture data that we then can apply to our 3D character rigs. Huge time saver compared to manual animation. 👨🚀 Wonder Dynamics: This browser-based VFX tool enables us to replace people in live-action footage with 3D characters within minutes. This eliminates the need for costly and time-consuming manual labor. 👨🚀 Midjourney: This AI-powered tool enhances the ideation and art direction phase in pre-production. It provides new perspectives and references, allowing us to visually "daydream" together with AI. 👨🚀 Voice-over AI: We now have various AI tools that can generate voiceovers, and the quality has significantly improved. These tools produce natural-sounding voices that can be directed, helping our clients save money on voiceover licenses. By incorporating these AI technologies into our animation production process, we streamline workflows, enhance creativity, and deliver high-quality results efficiently. #ai #animation #content #production Links in comments 👇

📺 Just dropped: Episode 53 of The Shift AI Podcast featuring Matthias Wittmann, VFX Supervisor at Digital Domain, exploring the future of AI in Hollywood and beyond. From his Oscar-winning work on "Benjamin Button" to creating truly autonomous virtual humans like Zoey, Wittmann shares fascinating insights on how AI is revolutionizing visual effects while highlighting the critical distinction between pre-recorded digital likenesses and genuinely autonomous digital humans. In the podcast, Wittmann explores the complex technical challenges behind creating responsive, autonomous virtual humans, how AI-powered tools are transforming animation and VFX workflows, and the surprising potential applications for autonomous characters beyond entertainment - from education to companionship for the elderly. Don't miss this deep dive into the cutting edge of digital human technology with one of the industry's pioneering innovators. Check it out on YouTube - https://lnkd.in/gHkx3ChD Check it out on Spotify - https://lnkd.in/gpuPfdkr

Netflix just used generative AI in production for the first time - and it's about enhancement, not replacement.🚀 Their sci-fi series "El Eternauta" used generative AI for VFX - completing a building collapse sequence 10x faster than traditional methods. The kicker? This show literally wouldn't have been possible at its budget without AI. Netflix co-CEO Ted Sarandos nailed it: "AI represents an incredible opportunity to help creators make films and series better, not just cheaper." 🎯 The real story: AI is a creative multiplier. Small productions can now achieve big-budget visuals, freeing up resources for what matters most: compelling human storytelling. Bottom line for digital creatives: The future isn't human vs. AI. It's humans + AI tools pushing creative boundaries further than ever before. Don't ask whether you should embrace these tools - ask how fast you can master them to make your vision. 💪 #genai #creativity #ai #creativetech #Netflix #VFX #Storytelling

When AI first started showing up in the VFX world, I’ll be honest I had my doubts. Was it just hype? A threat to our jobs? Another “innovation” that wouldn’t stick? Fast forward to today: I couldn’t have been more wrong. AI isn’t here to replace us, it’s here to supercharge us. In my Studio, me & my team have started integrating AI into parts of our workflow, and the results have been eye-opening. Here’s what’s shifted: 𝐑𝐨𝐭𝐨𝐬𝐜𝐨𝐩𝐢𝐧𝐠 𝐚𝐧𝐝 𝐜𝐥𝐞𝐚𝐧𝐮𝐩? AI now takes care of those time-consuming tasks in a fraction of the time. Less time clicking. More time creating. 𝐌𝐨𝐫𝐞 𝐫𝐨𝐨𝐦 𝐟𝐨𝐫 𝐬𝐭𝐨𝐫𝐲𝐭𝐞𝐥𝐥𝐢𝐧𝐠. 𝐌𝐨𝐫𝐞 𝐟𝐫𝐞𝐞𝐝𝐨𝐦 𝐭𝐨 𝐞𝐱𝐩𝐞𝐫𝐢𝐦𝐞𝐧𝐭. The creative space we’ve always wanted is suddenly within reach. It feels like we’ve added a team member who doesn’t sleep, never complains, and is amazing at the repetitive stuff, so we don’t have to be. For anyone still unsure about bringing AI into your creative process, here’s my honest advice: 𝐒𝐭𝐚𝐫𝐭 𝐬𝐦𝐚𝐥𝐥. 𝐓𝐫𝐲 𝐨𝐧𝐞 𝐭𝐨𝐨𝐥. See how it fits into your workflow. You might be surprised how quickly it becomes essential. Now I’d love to hear from you: 𝐖𝐡𝐚𝐭’𝐬 𝐨𝐧𝐞 𝐩𝐚𝐫𝐭 𝐨𝐟 𝐲𝐨𝐮𝐫 𝐜𝐫𝐞𝐚𝐭𝐢𝐯𝐞 𝐩𝐫𝐨𝐜𝐞𝐬𝐬 𝐲𝐨𝐮 𝐰𝐢𝐬𝐡 𝐀𝐈 𝐜𝐨𝐮𝐥𝐝 𝐡𝐞𝐥𝐩 𝐰𝐢𝐭𝐡? Drop it in the comments. Let’s learn from each other because the future of VFX is moving fast, and we’re all in this together. #VFX #Aitechnology #Ai #AR #tech