Building an AI Sound Design Pipeline Inside DaVinci Resolve | BC+AI Film Club April 2026
· 5:23
- Recorded at
- AI Film Club: 04/09
More from this event
Notes & highlights
Kevin Friel describes, rather than demonstrates, a custom sound-design pipeline built around DaVinci Resolve. He explains that the planned live demo was unavailable and that he would publish a separate demonstration later at 00:07.
Friel says his script connects the DaVinci Resolve, Google Gemini and ElevenLabs APIs at 00:53. In his account, Gemini scans the picture and writes prompts for Foley, atmosphere and other sound layers; ElevenLabs returns individual audio files; then the helper names, bins and places those files against the timeline at 01:48.
He reports that a first pass can contain about 70 tracks and estimates that roughly 30 percent may be unusable, leaving an editor to subtract and refine at 02:22. He also describes Resolve Studio's AI audio mixer as a starting point that categorizes tracks and applies processing which remains adjustable at 02:46. Near the end, Friel again states that the recording contains no visual proof of the pipeline at 05:03; the performance figures above remain his report.
Explore more Film Club practice, upcoming Film Club events, and the complete video archive.
Transcript (auto-generated)The full spoken text of this recording
YouTube generated this transcript from the recording. Kris Krüg checked the names in it against the recording. The wording and punctuation are still the machine’s, so quote from the video itself rather than from this page.
0:01[music] >> So, we're uh we're pretty much wrapped up, folks. Um I was going to show a demo of a tool that I vibe coded, but um the tool be broken uh in its alpha test phase. We had some terrible issues with internet this morning, and we're running a little short on time. So, what I'm going to do is record a real quick uh demo video and put that out on LinkedIn on our uh WhatsApp channel as well. Um the nuts and bolts of it is is I have a requirement for um a film that I'm post supervising on right now. Um it's going to be shown on Monday, >> [laughter] >> and we have several sequences that don't
0:40have sound design yet. The edits are going to be ready tomorrow morning. So, I saw this in advance and wanted to prepare, you know, my team to just get sound design as quickly as possible. So, I was aware that there was an API inside of DaVinci Resolve. I was aware of an API on Google Gemini. I was aware of an API on 11 Labs, and I made a little um uh script and MCP server and API call around that analyzes video, scans the video like um the best Hollywood sound designers around. I gave it the creative scope um of legendary sound designers like Ben Burtt and Gary Rydstrom, etc., etc. It crafts prompts in that context
1:25for um foley, for atmos, for, you know, all the layers of sound design that that you really want to get out of uh a tremendous visual project. And it will create prompts based on what it sees in the video. It will also analyze the the audio, too, but I haven't I haven't even tried that yet. I just wanted to see what it could get in the visual sense.
1:48Gemini scans it, gives those prompts. The helper tool sends those prompts to 11 Labs. 11 Labs API brings back all of those, uh, elements, the individual wave files, and then the helper script with all the correct file names, the correct binning, brings them into DaVinci Resolve and puts them time code accurately to what it saw in the timeline. And it works really well when you're hardwired, but not over Wi-Fi. So, >> [laughter] >> I will put a clip out to, to show you that. The stuff that comes out of it is is blowing my mind already. Just getting your first pass sound design with 70 layers of track sometimes. You know, maybe 30% of those are duds, but you you
2:33start to take away and you have so much richness in it already. So, that's that's my like blind explanation of it, but I'll post a clip about it and and get that in front of y'all. The the real cherry on top of that is I also discovered DaVinci Resolve's um, native AI auto mixer to, tool which is available in the studio platform of DaVinci Resolve as well.
2:57It will analyze all of your audio tracks, categorize them, and give you a really solid first mix to start tweaking with. It will take into consideration your dialogue, it will take into consideration your music cues, and all of your music beds that are in there. It will apply multi-band compression, it will apply um, voice isolation if there's any noise inside of the footage as well. These are all completely pliable, tweakable after the fact, tools inside of DaVinci Resolve. So, this film that I'm working on has really solidified even more to me. I am a DaVinci Resolve zealot, but I am a thousand percent more so now. I am color grading and compositing on the fusion
3:48page tab in 2K and 4K and it all syncs and talks to each other. We have five editors working on this right now across, you know, completely different boxes. We're working across Blackmagic Cloud and it is working spectacularly well across editorial, narrative, sound design, color, and now compositing on my side as well of AI generated elements for visual effects augmentation. So, I'm I'm really encouraging everyone to if you've been kind of nervous about getting into DaVinci Resolve, it is so phenomenal how they've been able to bring everything out in one unit. It's a one-time fee for the studio edition as well. It's about 299 American, so just about 400 Canadian.
4:38But, that is forever. You get updates forever out of it. It's not a perpetual fee. And I'm telling you if you're curious about vibe coding at all, if you've spent 10 minutes in Claude code, the API in DaVinci Resolve is so malleable and flexible to what you want it to do, you can do it pretty much anything. So, and you can build your own pipeline with it basically. So, I don't have visuals to show you that it actually proves that it works it, but I will give you real ones not generated visuals. I swear on my heart that they will be real and I can't wait for you to see this and get your feedback on