Skip to content
Opens in a new window
Everything I learned about working with local AI
08 October 2026

Everything I learned about working with local AI

The Xero for Hire Podcast

About

In this episode of The Xeroforhire Podcast, Xeroforhire shares the lessons, frustrations, and discoveries from building an AI-assisted anime trailer for Harbinger: Legend of the Seven Brothers. After weeks of experimenting with local AI models, downloading massive files, troubleshooting workflows, and generating hundreds of seconds of footage, he breaks down what actually works—and what doesn’t.

The episode begins with an important distinction: artificial intelligence isn’t one technology. Large language models, diffusion models, and AI agents serve different purposes, and understanding those differences is essential when building a creative workflow.

Using ComfyUI on a consumer-grade laptop, Xeroforhire experiments with several video-generation models, including MiniMax H3, Wan, and LTX 2.5. He discusses the limitations of each, the importance of matching software to available hardware, and why polished YouTube demonstrations often create unrealistic expectations for independent creators.

Much of the conversation focuses on practical techniques developed through trial and error. Character reference sheets, carefully structured prompts, visual starting frames, and consistent style instructions become the foundation of his animation process. He explains how overly complicated prompts can produce unexpected results, including an amusing problem where nearly every scene contained a campfire because fire and smoke had been included in his general style instructions.

He also explores the challenge of achieving a specific visual aesthetic. Although his goal is the detailed, hand-drawn appearance of 1990s anime OVAs, H3 frequently produces imagery resembling older television anime. Rather than abandoning the project, he uses stronger visual references and alternative models to improve individual shots.

One of the episode’s most useful discoveries involves random seeds and iteration. By duplicating ComfyUI workflows across multiple browser tabs, assigning individual scenes to each tab, and generating several variations with different seeds, he develops a more efficient production process. This allows him to queue multiple shots, compare results, and select the strongest takes without constantly rebuilding his workflow.

The discussion concludes with practical advice about render times, managing exported files, and using Syncthing to review generated footage on a phone while the computer continues working.

Throughout the episode, Xeroforhire emphasizes the difference between demonstrating AI technology and actually using it to complete a creative project. His first Harbinger trailer is far from perfect, but it represents a working foundation that can be refined through experience.

The central takeaway: Learning AI filmmaking isn’t about finding a magical model that does everything. It’s about understanding the tools, developing a repeatable workflow, managing expectations, and being willing to experiment until the results serve the story.

The episode closes with a preview of a possible second installment covering scripting, shot planning, editing, and other production techniques.

Timestamps

00:00 – Introduction: Everything I’ve Learned About AI

00:43 – AI Isn’t One Technology: LLMs, Diffusion Models, and Agents

01:51 – Getting Started with ComfyUI and the Reality of Large Downloads

03:34 – Hardware Limitations and Unrealistic AI Tech Demonstrations

05:55 – Researching AI Tools and Getting Help from Frontier Models

07:48 – Finding the Right Video Model: H3, Wan, and LTX

10:39 – Building an Animation Workflow with Character Reference Sheets

12:36 – Writing Effective Prompts and Creating Style Preambles

13:20 – The Challenge of Reproducing 1990s OVA Anime

15:01 – Artistic References, Positive Prompts, and Negative Prompts

17:00 – When Prompt Instructions Go Wrong: The Campfire Problem

17:50 – Controlling Generated Music and Audio

19:10 – Writing Clear Instructions for Characters and Actions

20:53 – Using Visual References and Starting Frames

21:36 – Why First-Frame-to-Last-Frame Animation Didn’t Work for Me

22:53 – Using Multiple ComfyUI Tabs to Queue Scenes

24:04 – The Secret Ingredient: Random Seeds and Iteration

25:36 – Render Times, Export Management, and Syncthing

26:33 – What’s Next: Editing, Shot Planning, and the Harbinger Trailer

27:04 – Closing Remarks: Buy the Book and Stay Holy



This is a public episode. If you'd like to discuss this with other subscribers or get access to bonus episodes, visit xeroforhire.substack.com/subscribe