Rusty humanoid robot Rustboy with one glowing green eye, standing against a black background.

RustBoy AI: Learning About Agents by Making a Movie

Recently, I started using Grok Bot to learn how agents communicate, divide up work, and work toward a shared goal. I wanted a real project, but something safer than using them for my work.

So, naturally, I asked them to make a movie.

A project I never forgot

Back in the early 2000s, I came across Rustboy, an animated short-film project by Scottish illustrator Brian Taylor. He was creating it largely on his own, using a Mac and a 3D package called Infini-D, and documenting the process online. You can learn more about the original project at RustBoy.com.

At the time, that kind of animation felt like something you needed a studio and a render farm to produce. Seeing one person bring this little rusty character to life stuck with me.

More than twenty years later, it seemed like the perfect inspiration for an experiment.

I’m betting on two things: agents will become a bigger part of how we work, and the models behind them will keep improving. The idea is to reimagine what Rustboy could look like today and see whether a team of AI bots can work together and produce a better film as those models improve over time.

I started with one director

I gave a director agent its goal and responsibilities, then asked:

What do you need?

First, it needed video generation. We created a renderer agent to work with Grok Imagine, with the director overseeing it. The renderer brought me into its browser to sign in, and then they started producing clips.

That was pretty awesome. It also made the gaps obvious.

There was no music, so we added a sound-mixer agent. There wasn’t much of a storyline, so we added a writer.

The team grew as we discovered what was missing. I was still pointing out many of those gaps, but the agents were taking responsibility for the work.

Making clips is easier than making a movie

The biggest challenges, even today, are continuity and the storyline.

Rustboy can look different from one clip to the next. An individual clip might look good without matching anything around it. And even with a writer, we’re still working on getting those scenes to tell a story rather than just making a collection of clips.

Generating more footage doesn’t necessarily get us closer to a coherent movie.

We added a quality-and-continuity agent and approved a shared character reference, including views from nine angles. That gave the team something concrete to check against. (RustBoy.ai)

They’re still struggling with it. I help with direction, but I’m trying not to manage every shot. Finding that balance—enough guidance without taking over—is part of the experiment.

The updates needed work, too

Early status reports would say things like “Scoring KF 118 still,” or mention that something was skipped without explaining why.

I could tell they were busy. I couldn’t tell whether they were making progress.

I had to make the expectations clearer: tell me what changed, what failed and why, what happens next, and whether you need a decision from me.

Getting agents to communicate with each other is one challenge. Getting them to explain their work to someone checking in once a day is another.

Building it in public

I had registered RustBoy.ai a while back, so I asked the director to build a website and start documenting the project.

Now the agents prepare daily blog posts, upload media, and submit website changes through pull requests. I review the work once a day, approve it, and either tell them to proceed or redirect their attention.

This isn’t a one-prompt movie. I’m still involved, and we don’t have a finished film. But the agents are producing footage, maintaining the website, and documenting what works and what doesn’t.

My goal is to see how far this reimagining of Rustboy can go as the models improve, with the process happening in public. Better models should help, but they won’t replace the need for a shared story, consistent characters, clear responsibilities, and useful feedback.

That’s what I’m learning along the way.

You can follow the experiment at RustBoy.ai, including the parts we’re still trying to get right.

Next
Keyv and Cacheable - Chaindrop Worm