Audition Agent

• 

  Audition Agent

Inspiration

AI filmmaking is getting good enough to create characters, performances, and complete short films, but there is still a gap when a filmmaker wants to bring one of those characters into the real world.

We built Audition Agent around a simple question: what happens when an AI-generated character needs a human actor?

Instead of treating casting as another ranking problem, we wanted an agent that handles the repetitive work around casting while leaving the creative decision with the producer.

What it does

Audition Agent turns a finished AI film into a human casting workflow.

Upload a film and the agent understands the story, characters, dialogue, and key moments. It generates an audition with directed takes and references back to the original film, publishes an actor-facing audition page, and organizes incoming applicants in one casting board.

The producer can then ask questions, review evidence, shortlist or pass applicants, and prepare follow-up actions conversationally.

For shortlisted performers, CALL-E can deliver a producer-authored disclosure about how AI was used in the role, confirm whether the performer still wants to proceed, capture questions that need a human answer, and coordinate a callback.

The agent handles the process. The producer makes the casting decision.

How we built it

Audition Agent uses Strands Agents with Amazon Nova Lite as the main agent and reasoning model.

Film understanding combines Gemini video analysis with Amazon Transcribe for grounded dialogue and timestamps, then Nova turns that into structured film context. A separate Strands writer creates the audition package.

The app uses FastAPI, React, SQLite + FTS5, Amazon S3, and server-sent events so agent actions update the casting workspace immediately.

Challenges we ran into

One challenge was keeping the agent useful without letting it become an automated casting judge. We deliberately built no ranking tool and added guards against ranking-shaped actions.

Video grounding was another challenge. Visual understanding and transcript timing come from different systems, so dialogue references are matched back to real transcript timestamps instead of trusting model-generated timing.

Phone calls also required stronger boundaries than normal software actions. We added explicit producer confirmation, durable call intent, idempotency, masked numbers, fail-closed result handling, and no automatic redialing.

Accomplishments that we're proud of

We built a complete workflow that starts with an AI-generated film and ends with a real human casting conversation.

The most important accomplishment is that the agent does meaningful work without taking over the producer's judgment.

A producer can go from:

AI film → generated audition → applicants → shortlist → real phone follow-up

inside one workspace.

We are also proud that the CALL-E interaction does more than schedule a call. It gives the performer important context about the AI-created role first, then brings their decision and unanswered questions back to the producer.

What we learned

We learned that the strongest agents are often not the ones making the biggest decisions.

For this workflow, AI is most useful when it understands media, organizes evidence, prepares actions, and removes repetitive coordination. The creative judgment itself is better left to the human.

We also learned that different models can play specialized roles: video perception, transcription, reasoning, and action orchestration do not all need to come from the same model.

Most importantly, real-world actions need stronger boundaries than chat. A phone call should happen because a human explicitly approved it, not because a model inferred that they probably wanted one.

What's next for Audition Agent

Next we want to move from a single-producer prototype toward a production-ready casting workspace.

That includes authentication and multi-project support, durable background jobs, stronger media processing, callback/calendar integration, and better collaboration between producers and casting teams.

We also want to keep exploring the central idea behind Audition Agent: AI should make human creative collaboration easier, not replace the humans making the creative decisions.

Built With

• amazonnovalite
• amazonnovamicro
• amazons3
• amazontranscribe
• call-e
• fastapi
• gemini3.7
• react
• sqlitefts5
• strandsagents