TTAP: Text To Audio Processor

Automated dubbing software

Comment

TTAP

Text To Audio Processor

Why?

Many foreign films have English subtitles but no English audio. Subtitles aren't a viable or enjoyable option for many people. TTAP makes formerly inaccessible media accessible for people with reading/vision impairments, when audio is unclear or there is no dubbing, or for people who just like their subtitles read out.

How it works

TTAP is a client-side live automated dubbing program. It uses Google's Tesseract OCR system to read subtitles off the screen, making it flexible for use on any streaming platform. The text is then quickly read aloud by the Unreal Speech API in a configurable voice. A minimal frontend written in customTkinter allows you to tell the program where the subtitles are and change the voice settings. Then it gets out of your way to let you enjoy the movie.

Challenges

We were originally going to do a completely different project involving security. When the tech we were going to use proved to be unviable in a short timeframe, we pivoted halfway through the project. This left us little time to develop TTAP. TTAP was going to be the name of the original project, and we reused it for this one. Text To Audio Processor is a backronym!

Accomplishments

The crux of the project was making it fast enough that the AI voice would speak in almost real time. We tried many Tesseract wrappers before settling on tesserocr. Quick TTS was also difficult; local models were much too slow and most APIs left us wanting more customization and a slightly faster response time. By choosing tesserocr and Unreal Speech and then optimizing the code extensively, we made the system so fast that the voice starts speaking 0.6s after the subtitles appear.

Who?

TTAP was written by Ray Chen, Tyson Caul, and Sark Asadourian for the UofTHacks 2025 hackathon.

What's next for TTAP?

  • Better subtitle detection so the user doesn't have to draw the bounding box
  • Support for subtitles and speech in languages other than English
  • Even faster response time
  • Adaptive speech pacing to keep up in scenes with quick dialogue

Built With

Share this project:

Updates