DubIT

Think Global, Speak Global. 🌎 An AI-Powered Enterprise Dubbing Solution For Industry Leaders.

Comment

What is DubIT? 🎤

DubIT is our integrated AI-powered automatic video dubber, intended to be used by industry giants such as Netflix, Prime Video, Hulu, Disney+ and more. Have you ever felt frustrated that a show you wanted to watch wasn't presented in your native language, and you just didn't want to bother with subtitles? With DubIT, all the end user would have to do is click a button, and you'll be able to watch any show or film in any language.

Our Inspiration 💡

The Language Gap

  • 40% of users report difficulty finding content available in their native language.
  • 65% of multilingual users say they watch content in non-native languages but prefer native-language experiences.
  • 30% of households indicate they would watch more international content if dubbing options were available. ### Streaming is Everywhere
  • 88% of households are subscribed to one or more streaming platforms, with the average number of services subscribed to is 4.1.

As on-demand streaming continues to dominate the entertainment industry, there is a large market cap when considering the global market, and the opportunities for language expansion. Most media on Netflix, Prime Video and Disney+ is only available in 2 or 3 language dubs. With DubIT, streaming giants can tap into this market growth potential.

Benefits of DubIT 💸

Unlock Global Markets

Streaming giants can tap into under-served regions where content is limited. Imagine the potential for streaming in emerging markets. With DubIT dubbing, you can expand reach to non-English speaking markets.

Expand Content Accessibility

Offering multiple dubbed languages enables streamers to cater to a larger, more diverse audience, and existing customers are more likely to stay with platforms offering enhanced viewing experiences.

Increase Customer Satisfaction

Users who face language limitations will have the ability to enjoy a wider range of content, enhancing engagement. It allows users to explore content they may have otherwise ignores, boosting content consumption.

Content Longevity and Flexibility

Streaming giants can retroactively breathe life into older shows and movies by adding dubbing in high-demand languages. This will also allow platforms to release content quicker in new languages, increasing the velocity of content expansion globally.

Monetization Opportunities

Streaming giants can offer AI dubbing as a premium service, leading to an increased ARPU. By making content accessible in new languages, platforms can partner with international content creators, creating new revenue streams.

Technological Leadership

By adopting and integrating an AI dubbing service, streaming giants serve to position themselves as leaders in innovation and technology-driven customer experience. Promoting content in multiple languages also aligns with corporate values around accessibility, diversity, and inclusion.

Features 🎮

  • The first integrated solution for streaming giants. AI video dubbing exists, but not yet on the enterprise level as we are aiming for.

  • Over 100 languages supported via use of Google Cloud API.

  • Trained with data from the integrated platform; if we partner with Netflix, DubIT will use actors from Netflix's database of films and TV shows, decreasing costs.

  • The translated dub still sounds like the original voice.

Our Tech Stack 💻

Our backend service is built in Python, utilizing Google Cloud APIs such as Speech and Translation. Additionally, we use FFMpeg for audio and video file handling. This is not a API wrapper; this is a custom solution intended for streaming giants. The frontend demo webapp was created using Next.js/React and Flask with help from libraries such as framer-motion.

Challenges and What We Learned ⛰️

The biggest challenge was getting all of the APIs and technologies like FFMpeg and Google Speech to work with each other seamlessly. Since we didn't want to simply call an existing service, we instead went through audio/visual processing and Google Cloud APIs. Although it was difficult, through research and trial and error, we eventually got a prototype working!

Through this project we learned not only about services such as Google Cloud APIs and FFmpeg, but also a lot about design and the business side of things. Being a business-centred hackathon, GeeseHacks has certainly been a unique experience for those of us that have competed in hackathons before.

The Future of DubIT 🔭

We believe the concept and technology of DubIT could be utilized to an extreme benefit by one or more streaming service platforms. There is no doubt in our mind's that something similar to this will appear in the near future, and DubIT wants to get in early. Some features we want to develop further are:

  • A studio to manually edit dub voices and timings.
  • Integration with the content provider's platform to better train on voice data.
  • Work with Google (or similar service providers) to ensure carbon neutrality.
  • Use GenAI to adjust the actor's mouth movements with the translated speech.
  • Of course, get it integrated with major streaming platforms!
Share this project:

Updates