StreamGuard | Devpost

StreamGuard Home Page

Intro

Have you struggled with tongue slips? Done either by your favorite streamer or perhaps even yourself? Happens occasionally but when it does, it calls for disaster for the creator. One wrong word could mean losing portions of your audience, or worse, a long developed career.
This is where StreamGuard come to save the day...

StreamGuard is an innovative content moderation tool designed to enhance the live streaming experience by filtering out unwanted language in real-time. Whether you’re a Twitch streamer aiming to maintain a family-friendly channel or a broadcaster ensuring professional standards during live interviews, StreamGuard has you covered.

Why StreamGuard?

In the fast-paced world of live streaming, maintaining a clean and professional broadcast can be challenging. StreamGuard provides a reliable solution to prevent unintended slip-ups from reaching your audience, protecting your brand and ensuring a positive viewing experience for all.

Inspiration

StreamGuard was inspired by the need for live streamers to protect their livelihoods, as well as the desire of certain consumers to moderate the content they listen to.

How we built it

We built StreamGuard's frontend using Next.js and Tailwind CSS for smooth transitions, creating a visually appealing user interface, all deployed on Vercel for easy updates. The backend, powered by Flask, handles HTTP requests, while PyDub and PyAudio manage audio processing. We leveraged PyTorch for machine learning alongside with OpenAI’s Whisper providing real-time speech recognition for accurate transcription and filtering. Real-time communication is facilitated by Socket, ensuring minimal latency. Finally, our final goal is for StreamGuard to be deployed on Google Cloud Platform, where it can thrive in larger clusters, taking advantage of the larger AI models via more resources.

How to Real-Time

As you might imagine, with one big difficult problem, you must first break it down into many manageable chunks. So that is exactly what we did! Our software breaks down the live feed into 1s to sub 1s segments which are processed individually then put together to create the final transcript you see. The system is intelligent and hence the chunk cuts happen when you finish the word you are speaking, i.e. the instant before you say the next word in your sentence. This opens many gateways and possibilities for us, concurrent-programming/parallel-computing, ability for APIs and faster response times are a few to mention. But for this project we are using concurrent programming at small scale to get the job done.

Challenges we ran into

Creating a Real-Time-System is easier said then done. Here's a peek into issues we faced and challenges that almost made us scrap the idea:

Challenge1 - Difficulty Moving Data
Most of the systems you interact with on the internet run on traditional API calls with fixed amount of information. You cannot send a stream of data over http. Therefore, you are very limited in terms of frameworks and communication methods you can use.

Challenge2 - Unfamiliarity Although we had a member familiar with real time systems, creating a working application from the ground up created many new ways one can mess up. We ran into countless errors we had to debug, libraries and frameworks we were unfamiliar with, new technologies, and many other challenges during the weekend.

Accomplishments that we're proud of

We are proud of creating a unique idea with a legitimate business use case, and all the work we put in to create a proof of concept.

What we learned

We learned more about our respective tech stacks, as well as about the design, collaboration and development processes and of course; about real time systems.

What's next for StreamGuard

We do have a Roadmap for StreamGuard on our git repo which we hope to someday accomplish. But here's a quick summery:

Customized alerts - maybe you don't want to have a beep sound effect but want to mute or play your own sound effects!
Dashboard - A web-based dashboard for configurations and monitoring.
Live Translation - In addition to live transcription and live moderating, our system allows for development of live translation removing the language barrier for audiences speaking different language