Lingo Live | Devpost

Inspiration

The concept of Lingo Live initially emerged as an enhancement for video conferencing platforms such as Zoom and Google Meet, where we envisioned creating an online translator for seamless multilingual meetings. However, we soon recognized its potential for broader applications, extending beyond meetings to include YouTube tutorials and virtually any website with audio content.

What it does

Lingo Live is a Chrome Extension designed to translate audio from Chrome websites into a variety of selectable languages. It then plays back the translated audio in the chosen language.

How we built it

We utilize the chrome.tabCapture API to capture audio from Chrome websites. This audio is then translated using Gemini for highly accurate translations. After translating the audio, our program converts it into text, translates it again, and finally converts it back into audio using the Amazon Polly API for text-to-speech (TTS). The translated audio is then played through our extension, allowing the user to hear the translations in real time.

Challenges we ran into

Our most significant challenge was integrating various components that operated on separate technologies. Bridging these technologies required us to rely on external solutions such as Firebase storage to facilitate the transfer of audio between different programs. Additionally, grappling with technologies unfamiliar to our team posed a significant challenge. With many of us lacking proficiency in JavaScript, and considering that much of the required technology relied on it, this presented a significant roadblock during the competition.

Accomplishments that we're proud of

We are pleased with the efficiency of our translation and speech process, especially during optimal internet conditions. Additionally, we are proud of the user interface design and thematic elements of the extension.

What we learned

As we dove into several new tech tools like the chrome.tabCapture API, Gemini, and AWS Polly, we also gained valuable experiences by fostering cohesive teamwork. Since most of us were used to solo projects, collaborating effectively posed as an initial challenge. We learned to divvy up tasks fairly and ensure everyone stayed on track.

What's next for Lingo Live

Due to the time constraints of the hackathon, we were only able to implement the core functionality of the program. However, we have exciting plans to enhance the extension's features in the future. One idea is to tailor the translation options based on the type of tab being recorded. For instance, in a Google Meet session, users could have the ability to toggle which participant's audio is being translated.