LingoSpell

Inspiration

At our university, a majority of the student population consists of international students. While lectures are delivered in English, many students struggle to understand the materials because of the language barriers.

We noticed that international students often spend more time translating lecture materials than actually learning the concepts. Watching recorded lectures repeatedly, pausing to look up unfamiliar terms, or relying on incomplete notes can make learning frustrating and inefficient.

Our project is inspired by the idea of magic spells in Harry Potter. We imagined a tool that could instantly “cast a spell” on any lecture and transform it into the student’s own language helping them learn the way they think best and we named it LingoSpell.

What it does

LingoSpell is a web application that allows students to upload their lecture videos and select their preferred language for translation. The application then generates a new video in the chosen language. Additionally, students can upload their lecture PDFs and notes, and LingoSpell will provide a summary and explanations in their selected languages.

How we built it

LingoSpell was built as a full-stack web application designed with a strong focus on efficiency and accessibility. The frontend was developed using React, providing an intuitive interface where users can upload lecture videos or PDFs, choose a target language, and interact with results in a smooth and responsive way.

On the backend, Flask is used to handle file ingestion, request routing, and communication between AI services. For language understanding and generation, we integrated the Gemini API to perform speech-to-text transcription, multilingual translation, and concise summarization of lecture content. To make the experience more immersive, we used ElevenLabs to convert translated text into natural-sounding speech, enabling high-quality voice dubbing for lecture videos.

By combining direct streaming with AI-powered language processing, LingoSpell delivers translated audio, transcripts, and summaries in a seamless pipeline, allowing users to focus on learning rather than struggling with language barriers.

Challenges we ran into

As students with backgrounds in Applied Mathematics & Statistics, Electrical Engineering, and Computer Engineering, we initially faced a steep learning curve in full-stack development, since most of our coursework had focused on hardware systems and data science rather than end-to-end web applications. Another major challenge was designing an efficient processing pipeline. We realized that a traditional approach of downloading files and sending them back would introduce unnecessary latency and overhead. To address this, we rethought our architecture and implemented a direct streaming solution, allowing content to be processed and delivered more efficiently in real time.

Accomplishments that we're proud of

We are proud of successfully implementing the Eleven Labs API to translate the uploaded videos into the user's preferred language, especially since this was our first time working with the Eleven Labs API.

What we learned

We learned a lot about video processing and the challenges that come from serving videos which was different from handling file I/O with PDFs or text files. It forced us to make considerations about how to efficiently retrieve the video from the backend to the frontend, ultimately forcing us to look more into byte streams in Python. We also learned a lot about the ElevenLabs API and how to be resourceful with the tools it provides in the free tier. It allowed us to get creative and try to use Gemini for tasks like video summarization and ElevenLabs for dubbing to get the best of both worlds.

What's next for LingoSpell

We would like to add more quality of life and accessibility options like closed captioning on dubbed videos for those with hearing impairments. Moreover, add more flexibility with file types like powerpoints and pdfs with many images which are often common file types for lecture notes. Additionally we envisioned adding more fun and engaging features that would keep students motivated like adding an option to AI generate songs using the lecture notes and ElevenLabs. Once these features would be implemented we would like to make the application widely accessible by hosting the app on Vercel.

Built With

• elevenlabs
• flash
• gemini
• python
• react
• typescript