Lecture Agent | Devpost

UI With integrated slideshow and Audio Player
Landing Page with File Upload
Sign in Page
Post Processing Lecture Homepage

Inspiration

3 AM, just the light of Jacob's computer in his dorm room. Reading through 3 weeks' worth of lecture slides to make up for getting the flu. With a teacher who doesn't do recorded lectures, the task of learning multivariable calculus was up to him. After ten slides he couldn't seem to keep my eyes open. The same boring format with no one to explain it to him made him vow to never make anyone go through what he had to endure. Our team set out to make a program to do just that. Lecture agent ensures that no matter your circumstances, you're never left behind. With Lecture Agent, you can study class materials from anywhere, anytime with a personalized AI-driven professor who delivers dynamic lectures based on your course content.

What it does

Lecture Agent takes those dreadful class slides that feel like quicksand to go through alone and builds your very own AI lecture presentation. It takes the uploaded slide notes and intelligently generates a fully narrated, professor-like presentation, breaking down even the most complex concepts into digestible explanations. This allows users to experience a seamless, interactive lecture experience.

How we built it

We built out the backend by taking each slide and the text on that slide and saving each side individually with its core information. We then leveraged fetch ai's framework to utilize their agents intensively throughout our application. We developed agents to handle each part of the text and image scraping along with the communications with OpenAI's models to generate the lecture script. We then take that lecture script and convert it to speech using OpenAI's Speech API.

Challenges we ran into

We faced difficulties in synchronizing the generated audio with the presentation of slides. Additionally, processing the images and text to generate the lecture script took longer than expected. To optimize performance, we had to find a way to display the previous slide while the next one was being processed, minimizing the perceived delay for users. Getting the Fetch.ai Agents to deploy and communicate with each other was a very tedious process. We also had trouble integrating a PDF reader into our frontend and we were able to solve this by simply displaying the images of each slide separately.

Accomplishments that we're proud of

We were very proud of the increased reliability of our lecture content that came from incorporating both text and visual data from the slides in our lecture generation. Providing images alongside the lecture text allowed for more content-rich explanations. Displaying the slides alongside the audio player that

We were able to distribute each of our backend tasks to fetch AI agents. We utilized six Fetch AI agents, which communicate autonomously to manage text extraction, image processing, script generation, and voice synthesis, ensuring smooth and reliable performance.

What we learned

We gained valuable experience integrating Fetch AI's UAgents framework, optimizing agent communication to effectively delegate tasks. Additionally, we improved our ability to manage large-scale data processing using decentralized agents, which significantly enhanced the scalability of our system. We also deepened our understanding of integrating multiple microservices and ensuring seamless communication between them

What's next for Lecture Agent

Features coming soon:

ChatLA: This feature allows real-time Q&A interaction between students and the AI lecturer, enabling users to ask questions and receive accurate, on-the-spot answers based on their lecture material.

LiveTranscription: Gives users the ability to receive real-time transcriptions of the lectures, further enhancing accessibility and usability.

LiveTranslation: Real-time translation of slide and lecture audio material for nonnative speakers that allows users to get the help they need understanding their material in a language that feels comfortable to them

VoiceChoice: Allows the user to pick the most engaging lecturers' voice for them.