SlideProf | Devpost

SlideProf Mascot

Inspiration

Many professors upload their slides without annotations, making it super frustrating for students to review afterwards. One of them taught my class. It was midterm season so my friends and I met to review together before an exam. We tried rereading the lecture slides hoping to understand better, only to get more confused. Random equations popped out of nowhere. Slides went from 1 + 1 = 2 to ways to solve B-tree with doubly-linked lists using dynamic programming at O(1/n). It was 2am so there was no way to get 1-on-1 help with a TF or professor. Spamming ChatGPT kinda helped but eve this was limiting because we couldn't picture or visualize the concepts in our minds. This is why I created SlideProf for those who are like the me back then trying to learn better with visualizations.

What it does

You upload your lecture slides and the system will scan them. You can read through your slides and review them as you normally do. If you don't understand a certain part, you can click the chat, easily hold click and drag to select the region and ask our SlideProf chatbot. SlideProf will explain just like how your professors normally would in a lecture with their tablets, by explaining while annotating and drawing things so the students can better understand the concepts. There is also another feature where you can put in your lecture Echo360's video and it will create annotated notes taken from your professor's audio. A good tool for those who are sick!

How we built it

Architecture

When you upload some slides, our Vercel frontend built with Next.js and Tailwind will send it to the data processing service deployed on Defang. Deploying a FastAPI app on Defang was a lot simpler than we expected and we got our own domain too. Would definitely recommend.

In the backend, the slides will be formatted and then sent to our other service to be analyzed. First, depending on your question, it will choose which visual format that fits best with your problem. Then, it will look around to see if there's any space to insert it in, and if not it will erase that blocking region. Then through GPT-4o API, it will answer your question normally like ChatGPT. The twist here is that it splits its answer into two parts, one for speaking with a voice model, and one for rendering on the slides. Our app will try to sync them so that the displaying and talking feel as natural as possible.

Challenges we ran into

At first, we couldn't get the OpenAI API's responses to fit with our format so there were a lot of trials of drafting the right system prompt. The hardest part, though, was how to position the shapes and the equations so that they look natural, as well as how to sync the speaking and the rendering times. The positioning was tricky because the PDF's dimension and the dragbox's coordinates on the screen were different, so we normalized them so that the coordinates only go from 0 to 1. It's not the best way still as things can be too far or too close to each other. For rendering shapes so that it feels natural, I cut them into different parts with timestamps that match with the speech. These two problems are what got us up all night, but we shalt persist and we did (no we got cooked).

Accomplishments that we're proud of

Learning how to display the visualizations in an intuitive way - Eric

Shaking hands with David Malan was definitely a big win since he was the one who got me into coding - Bao

Sleeping on four lecture hall chairs for a night - Hung

Got ranked up in leagues and learned React I hope - Vu

What we learned

Learned what it felt like to use GPT without worrying about money (Thank you our goats David and Julianna)

I don't know I need sleep - Hung

I learned so much I don't think it fits in my tiny brain - Eric

Zzzzzzz - Vu

What's next for SlideProf

The Echo360 feature runs smoothly on the backend thanks to Hung but it hasn't been implemented on the frontend yet. We aim to continue working on it in the future.