Inspiration
With online classes going on, it is easy to lose focus in class. Maybe you don't have time to rewatch lecture, or you simply prefer reading over listening. No matter what your reason, it is never a bad idea to use SUMO.
What it does
SUMO is an application that takes in a lecture recording and returns you an annotated transcript along with a brief summary of your lecture. This can be extremely convienient for students who want to just read through their lecture, with some notes attached and links to external web pages for more information. In addition, we provide a summary of the whole lecture which students can quickly use to study for an upcoming exam.
How we built it
Our frontend is built in React and our backend is written in flask. For the transcription, we used Google cloud API's to transcribe the whole recording, and we sent that transcription to Azure to perform analysis.
Challenges we ran into
We ran into many challenges when making this application. We had to make sure that Google was able to transcribe everything properly no matter the length of the lecture. Because of computation limitations, it may take a significant amount of time to transcribe a long lecture. In addition, we had to do a lot of research on NLP, and really understand what we were doing. At first we built our own abstractive text summarization API using the transformers library and PyTorch but we quickly realized that our computers were too slow. We then passed our data onto azure to provide us with an analysis.
Accomplishments that we're proud of
We were able to coordinate our efforts, and worked as a team throughout the entire Hackathon. As a team, we ensured that we were all communicating, especially when difficulties arose, resulting in less obstacles being in our way to meet the goal (uncommon when a group of people that don't know each other are put together). Also, we were all able to contribute to this project while giving our best, resulting in becoming better programmers, and growing as individuals.
What we learned
We Learned how to use new NLP tools with Azure, along with efficiently handling the requests, and sending them to the Front-End. When it comes to the Front-End, we all improved/learned how to animate better our components.
What's next for SUMO
Keep adding elements to the UI with improved UX. Improve summary details by using TensorFlow, and implementing a model containing a Deep Neural Network. Improve the amount of time the uploading process takes, which would result in an improved UX.
Log in or sign up for Devpost to join the conversation.