EdZy | Devpost

Download URL
Keyboard Shortcuts
Capabilities
Example Implementation

Track

Education

Inspiration

I'm lazy. You read that right. Very lazy. I want things done fast, easy, quick. I was in bed, reading an online textbook, in a position where my hands didn't feel like touching the keyboard. Few minutes into my automata and complexity textbook, I had no clue what I was reading (maybe I should skip class less often). I knew the next best way to learn the concept was to watch a video.

BUT HOW!?

I'm nicely comfortable in a position I don't want to change to go on YouTube.

WHAT DO I DO!?

Ended up making EdZy at Hack Princeton as a solution to make my job easier.

What it does

It helps you find the answer to your topic/question instantly in the format of a video. It lands you in the exact timestamp matching the answer to your query using sophisticated algorithms. You can use Command + E (for Mac ONLY) and Ctrl + Q (for other devices) after highlighting the text you want to search to use the extension. Get results in 1-2 seconds.

How I built it

The frontend of the app was built using a chrome extension framework and APIs, as well as fundamental languages like HTML, CSS, JS, and Bootstrap. The server was hosted on an Azure T4 GPU, pros to being backed by Microsoft for Startups, and was used to run high inference implementation code. Prior to code base migration, the code was unit tested on a NVIDIA RTX laptop, provided by MLH. The algorithm utilizes parallelism in the pipeline to process multiple tasks at the same time. YouTube's search and transcript APIs are used to fetch the necessary data contents necessary. All-MiniLM-L6-v2 embeddings from HuggingsFace are used to extract chunks across the data. A novel algorithm is in place using turing machines, recursive divide and conquer sub-algorithms, and other inputs to extract answers based on queries.

Challenges I ran into

One challenge I ran into was getting access to an Azure T4 GPU. The GPU market is terrible and this one was hard to pull off after a lengthy email conversation and ticket. Another challenge was reducing the latency. When I started building this algorithm, it took nearly 30 seconds. Now it gets the job done in 1-2 seconds. This is all thanks to all the optimizations in place, from converting the model to ONNX files to using optimal data structures. Another challenge was limited time for testing. So I got my friends to test the product I made and give user feedback on areas where it failed. Based on feedback I received, for example limited context, I was able to expand on ideas; for example, the chrome extension takes more content than what you highlight under some scenarios (based on another semantic comparison algorithm) in order to produce more accurate results. One last challenge I faced, in addition to other multiple minor ones, was integrating the extension with OpenAI. Web scraping was a lengthy task and involved a lot of testing, rather than just writing code in one-go and compiling it in one-go, which took a lot of time, but the end product was really useful based on the feedback I received.

Accomplishments that I'm proud of

Having a 1-2 second latency to provide very accurate results is simply fascinating to me. Also, I'm not much of a video creator guy myself, and making the video on Canva was a fun experience and I loved the output video I made. Definitely one of my biggest accomplishments in this hackathon so far.

What I learned

How to optimize models using ONNX so save 2x runtime
Implementing job queue on the server for requests
Chrome Extension APIs in depth
Implementing and evaluating models like TransDETR and other data extraction models
How to make videos on Canva

What's next for EdZy

Turns out, everyones likes it (or maybe they are being nice about it). Anyways, look forward to a production version, around two weeks, once beta-testing and iteration over user feedback is over next week :) Also after talking with a few startup business owners, realized potential for a B2B solution using the algorithm I built in this.