Rate My Hack | Devpost

Sample query.
Scraping all projects from Devpost.

Inspiration

We find it hard to come up with hackathon ideas. We also wasted lots of time trying to determine if an idea had already been implemented before. That's why we thought we would develop this app as a way to help us determine our next project and avoid duplicating other people's ideas.

What it does

Rate My Hack uses Beautiful Soup to scrape every single Devpost project ever (yes, all 70000+ submissions!) and create a data set containing descriptions for each project. Then, machine learning and natural language processing is used to determine the most similar past projects to a given input search string. The app then calculates the originality of the hack as well as its chances to win based on similarity to past winners.

How we built it

We used Python and Beautiful Soup to build The back-end and web scraper. For the front-end, we used React.

Challenges we ran into

Google Compute Engine turned out to be very difficult to make work with REST requests: we spent an entire night just on that. React Native also presented unexpected devtools-related challenges, to the point that we decided to switch to a web app.

Accomplishments that we're proud of

Our ML algorithm actually produces semi-coherent results: reaching an adequate level of quality required a LOT of careful tuning.

What we learned

Scraping every single Devpost submission takes A LOT of time. More importantly though, tying the fate of a project to an external technology, be it even open-source, is a really bad idea.