FrankieBot The MLH Discord Bot

A Discord Bot that helps you keep track of all the upcoming & recent MLH hackathons. It comes with a subscription feature that notifies you as soon as a new hackathon is announced

Comment

💡 Inspiration

Last week when I decided to check for upcoming MLH weekend hackathon, I had to spend a considerable amount of time finding the actual page of the hackathon. First I visited the Devpost website and tried filtering through the plethora of hackathons posted there only to give up a minute or two later. I was only able to find the details about the Birthday Bash hackathon after visiting the MLH website. After discussing this with my team mates I realized that I wasn't alone. FrankieBot is a discord bot that helps streamline the process of looking for hackathons for hackers!

⚙ What it does

Every 30 minutes The web service scrapes the official MLH website and obtains a list of upcoming & past hackathons. These hackathons are then stored in the database and are exposed to the FrankieBot using a handful of API endpoints. The list of the used API's could be found at the following postman documentation link

Whenever the web scraping service identifies that there is a new hackathon record that was not saved on the database after a scraping run, it send a SMS message to all the users that have subscribed for hackathon notification.

The FrankieBot discord bot acts as an interface and uses the above API's to display information about recent & past MLH hackathons based on the users requirements.

🔧 How we built it

  • Nodejs - We used nodejs along with expressjs to create the webservice for our application
  • Puppeteer JS - PuppeteerJS was used on top of the nodejs webservice to scrape the MLH website.
  • CockroachDB - The scraped data was then stored on a cockroachDB database instance.
  • AWS - An amazon EC2 instance was used to host the above mentioned scraping web service.
  • Discord JS - We used Discord JS to create the FrankieBot discord bot.
  • Heroku - This discord bot deployed using Heroku
  • Twilio - We used twilio to send sms notifications to subscribed users when a new hackathon is announced.

💪 Challenges we ran into

  • None of us had any prior experience in web scraping. This meant we had to do some extensive research before we were able to scrape the data from the MLH website.
  • We initially wanted to use Heroku in order to deploy both the services (i.e the web scraping service & the discord bot). However after hosting the project we noticed that the scraping service was not running properly due to the underlying OS of the heroku instance. Due to this we decided to provision a separate EC2 instance using a Linux AMI to run the web-scraping-service. Since all of us were from windows backgrounds, this also proved to be a challenging task as we had little to no knowledge on the usage of linux commands.
  • We were in two different time zones, which made keeping in sync initially difficult.

📌 Accomplishments that we're proud of

  • We eventually delegated work effectively, which made development smooth. We also made sure that everyone worked together seamlessly based on their skill set and had brief knowledge sharing sessions to keep everyone on the same track.
  • We created a fully functioning, complete discord bot that believe would assists hundreds if not thousand of users in the future.🥳

📚 What we learned

Most importantly we learned the importance of collaboration: we split the task of making the bot between us, and we were able to help each other with our different tasks. Members who were new to Javascript were also able to learn more about using the language!

⏭ What's next for FrankieBot?

  • Filtering Events based on specific locations, regions, ages etc.
  • Collaborating with the MLH team to integrate the FrankieBot into the official discord server.
  • Expanding beyond MLH to scraping sites like Devpost (possibly creating an unofficial API for devpost?)

Built With

Share this project:

Updates