Blockify

Blockify is a tool that can cover up labels in diagrams for studying.

Comment

Inspiration

We were inspired to work on this project after speaking with a current medical student, a brother of one of our teammates. He discussed how he needs to study diagrams for his medical school tests. The problem is that to quiz himself on his diagram knowledge, he has to manually cover up the labels by himself or hope that another classmate has already done so. We realized that by using artificial intelligence, we can automatically detect text on diagrams and cover it up. A tool using this technology would help students (medical or not) save time when studying for their exams. This encouraged us to create Blockify.

What it does

Our project automatically covers up text or labels in diagrams with black boxes. These boxes can be toggled to reveal and hide answers. This tool allows students to avoid manually covering labels, saving their time when studying for exams.

How we built it

Express.js was utilized with the Handlebars templating engine to create a dynamic website. We used Google Cloud Vision API to detect bounding boxes of the text in pictures. Through the use of the Canvas API and Vanilla JS, we generated an image with bounding boxes covering text for the perfect study tool.

Challenges we ran into

  • Storage and rendering of uploaded images w/o database
  • Templating engine limitations for sending image coordinates & properties from backend to frontend with Express.js
  • Proper alignment of bounding boxes over text

Accomplishments that we're proud of

  • Minimalistic and slick frontend
  • Intuitive and user-friendly UX
  • Generating the bounding boxes using the Google Cloud Vision API

What we learned

  • Understanding of Express.js components for routing behaviors
  • Understanding of Canvas API for graphics
  • How to use the Google Cloud Vision API for text detection
  • Integrating Node.js libraries with Express.js

What's next for Blockify

  • Allowing for PDF & other input file types
  • Adding a gamification aspect to the website such as a leaderboard system, fill-in-the-blank, etc.
  • Changing the color of the bounding boxes based on the background color of the image
  • Developing a mobile application
  • Deploying the application to an external server

Built With

Share this project:

Updates