-
Platform UI
-
Image Recognition of Characters
-
Training AI
Inspiration
A widespread problem for university students in our class is that our professors write too quickly on the blackboard while talking and it is impossible to copy everything down and get a good understanding of the material at the same time. To solve this issue, we built a platform that takes care of the note taking for you - all you need is a picture of the board to have neat notes transcribed for you in Google Docs like format.
What it does
BoardScribe is a classroom collaboration social network environment that uses image recognition to transcribe blackboard notes from a photo. It allows for notes to be uploaded and transcribed, but that's just the first step. Once notes are uploaded students and teachers can make comments to clarify things and ask questions.
How we built it
The image recognition and machine learning were handled by Tessaract.js and Clarifai respectively. We were able to get pictures of individual characters out of pictures of text and pass those into Clarifai to transcribe into characters. We trained the AI extensively with over 1000 handwriting samples of different characters and continue to pass through different inputs to improve it day-by-day. For the platform we used the Django web framework and the SQLite database to store user's notes and group notes by class. Students also can comment on each note with suggestions and corrections using their facebook accounts
Challenges we ran into
The Clarifai API took a while for us to get the hang of. Building the platform took many iterations.
Accomplishments that we're proud of
We got the image recognition down.
What we learned
We learned a lot more about machine learning and image recognition.
What's next for BoardScribe
We will continue to develop a MVP and also potentially include the MYO Armband into our design to track chalk/marker movement.
Log in or sign up for Devpost to join the conversation.