Inspiration
We wanted to work with imaging in some way. And build something neat.
What it does
DrawingBuddy is here to help you draw! With the desktop client, you can select an image that you'd like to learn how to draw. You'll help the computer segment it by selecting a thresholding value and choosing with contours to include in the final segmentation. Then, the image will be sent up to firebase where it waits for you to tackle the challenge of drawing it on your Android phone. The Android client will let you select an image from your database to practice on. You'll be able to select the difficulty, which will determine how much guidance you get. When you draw along the segmentation, you'll be guided by a coloring scheme that indicates how close you are to drawing the segmentation, whether it is visible or not. You'll get a fun GIPHY gif that varies depending on how well you did after you finish, and your results will be stored in the database, where they are used for analytics in a web app.
It also now validates signatures by means of the energy calculation! Hopefully we have time to build that into the Docusign API so that you can only make API requests after your signature is validated.
How we built it
The image can be segmented in two ways, either through intelligent scissors or through programmatic contouring. In either case, the image in question will be blurred through a Gaussian and differentiated through a Laplacian to bring out edges, and then thresholding and various morphological operators will be applied to further accentuate the object in question. Then, the image will either be contoured programmatically or the user can assist in the intelligent scissors process. Once this is complete, we'll have a contour to store in our database for a given image.
We display the contour by plotting points on a canvas. We determine accuracy based on energy, where we view energy like one might a potential energy; the further you are away from the object, the more energy you have. We compute the shortest distance to the segmentation through a BFS and use this is as the energy. Your drawing will display both your current energy as the cursor and your average energy as the path you've drawn.
Once you finish drawing, we save your drawing and send it back to our database along with analytics. We then give you a gif based on a keyword mapped to by your accuracy score.
Our web app will show your progress over time, as well as offering analytics on your progress.
Challenges we ran into
Liam spent two hours on a bug because he mixed up the letter x with the letter i.
The implementation of intelligent scissors was...slow. And also pre-processing the image for it is hard.
Accomplishments that we're proud of
Completing algorithms for segmentation and tracing/energy computation. Linking the application across different platforms/integrating was good and smooth. We're also super happy with the python UI for image selection and morphological thresholding.
What we learned
More about openCV, how to firebase, more debugging, and how to integrate multiple components. Also teamwork! Woohoo.
What's next for DrawingBuddy
Machine learning, of course.
Log in or sign up for Devpost to join the conversation.