Inspiration
Origami is cool. Machine vision is cool. Together, even cooler.
What it does
The application is for origami lovers and people who are only beginning to learn origami. This step by step visual guide will allow you to easily follow and verify your steps. Using machine vision from Microsoft Azure and OpenCV, you'll never struggle to make origami again! After folding each step, the user holds it up to the camera, and if done correctly the AI will tell you the next step.
How we built it
We used Microsoft Azure Custom Vision training to create a model that could label each of the steps of our origami heart. We took multiple pictures of the paper at each step and manually labelled them all for training. After we go the model, we were able to use the API to return back which step the paper was at. Using OpenCV for our GUI, we were able to capture webcam images. We also used the windows speech to text to read off all the steps. This project was built mostly in Python and our front end using Flask.
Challenges we ran into
Upgrading Python. Choosing which machine vision API would work best to read simple shape and colours. We found openCV was really hard to use for our purpose because the background was very noisy and lighting conditions were not great. Furthermore, Google Vision was not able to label multiple shapes and colours for our purposes. We learned that machine vision is a very tough field to work with.
Accomplishments that we're proud of
We somewhat have a finished product. We did not sleep. We learned a lot. We had fun bowling.
What we learned
That changing Python versions is difficult and Tensorflow only works for 64 bit Python.
We learned how to use openCV and Microsoft Azure's custom vision. It was cool to see the machine learning training for our project.
What's next for origamigo
We were planning on implementing an AR feature where the application would be able to show you the next steps visually, shown right on your piece of paper. Furthermore, adding more labels and text to describe steps. There are so many different origami projects we could add as well but we did not have time to train that many steps. We would like to have a more complete front end for this application and make it look as fancy as origami feels. It would also be nicer to further gamify this application. But, this type of application could be used for tons of hands-free step by step tutorials such as cooking, crafts, etc!
Customer Feedback
"" I find it really annoying that I have to switch my gaze between my origami and my TV, because I clearly like TV more than origami. Now with AR, I can make my origami without looking at the instructions at all. " - Jacky, 2019
Log in or sign up for Devpost to join the conversation.