Inspiration
We were very much inspired by the capabilities of Watson. We realized the popularity and utility of face recognition and decided, why not apply this to sign recognition as well.
What it does
This program allows the user to practice their Sign Language by displaying a command to demonstrate a certain sign and then letting the user take a picture of them signing that particular word. If the user gets the sign correct, the quiz moves on to another word. If the user gets the sign wrong, the program will tell the user what letter they signed, and loops until the user gets the sign correct or terminates the program.
How I built it
We trained Watson with many images of hands doing sign language. Then we made a quiz where pictures taken with your laptop camera are sent to Watson, and Watson tells you if you did the right sign.
Challenges I ran into
Training Watson to recognize the different hand signs was the most challenging part. We initially started with just a white background and 10 pictures per each letter, due to the limited number of photos (250) we are limited to with the free version. But that was not enough pictures for Watson to accurately recognize the sign. We then decreased the number of signed letters and increased the number of reference pictures, along with different backgrounds. The more photos the more accuracy.
Accomplishments that I'm proud of
That we were able to build this project with just a team of two.
What I learned
Before this project I had never used Python or Watson before. I learned the syntax of python and how to delete and create a classifier on Watson. My teammate learned a lot about OpenCv particularly in webcam and photo manipulation.
What's next for ASL Quiz
We want to increase the amount of letters and words in the quiz. Without the 250 photos restriction by the trial version of Watson I definitely think this is feasible.
Log in or sign up for Devpost to join the conversation.