Inspiration
People with hearing loss generally have a hard time communicating with others because most people don't know sign language. Translating sign language to text (or speech) would alleviate daily inconveniences and can assist the communication in case of an emergency.
What it does
Uses webcam video capture via OpenCV and using a trained tensorflow model, we classify the sign language into letters to display to another person
How we built it
Our team found an old repository with some work on computer vision for sign language. We set it up on an Azure Virtual Machine, found some reasonable training data from Kaggle and trained the model ourselves. Then we proceeded to setup the modify some openCV code to make it compatible with the latest version of Python so that we could gain webcam access.
Challenges we ran into
Learning a new programming language. Working with technologies we weren't familiar with as well as having a team with no major software background. Coding with Spicy noodles.
Accomplishments that we're proud of
Hackathon environments are meant to bring people together for collaboration and challenge participants to learn new things. Our team didn't have much experience in AI but we were eager to learn new things over the span of the weekend. After this weekend, we are very proud and feel like we accomplished that. Through teamwork and perseverance, our team is happy to say that we have a project to demo and show everyone.Everyone brought something to the table and although we aren't AI experts, we can proudly say that we all got a better idea on how to implement AI.
What we learned
Hannah and Wen learned about Jupyter notebooks and got a glimpse of the potential of AI. Alex and Tyrone learned more about azure cognitive services,Tensorflow and OpenCV. Larry decided to venture out of his comfort zone and learned about web technologies used to create a backend and the front end to host the project.
What's next for flowNote
Training our model to accept full word motions and translating sign language to speech would allow a faster means communication. Additionally, we must also tntegrate the standalone scripting app into the web platform to allow users to access this service online.
Log in or sign up for Devpost to join the conversation.