-
Logo made in Adobe Illustrator
Inspiration
Wanted to connect in a human way with some friends made online.
What it does
It uses a spaced repetition algorithm and a convolutional neural network for image recognition to help you learn and memorize ASL.
How we built it
Languages used Python HTML/CSS
Modules used Flask OpenCV Keras MediaPipe
Model which we used
asl_alphabet_9575.h5: by VedantMistry13 on GitHub
Challenges we ran into
Model was overfitting and would take too long to train with the resources we had available to us so we use a pre existing model.
Accomplishments that we're proud of
Can recognize ASL hand signs with some accuracy, also made a GPT powered bot to answer all of your ASL questions!
What we learned
How data augmentation works What a the layers of the CNN are exactly doing The 1/10th rule for parameters. need. You need data at least 1/10th the parameters for any usable results Importance of VRAM in training neural networks
What's next for Sight Vision
Improving accuracy of the predictions and gamification as well as a better short term memorization algorithm.
More detailed feedback on how to improve gesture in natural test. Such as by using the gpt-4 model with image inputs , and an appropriate prompt.
Log in or sign up for Devpost to join the conversation.