Inspiration
My partner and I were interested in ASL from personal connections to the deaf and hard of hearing community, so we wanted to create a translator for full sentences in ASL.
What it does
The user either manually inputs a message to be translated or takes in a txt file provided by the user to be translated. It then plays a series of mini videos demonstrating each sign in the provided sentence.
How we built it
We first found the online resource at handspeak.com, which featured not only an 8021 long dictionary of ASL words, but a video of each being performed. After downloading and indexing all of them, we built a system to filter out unnecessary words or anything not in the dictionary, as ASL grammar is notably simpler than that of English. Once we have the simplified string, we connected it to the previously mentioned video library, and play all the relevant entries in succession.
Challenges we ran into
Despite the convenience of having all of the videos in one place, they were of varying quality and resolutions. This put a wrench in our original plan to simply concatenate the videos, so we then changed course to create a playlist of videos instead. In addition, there are the inevitable hardships and processing time involved in any project of this scale; 8021 may not be a lot of words, but it certainly is a lot of videos. The final issue was rather unexpected. We hadn't anticipated that simply having the program launch the playlist for the user would be as large an obstacle in the end.
Accomplishments that we're proud of
Despite our difficulties with the display of the playlist, we eventually succeeded in presenting the final translation outcome to the user without sending them on a hunt through their files to find where it had been created. We are also pleased with the fact that our translator outputs a series of videos rather than pictures, as still guides can be rough for those new to ASL. Furthermore, we are glad that at the end we had extra time to include a function which will help to provide more complete translations by providing the user with synonym signs in place of words our dictionary did not contain.
What we learned
We learned a lot about the syntax of ASL and how it differs from English. We also were able to explore many functions within the nltk library. Despite not all of it being used in the end, we learned many different functions to organize language available to us within Python. Additionally we were able to learn more of the uses of our terminal and how to automate its use within Python as to ease the burden on the user in using our program. This program allowed us to review and delve more into reading and writing files.
What's next for Eng2ASL
Had we had more time we could properly implement ASL grammar rather than settling for Pidgin Signed English. Furthermore we would hope to have the program recognize words which do not exist in the English language like names and then provide the finger-spelling for the names. Currently if you wanted to translate, "My name is Simrohn" the user would have to input "My name is S i m r o h n" which is not an intuitive design for the users.
Log in or sign up for Devpost to join the conversation.