Inspiration
Our friend’s grandmother had recently been diagnosed for help. His grandmother was one of his closest friends, but without the ability to communicate with her, hope had been lost. Thus we wanted to develop an app that allows for easy and simple communication between my friend and his grandmother. But our mission didn’t stop there. We wanted to further develop an app that allows all people with disabilities to communicate to all people(including those with disabilities). Thus, our app allows for seamless translations between and to speech, text, morse, sign, and braille.
What it does
Our app is quite simple but powerful in application. Users are allowed to input any form of initial data, an image of a sign language sign, braille keypad, text, speech, or a morse code tapping pattern. Our program can output any of the aforementioned inputs as well. Thus a blind person can translate their braille into a sign to communicate with a deaf person. And a deaf person can convert a sign to a morse code pattern to communicate with blind people. The possibilities are endless, and we hope to make a lasting impact on the disabled community through our universal translator.
How we built it
We utilized the Swift programming language to develop the mobile application. We also utilized machine learning algorithms to further comprehend different types of languages (such as sign language). In addition, we also utilized natural language processing algorithms to comprehend some of the text that the user inputs. The mobile application ties in all of these models into one service.
Challenges we ran into
We originally had a low accuracy while predicting the sign language shown. Because of this, we were not able to receive accurate results for sign language conversion option. However, we later utilized a better architecture to solve this issue.
Accomplishments that we're proud of
We were extremely proud of the app that we had developed in general because of its capabilities. We were highly glad that we were able to develop an app that could positively impact our society and loved ones greatly.
What we learned
From the overall development process, we learned how important collaboration is, for we understood how different components of a mobile application are simply complicated to make for just one developer. Without the collaboration of multiple developers, developing such a project would be extremely complicated.
What's next for Sermo
For the next version of the app, we plan to add a feature where sentences can be translated to sign language and braille. This would make it much more efficient for people who need a universal translator.
Log in or sign up for Devpost to join the conversation.