Inspiration
The inspiration behind Decibel comes from the experience of learning a foreign language in high school. One of our team members was on vacation with his family in India one summer, and came across a Spanish lady, who did not speak a word of English or Hindi, the local language.She was trying to request lodging at a monastery after reading about it online, not knowing that they had a no-guest policy. The chief monk and the lady had trouble communicating and our team member tried to act as the translator, with their working knowledge of Spanish. However, to his horror, the lady could barely understand a word he was saying, owing to his terrible pronunciation in Spanish, one that was vastly different from the way native Spanish speakers talk. After a lot of attempts and wild gestures, the trio were able to communicate and resolve the problem, but it came to light that pronunciation is a very important aspect of learning a language. That is where Decibel comes in: to help a new learner work on their pronunciation, so that mediators worldwide can rejoice.
What it does
Decibel is a program that helps users work on their pronunciation while learning a new language by letting them record a sentence and compare it with a Google text-to-speech recording of the same sentence to get feedback. It is an audio analysis program that shows a random sentence in the target language and requests an input audio from the user. It extracts its Mel-Frequency Cepstral (MFC) coefficient and matches it to that of a Google Cloud TTS audio output of the same sentence. We use the process of Dynamic Time Warping to achieve this comparison between the two temporal sequences. The matching produces a numerical value which we present as the percentage similarity between the human-generated and the machine-generated audio files.
How we built it
We started with creating a basic flask application and learnt to manipulate that. While that was happening, we also kept researching into the audio processing part of the project. Flask required us to understand the basics of the way a web server and its requests works, it taught us about the idea of ‘routes’ inside a web application. In the end, we all distributed the work amongst ourselves and tried to learn, experiment and create as much as we can. The workflow kept flowing.
Challenges we ran into
The largest challenge we faced was regarding audio file manipulation. We needed to find a way to compare the two audio files in such a way that enabled us to give meaningful feedback. It took a lot of research to learn about Mel-Frequency Cepstral coefficient and Dynamic Time Warping and to incorporate these algorithms to compare our audio files was our biggest challenge.
Accomplishments that we are proud of
We learned a lot and managed to complete what we thought would be a ‘far fetched’ project. We learned how to implement Flask, different ways for manipulating audio and coming up with an idea that could help language learners enunciate in a flawless manner.
What we learned
We learned to implement Flask - a web framework for Python. We explored the various ways audio could be manipulated and compared. There was quite a bit of math involved in Dynamic Time Warping, and while the concept itself was a little difficult to grasp, we are extremely proud that we managed to incorporate it into our project.
What's next for Decibel
In a world becoming increasingly closer knit, it is not a surprise to see more and more people attempting to learn languages beyond their native tongue. Decibel has a few ideas for future updates: Perfecting our matching accuracy, and giving personalized feedback on where the user could improve, such as identifying individual words.
- A spelling cross-check feature to help users perfect their phonetics while speaking.
- A plot showing the most difficult words in each language and the percentage of users who got it right, along with sample usage.
- Enabling users to feed their own text.
- Personalised training for registered users
Log in or sign up for Devpost to join the conversation.