TicTalk | Devpost

Home Screen
Stats

Inspiration

Over 136,000 international students choose to study in British Columbia each year, with learning English as their second language. With such a multi-cultural city, being able to communicate in English and different languages is crucial to connecting to others personally.

Despite the language learning apps out there, we realized that many of them do not give personalized feedback on the accuracy of their pronunciation and grammar - which is important to improvement in skills.

We wanted to make practicing languages easier, by creating an online coach (chat bot) that was accessible at anytime, as well as practice in a safe environment without judgment.

What it does

Our website allows learners to choose from a selection of languages to practice on. The chat bot and the user begin a conversation, and the user speaks into their microphone to continue the conversation. The user can then stop the conversation and see an analysis of their speech - such as pronunciation, grammar, hesitation (pausing/filler words) and vocabulary.

How we built it

We used speech-to-text package in npm to convert user dialogues into actual text for grammar analysis. We attempted speech analysis with Microsoft Azure as well.

The method we used to analyze grammar was keeping track of incorrect character counts and display the final grammatical correctness in percentage upon finishing conversation with the chat bot.

Website hosted with Azure

Challenges we ran into

Speech to Text - attempted multiple methods of changing speech to text, experimenting with multiple APIs, such that we were able to convert speech accurately into text into readable files

Speech analysis - understanding the method of grammar analysis to use and comparing it to a baseline

Limitations of Azure API - troubles converting BLOB to Wav files in JavaScript - but found an alternative solution

Accomplishments that we're proud of

Experimenting with new technologies Well-flowing user interface Integrating all our work together Working on a project that could benefit many people

What we learned

Without teamwork and supporting each other through the 24 hours, we wouldn't have been able to complete this project. Some seemingly simple concepts can be much more complex than we thought.

What's next for TicTalk

Chat bot speaks aloud Additional metrics for speech analysis Suggestions for conversations with the chat bot Create a social component (and link to social media) to the app, where you can practice with a friend and use the platform as an analysis tool