SnapSpeech

An application for students to enhance readily available resources for non-native English speakers as well as deaf students.

Comment

Inspiration

We hope to create a tool that enhances the learning experience for marginalized communities, particularly students with hearing disabilities and those who speak English as a second language. We seek to balance out education inequality in a classroom by providing students with equal opportunities to learn.

What it does

Our tool takes in audio in real time, converts the audio to text, and prints the live text to the domain while displaying companion images of complex words that are difficult to comprehend. With visual aids presented seamlessly as their teacher is speaking, students have a better learning experience.

How we built it

We wrote a Python script using Google's Text-to-Speech API to read a stream of microphone input and transcribe it into text. We then created a TCP/IP socket connection between the text-to-speech Python script (the server) and Flask web application page (the client) so the text could be printed along with images, in real time to the domain.

Challenges we ran into

User challenges - we have a large user demographic, and given the time constraint we were unable to spend one-on-one time with students for in-depth user studies. Finding a metric to determine the "difficulty" of words - we ended up using a list of the most frequently used English words, which only had 5000 entries. Programming - We weren't experienced with a lot of the tools/languages, especially Javascript, which was necessary to display the images to the web page in a dynamic manner.

Accomplishments that we're proud of

COLLABORATION: Although we formed our team from scratch yesterday, we were constantly communicating our progress with each other, asking for help, and celebrating every success--no matter how small. Even though everything didn't go as planned, we're incredibly proud of how much we've accomplished through our teamwork. TECHNICAL IMPROVEMENT: We've never used Python, APIs, or Javascript before, so these past 24 hours have been filled with a lot of experimentation and learning. We originally couldn't get the text-to-speech API to work, but persisted and found a solution. We're really proud of how far we've gotten.

What we learned

We've learned a lot about APIs, Python, and Javascript.

What's next for SnapSpeech

  • unfinished features: TCP/IP connection, JSS, supporting multiple users within our domain, creating options for specific subjects and languages
  • connect with gmail, docs, drive, classroom
  • offer tests to place student into appropriate level (assessment to continually improve the functionality)

Built With

Share this project:

Updates