-
-
Audibly being used in Google Meets
-
Audibly app interface
Audibly 👋💬 is a real-time accessibility app that translates sign language into live captions and speech, as well a speech into text. Audibly is designed for fast-paced meetings on platforms like Zoom or Google Meet, where deaf or mute participants are often unintentionally left out.
✌️ How It Works
Two-Way, Real-Time Communication:
- Sign-to-Text and Speech: Instead of rushing to type, users can respond instantly using sign language, which is translated into both on-screen text and spoken audio.
- Speech-to-Text: Spoken conversation is converted into live captions, ensuring accessibility for deaf and mute participants.
No Plugins or Permissions Required: Audibly operates by creating a virtual camera for the user. This allows it to work instantly with existing video conferencing platforms without requiring plug-ins, administrative approvals, or special authorization from an organization.
Enterprise-Friendly by Design: Because Audibly functions as a standard camera input, users can join meetings immediately, even in locked-down corporate or educational environments.
🤝 Why This Matters
Audibly overcomes accessibility bias in live conversations. Deaf and mute individuals can fully participate in conversations without fear of being missed, ignored, or falling behind. From classrooms and hackathons to professional and virtual meetings, Audibly addresses a daily communication gap that affects inclusion, confidence, and collaboration.
⚙️ Hard Problems We Solved
Sign language recognition demands accurate, low-latency computer vision that can interpret nuanced hand movements and timing without interrupting live conversation. We trained a custom TensorFlow machine learning model to recognize sign language in real time, and integrated VOSK for low-latency speech-to-text and pyttsx3 for immediate text-to-speech. All of these technologies run locally on the user’s computer, which makes Audibly faster, more reliable, and more private since no video or audio needs to be sent to the cloud.
🔮 Where This Goes Next
Audibly’s setup makes it easy to add support for more sign languages, such as Indo-Pakistani Sign Language, Chinese Sign Language, and British Sign Language. New models can be trained and plugged in without changing the rest of the system. With more training data and optimization, the translations can also become faster and more accurate over time, making conversations even smoother.
Log in or sign up for Devpost to join the conversation.