VOIA | Devpost

Project Description

VOIA is a Mixed Reality application designed primarily for deaf and hard-of-hearing individuals in Singapore, and they are also the main users of the Mixed Reality headset. The project combines practical communication tools with gamified learning experiences to support both real-time communication and sign language learning.

For practical communication, the Mixed Reality app enables two-way real time translation. It translates spoken words into on-screen text for non-hearing users, and sign language into voice output for hearing individuals. This allows deaf and hearing individuals to communicate more conveniently, even from a distance, without relying solely on interpreters or written messages.

For learning and engagement, VOIA features an interactive 3D storybook experience, where users sign specific words to progress through the narrative. This transforms sign language learning into an immersive and enjoyable activity, offering an alternative to traditional, static sign language books.

The Mixed Reality headset is supported by a companion website that extends learning beyond the headset. The website allows users to track their learning progress, practise through sign language spelling quizzes, and stay motivated by earning badges and points. VOIA can also be extended to be used by hearing individuals, who can engage with both the Mixed Reality app and website to learn sign language, track their own progress, and gain greater awareness of the deaf community, ultimately fostering more inclusive communication.

VOIA matters because over 500,000 people in Singapore experience hearing loss and most hearing individuals are not taught sign language in schools, leading to communication barriers and a lack of understanding of the deaf community. Access to communication is essential for social inclusion, education, and everyday participation, but deaf and hard-of-hearing individuals are often excluded from conversations that rely heavily on spoken language. In many real-world settings, deaf and hard-of-hearing individuals depend on interpreters or written communication, which can limit independence, reduce spontaneity, and create feelings of exclusion. Additionally, both deaf and hearing individuals often lack access to interactive and engaging ways to learn sign language, with existing resources frequently limited to static or text-heavy formats. This makes learning less motivating and reduces opportunities for meaningful understanding and communication between the two communities. Hence, VOIA aims to integrate communication, learning, and motivation into a single, connected ecosystem that benefits both deaf and hearing individuals.

VOIA creates measurable impact by enhancing communication, learning, and inclusion for deaf and hard-of-hearing users while raising awareness among hearing participants. Conversations without interpreters or written messages can be measured through successful exchanges and reduced misunderstandings. Learning progress is tracked via the storybook and website through quiz scores, badges, and story completion rates. Real-time participation boosts users’ confidence and inclusion, and it is assessed through surveys and interviews, and hearing users’ awareness and sign knowledge are measured before and after test results. Prototype testing using user interviews, observations, and activity tracking have validated these outcomes, combining quantitative and qualitative metrics to demonstrate VOIA’s impact.

Impact Explanation

VOIA addresses a real-world accessibility challenge faced by the deaf and hard-of-hearing community in Singapore, an underrepresented group often excluded from mainstream communication and educational technologies.

The project was informed by interviews with various users, including deaf individuals and beginner sign language learners. These sessions highlighted common pain points such as how hearing individuals do not understand sign language, difficulty keeping up with spoken conversations, especially in group or fast-paced settings and a lack of engaging tools for learning and practising sign language. Users also emphasized the importance of clear visual feedback, real-time responsiveness, and interactive learning, which directly shaped VOIA’s design decisions.

During prototype testing and user feedback sessions, users reported that VOIA significantly improved both their engagement with sign language learning and their ability to communicate naturally. Our deaf user highlighted that the real-time translation allowed them to participate in conversations without relying on interpreters, which increased their confidence and sense of inclusion in social and educational settings. User also noted that the storybook-based learning was a unique and engaging approach to learning sign language. They enjoyed progressing through the interactive storybook and appreciated the visuals and overall immersive experience. Hearing users noted that VOIA provided an accessible and motivating way to learn basic sign language and gain awareness of the deaf community, fostering greater empathy and understanding. Overall, user feedback demonstrated that VOIA’s design effectively supports learning, communication, and social inclusion, validating its impact in real-world scenarios.