Echo App

Echo is a helpful desktop pet for K-5 students that moves their desktop without using a keyboard or mouse pad by listening to what they say and controlling a pointer with their eye movements.

Comment

Inspiration

Echo is an accessibility tool that works together with children in elementary school who struggle using the keyboard and mouse pad. By removing the reliance on the keyboard, Echo will give children, especially those with motor control and emotional regulation difficulties, a greater sense of control over their device and give them more control over their computer with their body. Our goal is by using an embodied computer system with readily available hardware, children will feel empowered as they navigate through their online school.

What it does

Echo tracks users eye movements using the webcam and listens to users through their built in microphone, integrating both voice and eye movement as a way to navigate around their keyboard. Echo will pop up in their browser and respond to their commands while also providing helpful tips to make using their computer easier.

How we built it

All in an electron app, Echo uses WebGaser to track users eye movements once the mouse has been trained. We also use Azure to access the user microphone and run commands through speech to text.

Challenges we ran into

Integrating all of the code within such a short time frame and managing git version control is challenging. We hope with time

Accomplishments that we're proud of

We're proud of how we managed to scrap together a working demo under a 2 woman team and being incredibly resilient even when two of our team members dropped out due to unforeseen circumstances.

What we learned

We learned how to integrate many different publicly available technologies while learning how to manage versions using Git as well. The coolest to learn is WebGazer, an online webcam eye tracking tool with uses ML through tensor flow. It's also really exciting how many technologies we got to use that are readily available and utilize cloud servers along with AI. We learned how to do full stack web development and file management as well, feeling quite like an integrated team.

What's next for Echo App

We will continue utilizing out network to build this app and hopefully have it publicly available for Special Education programs nationwide.

Built With

Share this project:

Updates