ExploraVist

Making a AI device for the visually impaired that describes the users surroundings on command. By offloading processing to Gemini it can be made for $50, beating existing devices that cost $2500+

Comment

Inspiration

ExploraVist was inspired by the prohibitive costs of assistive technology for the 295 million people worldwide who suffer from severe visual impairments. The advent of wearable AI devices that can relay a user's surroundings using a camera has been game-changing for those who can afford it. Yet, the prohibitive costs of existing devices have kept these groundbreaking tools out of reach for many, with all existing solutions on the market being at least $2500. Our vision is to democratize access to these life-changing technologies by making them affordable and accessible to all, regardless of economic status.

What it does

ExploraVist is an AI-powered device designed to aid individuals with visual impairments by providing descriptions of their surroundings on command. Mountable on any pair of glasses, this device captures images and uses Gemini's API and Google Translate’s text to speech models to process and relay audio descriptions directly to the user. From reading road signs, to finding lost items, to interpreting complex text, ExploraVist offers a range of functionalities that cater to the daily challenges faced by visually impaired people. Our use of cloud-based processing and low-cost hardware components allows us to offer this device at a fraction of the cost of current market offerings, without compromising on performance.

How we built it

Our prototype was developed using a 3 dollar microcontroller and other open-source off-the-shelf hardware, with the entire setup costing less than $35 in components. Additionally, by focusing on cost and manufacturability during our hardware development, we have set ourselves up for a mass production cost of under $20. The prototype integrates a ESP32-s3 microcontroller for processing, a camera for capturing images, a microphone for voice commands, a speaker for audio feedback, and a large battery giving a 16-hour lifetime.

Challenges we ran into

The primary challenge was creating a device that was not only cost-effective but also reliable and user-friendly. Essentially, our hardware-first approach left us with very little memory and processing power to achieve our goals, making the CS side of the project difficult. In addition, with speed and hardware limitations in mind, all development was done with C++ in the Arduino framework, leaving us with much less breadth of capability and community support for development. A secondary issue was hardware development, which required multiple months of research and trial and error to land on a combination of cheap, reliable, and safe open-source components to use.

Accomplishments that we're proud of

Our team is proud to have developed multiple generations of working prototypes that effectively demonstrate our vision. Additionally, we visited a school for the blind and received a lot of positive feedback on our prototype, which was a awesome and motivating moment. Above all, we are proud of our cost: We have made a formidable alternative to existing $3000 devices for only $35!

What we learned

Throughout this project, we've gained many insights into the unique and often unexpected challenges faced by visually impaired people, and learned how to best address them within our design process. In addition, as college students, this challenging project has really forced us to level up on the technical side, improving our hardware, software, and human-oriented design skills across the whole team.

What's next for us

Looking forward, in the next few weeks we aim to give our device to visually impaired members of our community for multiple weeks. Based on this feedback, we want to work out the final kinks in our design, and pair it with the phone app we are developing. Then we hope to move from making a few prototypes by hand to small-scale production, getting our PCB designs made in larger batches and bootstrapping the remaining assembly of the devices ourselves. In addition, we want to open source our code and hardware designs so anyone with a soldering iron can make our original (and equally functional) design. Finally, we hope to scale further, bringing the life-changing independence provided by these devices to people around the world, not just those who can afford it.

Built With

Share this project:

Updates