Saight

Saight is an AI-based assistance with a smartphone mount, which can be attached to any white cane. This can help the visually-impaired navigate the world.

Comment

Inspiration

One of our teammate’s grandfathers suffers from diabetic retinopathy, which causes severe vision loss. Looking on a broader scale, over 2.2 billion people suffer from near or distant vision impairment worldwide. After examining the issue closer, it can be confirmed that the issue disproportionately affects people over the age of 50 years old. We wanted to create a solution that would help them navigate the complex world independently.

What it does

Object Identification:

Utilizes advanced computer vision to identify and describe objects in the user's surroundings, providing real-time audio feedback.

Facial Recognition:

It employs machine learning for facial recognition, enabling users to recognize and remember familiar faces, and fostering a deeper connection with their environment.

Interactive Question Answering:

Acts as an on-demand information resource, allowing users to ask questions and receive accurate answers, covering a wide range of topics.

Voice Commands:

Features a user-friendly voice command system accessible to all, facilitating seamless interaction with the AI assistant: Sierra.

How we built it

  • Python
  • OpenCV
  • GCP & Firebase
  • Google Maps API, Google Pyttsx3, Google’s VERTEX AI Toolkit (removed later due to inefficiency)

Challenges we ran into

  • Slow response times with Google Products, resulting in some replacements of services (e.g. Pyttsx3 was replaced by a faster, offline nlp model from Vosk)
  • Due to the hardware capabilities of our low-end laptops, there is some amount of lag and slowness in the software with average response times of 7-8 seconds.
  • Due to strict security measures and product design, we faced a lack of flexibility in working with the Maps API. After working together with each other and viewing some tutorials, we learned how to integrate Google Maps into the dashboard

Accomplishments that we're proud of

We are proud that by the end of the hacking period, we had a working prototype and software. Both of these factors were able to integrate properly. The AI assistant, Sierra, can accurately recognize faces as well as detect settings in the real world. Although there were challenges along the way, the immense effort we put in paid off.

What we learned

  • How to work with a variety of Google Cloud-based tools and how to overcome potential challenges they pose to beginner users.
  • How to connect a smartphone to a laptop with a remote connection to create more opportunities for practical designs and demonstrations. How to connect a smartphone to a laptop with a remote connection to create more opportunities for practical designs and demonstrations. How to create docker containers to deploy google cloud-based flask applications to host our dashboard. How to develop Firebase Cloud Functions to implement cron jobs. We tried to developed a cron job that would send alerts to the user.

What's next for Saight

Optimizing the Response Time

Currently, the hardware limitations of our computers create a large delay in the assistant's response times. By improving the efficiency of the models used, we can improve the user experience in fast-paced environments.

Testing Various Materials for the Mount

The physical prototype of the mount was mainly a proof-of-concept for the idea. In the future, we can conduct research and testing on various materials to find out which ones are most preferred by users. Factors such as density, cost and durability will all play a role in this decision.

+ 3 more
Share this project:

Updates