:. Curie .:

Assist google street view access to visually impaired people

Comment

Inspiration

We wanted to work with AI, and we found a way of making a technology we take for granted more inclusive.

What it does

It sends multiple Google Street View images to a Google vision AI which analyzes the images and then provides textual information to GPT which describes the scene. This makes Google Street View accessible to visually impaired people who can navigate through their computer only through text to speech.

How we built it

We built a website using HTML, CSS, and Javascript. Using multiples APIs, we created a process of geolocation, image analysis, and AI feedback.

Challenges we ran into

Time loss in learning the packages, libraries, and APIs along with the debugging and putting pieces together are challenging. We were experimenting with our own ideas and expecting a full, talented team at the welcome ceremony, but it never came into fruition.

Accomplishments that we're proud of

As groups of two, we are proud of the interactive landing page and beta-version of a web application using mainly the basics of web development in short period of time is impressive. Working along side with mentor is an asset towards product development and real-world experience.

What we learned

Relearned a lot of web development practices, learned how to call and use Google Cloud Map APIs as well as the Vision AI API. Learned how to process and convert images to make them compatible with certain APIs. Learned promises in Javascript.

What's next for Curie

The application will need a more robust backend system to handle the front-end as users and data increase. Generating a more advanced prompt to interact with is a possibility due to the fast development of Open AI (GPT-4 API, DALL-E 2).

Built With

Share this project:

Updates