DashBuddy

Check your dash, prevent a crash!

Comment

Inspiration

Have you ever turned on your car to a dashboard full of alarming, unspecific, flashing symbols ⚠️❗️❌? All of a sudden, you’re not sure if you just forgot to fill that windshield washer fluid💧 or if there’s a bigger problem that’s going to leave you stranded in the middle of nowhere. Modern dashboard symbols are nearly impossible to decipher and can put you in a panic🤯. DashBuddy is an app that can identify warning symbols and their potential causes, potentially saving you from a costly trip to the auto repair shop🚗⚙️!

What it does

The DashBuddy app lets you take a picture 📸 of your dashboard and identifies the warning symbol that is displayed. It then provides you with information on what your vehicle's issue could be and proposes a solution💡.

How we built it

We created our custom dataset by annotating 200+ images of dashboard warning symbols using Roboflow💻. We generated a training set of 400+ images using data augmentation for our custom object detection training model with a precision of 95%📈. The detected result is added to the prompt for the Cohere API, which uses language AI and is built on the GPT-3 model🤖 to determine what the issue is and generate a solution for the symbol🛠️. The Frontend of the mobile application was implemented using React-Native after designing and prototyping in Figma📱.

Challenges we ran into

A significant challenge we faced was the scarcity of datasets containing car dashboard images to train our image-detection neural network. We decided to instead create our own dataset using Roboflow, by taking and annotating 200+ images of dashboard symbols. Another one of our challenges was integrating the camera📸 functionality in React-Native with the object detection API.

What's next for DashBuddy

In the future, we hope to implement a vehicle damage assessment feature that suggests the appropriate action to take for external damage🚗💥.

Share this project:

Updates