Argus | Devpost

Inspiration

We saw that not all crosswalks are equipped with the 'beep-boop', so we wanted to create a technology that could help make cross-walks more accessible to the visually impaired.

What it does

Our app uses machine learning, specifically object detection, to identify when a crossing signal says to walk or not and can speak 'stop' or 'go' out loud to direct a visually impaired person when they are at an intersection.

How we built it

1) Gathering & Labeling Images We went up and down Forbes Ave and took pictures of crosswalk signals from different angles, at different times to gather a database of over 500 pictures to help train our model on recognizing crosswalk signals. Then we classified the images into the categories of 'stop' (the red hand) and 'go' (the walking icon), and then drew boxes around the red hand and walking icons and labeled them to 'stop' and 'go', respectively.

2) We used the TensorFlow software library developed by the Google Brain team to train an algorithm, over the course of ten hours, into recognizing the crosswalk signals and turning that into something that can be deployed on IOS devices.

Challenges we ran into

Deciding between Single Shot MultiBox Detector (SSD) and RegionalConvolution Neuro Network R-CNN (two types of object detection models) where SSD is faster but less accurate, and RCNN takes more time to train but yields more accurate results. Because of our time conditions and hardware limitations, we chose to go with SSD.

Another challenge we faced was how to convert the trained model into an app that can run on IOS devices.

Accomplishments that we're proud of

How accurate the model is given our time and data constraints (ie. only a couple hundred pictures).

What we learned

We learned the basic techniques for training a model to detect certain objects. We also learned that two hours is not a sufficient amount of time to create a decent video for our project.

What's next for Argus

We would like to integrate GPS services, so that it can detect when a person is at an intersection. We would also like to integrate motion detection to detect vehicles for additional safety. And being able to run the model on an android device.