-
-
Our Model detecting litter
-
The detected litter shows up on our app where volunteers can view its location
-
Volunteers can then clean up the litter and mark it off on our app
Inspiration
Every year, the US government alone spends $20 billion on cleaning up litter. The irony, however, is that most of this money actually goes into finding out where the litter is and finding/paying people to clean it up. If we can find a way to make it easier to find and pick up litter, we could potentially save billions of dollars, all while making it easier to keep our cities, oceans, and country clean and litter-free.
What it does
JunkViz consists of three separate parts: an AI model, a backend, and a mobile app. Our AI model is meant to be deployed to existing CCTV cameras around the country and is able to automatically detect litter on the ground.
Once our model detects litter, it sends a picture of the litter to our backend, where it is stored in our database and volunteers are notified of the litter's existence/detection.
Once notified, the volunteers can open the JunkViz app where they can see the location of the litter, go and pick it up, and mark that they have done so on the app.
The way we want to ensure that any existing camera can run our model is through the Azure IoT Edge service, which is essentially a "Docker for IoT devices" that enables us to pull the model from the cloud onto a particular camera. This means that any camera which connects to the cloud could theoretically run our model, making our solution scalable and cost effective.
How we built it
Our AI model is built off of the EfficientDet model and reads in video as RGB channel images, classifies each image, and then sends a request to our backend server with the classified image in order to upload it.
Our backend server is built in Flask and is connected to an MongoDB database and an S3 bucket for image storage, and our app (frontend) is built in React Native.
Challenges we ran into
We had a lot of trouble linking our machine learning model to our backend, especially because we often ran into issues with the model uploading files that were too large or taking too long to classify a particular image. We also had to play around a lot with our model and fine tune its activation threshold to ensure that we didn't accidentally classify non-litter as litter. Since there is no truly efficient litter detection model in existence, we had to do a lot of research to figure out how to effectively build this model.
Accomplishments that we're proud of
We're proud that we were able to link all the different parts of our system together and get a working MVP out before the end of the hackathon. Although we have lots of potential improvements and tuning possible in the future, we really love this idea and think that we were able to do it justice by being able to effectively craft so many distinct connected parts of this overarching system.
What's next for JunkViz
- Tuning the model for more effective classifications
- Introducing Azure IoT edge for our model/classification script in order to pull it into embedded systems/cameras in the real world
- Creating a reward system for volunteers/employees who are using the app
Log in or sign up for Devpost to join the conversation.