Inspiration
Our inspiration came from witnessing the challenges faced during natural disasters. We saw an opportunity to leverage AI to improve search and rescue operations and streamline debris detection, aiming to enhance disaster response efforts and aid in community recovery. We first got inspired by the fact that drones are the main source of data which we can get as input, i.e. a birds eye view of the disaster.
What it does
- Search&Rescue: Helps detecting and marking people in real time video to let authorities know where survivors are after a disaster. This will help in effective rescue operations.
- Debris Quantifier: Analyses video footage of the structure after the disaster and tells us how much of each material is present in the scattered debris. It also generates a heatmap of the debris on google maps. This will help in recycling/disposing the debris effectively, speeding up the rebuilding process.
How we built it
We first started building the layout of how to build it. We then worked on a simple frontend with a file upload functionality. Meanwhile we also started working on how to detect people from a birds eye view. We implemented a YOLO object detection model which would help us with this. We displayed the processed video as output.
We then moved on to the debris detector functionality. We integrated google maps API for the front end and we also worked on actually calculating the mass of the debris. We selected frames from the video. We analyse the debris from these frames and then send it to a database (MongoDB). We then map it on the google maps API using a heatmap functionality. We then changed the Search&Rescue functionality for the app to work with real time video footage.
We also added a functionality which looks at your house and tells you if there is any structural damage or weathering.
Challenges we ran into
We wanted to train our own ML model to detect debris but we couldn't find any dataset online. There was no dataset with a huge number of just debris. We combatted this by using Gemini which turned out to be pretty accurate.
What we learned
State of the art models like YOLO, How to integrate google maps and generate heat maps over it.
What's next for ResQVision
We want to train our own debris detection model and maybe use some sensors which measure stress and strain in buildings and alert authorities the second they sense fluctuations.
Log in or sign up for Devpost to join the conversation.