StormSurge | Devpost

Animal Detection
Damage Detection
Model Performance
Agent System
Damage Heatmap

Inspiration

As lifelong Florida residents, we've gone through numerous hurricanes and witnessed their devastating impact on wildlife and infrastructure. Our solution addresses three critical issues:

Animal displacement: Hurricanes force countless animals from their habitats, necessitating rescue efforts.
Damage assessment: Current methods relying on costly satellite imagery can be streamlined for efficiency.
Pre-hurricane preparedness: While we anticipate storms, their severity often remains uncertain until landfall.

Our innovative approach tackles these challenges head-on, offering a comprehensive solution to mitigate the effects of hurricanes on both wildlife and communities.

What it does

Our innovative platform uses the power of advanced AI technology to transform simple drone footage into comprehensive damage and wildlife assessment reports. The user-friendly interface integrates two key features:

Wildlife Detection: Our sophisticated neural networks automatically identify and locate displaced animals within the drone footage, making rescue efforts more efficient for pet owners and wildlife advocates.
Structural Analysis: By combining Convolutional Neural Networks (CNN) with Large Language Models (LLM), we provide detailed documentation of building damage and infrastructure assessment
Damage Prediction: Using Random Forest models on custom datasets created with web-scraping and sentiment analysis has allowed us to generate heatmaps of projected damages given the path of a hurricane.

How we built it

When building this product, we kept innovation in mind. Instead of directly calling LLMs for inference, we developed a robust agentic architecture using Microsoft Autogen to produce the PDF documentation. Think about it, 1 person doing 10 tasks vs 10 people doing 1 task. 10 people doing the task will make the production much better.

Additionally, we utilized state of the art models called YOLO and trained it on cloud GPUs and large datasets of over 50,000 images total. With this we had our agents include the YOLO inference as a tool and be able to fuse the 2 together.

As for the damage prediction model, we curated a dataset of over 100 entries from past hurricanes by Web Scraping news headlines from various cities and conducting sentiment analysis with NLTK. After that, we utilized scikit-learn's RandomForest Model and trained the custom data on it.

Challenges we ran into

Ideas are hard when coming into a hackathon. The initial idea was to predict where animals will go when a hurricane occurs. They oftentimes leave and hide in certain spots and we wanted to find those spots. The issue with this is the fact that there is very little if not any data on this. It is hard to predict this without the use of NLP models to read social media messages or news headlines. Regardless, we still built our own agent that would predict where animals could be hiding.

Accomplishments that we're proud of

Developing an Agentic Architecture instead of simply calling LLMs was a very cool experience, along with training 3 different ML models on several datasets.