HazardVision

HazardVision never blinks. While humans overlook risks, our AI watches every tool, every move, and prevents accidents before they occur.

Comment

Inspiration

Every day, nearly one million workers are injured on the job—and the majority of these injuries are entirely preventable. Visual hazards like unattended tools, blocked exits, and missing PPE (personal protective equipment) are responsible for most of them. Yet while AI innovation in fields like healthcare and finance accelerates, blue-collar industries—where lives are at stake daily—remain underserved.

Workplace safety is a crisis in the U.S.:

4,764 workers died on the job in 2022 (U.S. BLS)

That’s one death every 96 minutes

Non-fatal injuries cost the U.S. economy $167 billion annually (NSC, 2022)

We built HazardVision to bridge this safety gap. Our mission: bring real-time, intelligent oversight to factories, warehouses, and worksites—places where one missed step can cost a life.

What it does

HazardVision is an AI-powered safety assistant that monitors the workplace through real-time video. It automatically detects and classifies hazards, then alerts workers before accidents occur.

It detects:

Trip and fall risks (e.g., tools, cables, spills)

Improper PPE usage (e.g., missing helmets, gloves)

Unsafe equipment handling (e.g., stagnant knives, exposed saws)

Blocked emergency exits or cluttered walkways

Our system uses:

Live webcam input

Bounding boxes with severity-coded colors (green/yellow/red)

Real-time hazard logs

Audio alerts with pitch based on severity

It even adjusts classification based on context: a tool in use may be low-risk, but abandoned becomes a critical hazard. And if multiple hazards persist unattended, our AI agent can escalate by notifying a manager or safety officer—ensuring OSHA-level accountability.

How we built it

We designed a full-stack real-time computer vision system with agentic capabilities:

Frontend:

HTML/CSS/JavaScript for the dashboard

MediaDevices API for webcam access

Canvas API for real-time bounding box rendering

WebSockets for bi-directional communication

AudioContext API for hazard-specific alert tones

Backend:

FastAPI (Python) + Uvicorn for high-speed async handling

Ultralytics YOLOv8 for object detection (PyTorch under the hood)

OpenCV for image annotation

Custom hazard classification logic (severity scoring, event triggers)

Agentic pipeline powered by Vapi and ElevenLabs for escalated alerts

The frontend captures frames and sends them via WebSocket to the backend. There, the YOLOv8 model detects tools, objects, and unsafe setups. Based on context, we classify hazard severity and return bounding boxes and labels. If the situation escalates (e.g. abandoned blades or multiple unmitigated risks), our AI voice agent contacts factory management with a violation notice.

Challenges we ran into

Ensuring inference speed was fast enough for smooth live video

Designing a severity framework adaptable to various real-world tool usage

Syncing audio, visual annotations, and hazard logs in real-time

Learning how to implement agentic workflows using Vapi and ElevenLabs

Finding appropriate data to train YOLO for real manufacturing environments

Testing the model with real materials (tools, machines, gloves, etc.)

One of our teammates had to leave midway—requiring us to re-plan, regroup, and rebuild

Accomplishments that we're proud of

Built a complete real-time detection and alert system from scratch

Successfully trained a CNN to identify hazards and contextualize them based on environment

Achieved low-latency AI inference using YOLOv8 on webcam input

Created a live hazard dashboard with severity mapping, log tracking, and alert sounds

Formed a resilient team from across different countries and universities

Rebounded and finished the project strong, even after a teammate left mid-hackathon

What we learned

How to optimize AI models for real-time use in edge environments

How to integrate new technologies like Vapi and agentic voice systems

How to design for industrial UX—where alert clarity and speed can be life-saving

How to collaborate across time zones, adapt when things break, and keep pushing forward

What's next for HazardVision

Expand detection to fire, gas leaks, electrical sparks, and smoke

Partner with manufacturing labs and warehouse operators for field testing

Build a plug-and-play camera module for easy installation in existing CCTV setups

Continue training on more diverse datasets for better generalization

Bring the system to small and mid-sized factories across the U.S.—where safety staff is minimal and the risk is high

Built With

Share this project:

Updates