Sharingan | Devpost

Inspiration

Every year floods, earthquakes, and building collapses put rescuers in the same position: they know people are trapped somewhere inside, but not where, and every door they open blind costs time they don't have. We wanted a scout drone that could fly in ahead of a rescue team and tell them who's inside and where before anyone crosses the threshold.

The same problem shows up outside disaster response. A soldier clearing a room, a SWAT team going through a doorway, a firefighter deciding which door to breach first, all of them are making the same blind call under time pressure about what's on the other side. We built for search and rescue first because the stakes are the cleanest there, you're trying to save someone. But the core problem, knowing what's on the other side before a person has to find out the hard way, is the same one soldiers and first responders deal with every time they clear a structure.

What it does

SHARINGAN is a scout drone that flies ahead of a person, finds people through walls, floors, and openings using onboard computer vision and radar, and streams what it sees to a Meta Quest 3S headset worn by the operator, live, as an annotated tactical view. There's no cloud and no dependency on venue Wi-Fi. The link is wired and air-gapped, and we prove it on camera by pulling the network cable mid-flight and showing the feed keeps running.

The drone detects people, tracks them on its own, and relays their position to the ground in real time. That's the part that makes it actually usable in the moment instead of just interesting sensor data: the operator isn't staring at a laptop trying to interpret a feed, they're looking through the headset at tracked positions overlaid on the space in front of them. Know who's where before you decide what to do next. If a detection goes stale, the HUD says so, "SENSOR OFFLINE," instead of freezing on a last known frame and quietly lying to the operator. We also built a CoT/ATAK bridge so the same detections can flow into existing tactical mapping software, not just our own headset.

How we built it

The onboard compute is a Raspberry Pi 5 paired with a Hailo-10H AI accelerator, running person detection on a mono global shutter camera for line of sight and through opening cases, backed by radar for the through wall and floor case. We skipped stereo cameras on purpose: with two cameras that close together, depth error grows with the square of distance, so it falls apart right where you'd need it most. Instead we get bearing from pinhole camera geometry and range from a boresighted rangefinder, which stays accurate at the ranges we actually fly.

Flight autonomy runs on ArduPilot's Loiter and PosHold modes. The pilot still arms, launches, and positions the aircraft, but once it's in position the flight controller holds itself steady and the computer vision stack tracks on its own, so nobody is hand flying while also trying to read a video feed. Rig pose comes from a phone's ARKit VIO right now, with a Pi native alternative in progress that uses the IMU plus the same detection camera and drops the phone entirely. On the ground, the Quest 3S HUD is built natively in Unreal Engine with real WebXR passthrough, not a mockup, so detections show up overlaid on what the operator is actually looking at.

Challenges we ran into

Getting depth right without stereo took real iteration. The geometry plus rangefinder approach only clicked once we saw stereo's distance squared error problem show up in testing. Radar was harder: most of what's on the market doesn't meaningfully get through a typical wall at useful range, so getting a through wall reading that's actually trustworthy, not just a blob, took real work on sensor selection and signal processing. Keeping the link wired and air-gapped instead of leaning on venue Wi-Fi also meant solving problems most hackathon teams skip, since there's no cloud fallback if anything drops, so the whole pipeline had to work on a closed local link with nothing to fall back on.

Accomplishments that we're proud of

We built across five domains that don't usually sit in one team's hands at a hackathon: embedded compute, computer vision, radar, flight autonomy, and native VR development, and got all of them talking to each other on a live link by demo time. We're proud that the safety behavior is real and not just a feature on a slide: a stale detection actually shows as offline instead of hiding the problem. And we're proud we kept the engineering honest through the whole build. When something didn't work the way we wanted, like stereo depth, we changed approach instead of shipping the version that looked better on paper.

What we learned

Staleness handling matters as much as detection accuracy. A HUD that goes quiet when it should, instead of showing a confident looking stale frame, is the difference between a tool someone trusts and one that gets someone hurt. We also learned that cutting the cloud dependency, which felt like a limitation early on, turned into one of the strongest parts of the system. It works in a basement, a disaster zone, or a building with no signal, which is exactly where this needs to work.

What's next for Sharingan

Tightening the radar's resolution and range so it's not just a proof of concept but something an operator would actually rely on. Finishing the Pi native VIO path so the system doesn't depend on a phone at all. And testing the multi-node story for real: the architecture is built so a second and third drone are a scaling problem, not a redesign, and we want to actually prove that instead of just asserting it.Let me know if you want this saved to the project as a doc, or any more trims before you paste it into DevPost.