save-splat

Reads the rubble so the crew can decide where to dig first.

Comment

🛰️ save-splat

One scan. Every measurement. Zero guesswork about who to reach first.

save-splat is an AI tool for disaster rescue teams. It takes a LiDAR gaussian splat scan of a collapsed building, pulls real measurements out of it like wall lean and debris volume, and weighs how many people might be trapped alive to rank which sites a crew should reach first.

For the crew, it turns "that wall looks bad" into a drift number you can act on. For incident command, it replaces a whiteboard and a gut feeling with a ranked list that shows its work. It measures the damage and lays out the priority. A person still makes every call.

Inspiration 💡

After a big earthquake, a search-and-rescue team can show up to dozens of collapsed buildings with only a few crews to send. The order they work those sites in is, bluntly, who lives. That call is usually made with tools that do not talk to each other and do not remember anything: a handheld scanner, a paper recon form, an engineer's eye, and a whiteboard with site names on it. None of them turn a scan into a number you can compare one building against another.

There is a second problem too. "That wall looks bad" is not a measurement. Is it leaning 1 percent, or 6 percent and about to pancake onto the crew? Without a number you cannot rank the sites, you cannot hand the decision to someone else, and you cannot defend it afterward.

We wanted to take the scan a crew can already capture on a phone and get a real answer out of it, fast, in a form command can argue with.

What it does ✨

Feature Description
Scan ingestion Loads a Scaniverse or Polycam gaussian-splat .ply, or a .glb/.gltf mesh sampled into points. Streams with progress and guards against huge files.
Structural extraction Opacity-weighted RANSAC fits planes to the walls, floors, and slabs, with each point weighed by its own covariance.
Wall lean Reads each wall's lean as a drift ratio (rise over run) and sorts it into a severity band with a colour overlay.
Debris volume Finds the ground plane and measures the debris volume above it. A glTF mesh is metric, so the volume comes out in real cubic metres.
Triage ranking rho = (n * q * r * lambda) / max(0.1, tau), which is expected lives saved per crew-hour. Sites sort from highest to lowest.
Agent swarm 🧠 Five server-side agents, one per ranking input, each with its own evidence source and its own check. They propose, a person applies.
A/B scan slots Two slots for a before and after look. A visual toggle only, with no registration between them.

Example Userflows!!!

A crew scans three flagged buildings.

save-splat reads:
  Site 2 has a wall leaning 0.031 (severe), one debris pile of 14.2 m3
  above the ground, and a mixed collapse shape.
  It ranks above Site 1 and Site 3.

*The leaning wall lights up in the severe band on the overlay*
*The engineer sees 0.031, not "looks bad," and decides*

Command reviews the ranking before sending a crew.

They open Site 2. Every input is on screen: how many people are
believed inside, how likely they are trapped alive (no agent, by design),
the extraction odds, the crew-hours, and the collapse type.
The ACCESS agent proposed r = 0.6. The log shows the operator accepted it,
and the value it replaced. Nothing moved a slider on its own.

How we built it 🛠️

Input: a gaussian-splat .ply, or a .glb / .gltf mesh
        |
        v
  Vite + React front end  ->  /api/swarm  ->  reasoner on the server
                                              (Athena, OpenRouter, OpenAI, Anthropic)
        |                                            |
        v                                            v
  core/ (plain TypeScript, no Three.js, no DOM)   Zod validation
        |                                          (reject, do not clamp)
        |                                            |
  parse the .ply / sample the mesh                   v
        |                                     five agents, one per ranking input:
        v                                       RECORDS       -> n
  opacity-weighted RANSAC, seeded RNG             ACCESS        -> r
        |                                         VOLUME        -> tau
        v                                         MORPHOLOGY    -> collapse type
  walls, slabs, inclines                          CORROBORATION -> confidence
        |                                         (trapped-alive has no agent)
        v                                            |
  wall drift (tan of the tilt angle)          each proposal has its own check
        |                                            |
        v                                            v
  debris volume above the ground plane        an operator applies it,
        |                                     logged with the old value
        +--------------------+---------------------+
                             v
                       ranking.ts
                 rho = (n * q * r * lambda) / max(0.1, tau)
                             |
                             v
              3D view, plane overlay, ranked site list,
                     JSON / CSV export

Architecture Overview

Layer Technology Purpose
Frontend Vite, React 18, TypeScript The panel UI and the 3D scene, in one page
State useSyncExternalStore store Site records live here; their markers live in the viewer, keyed by id
Renderer Three.js Camera, a hand-written orbit control, markers, and the plane overlay. Splats draw as THREE.Points
Core logic Plain TypeScript, no Three.js, no DOM Parsing, RANSAC, ranking, export. Tested without a browser
Parsing Custom .ply and gaussian-splat reader Per-Gaussian covariance from scale_* and rot_*, colour from f_dc_*, streaming with progress
Geometry Opacity-weighted RANSAC, seeded (mulberry32) Planes, wall drift bands, debris volume
Validation Zod (core/swarm/schema.ts) Anything from outside is rejected, not clamped
Reasoner Node-only server/swarm/ Prompt, call the provider, run Zod, then verify
Providers OpenRouter, OpenAI, Anthropic, Athena One Reasoner interface, whichever key is set wins
Hosting Vercel functions, plus Vite dev and preview The same /api/swarm/* handlers run in all three
Run log Supabase Postgres pooler (optional) Appends each run to swarm_runs, and a failed write never blocks a run

Geometry and Core (TypeScript)

A gaussian splat is not a plain point cloud. Every splat carries a covariance that says how fuzzy it is, so instead of one fixed threshold for what counts as "on the plane," each point gets its own tolerance from its covariance. A sharp, confident splat has to fit tightly, and a blurry one gets more slack. Opacity weights the fit so faint noise cannot outvote solid structure. A .glb mesh has no covariance, so we sample points evenly across the surface area first (not off the raw vertices, which are dense in some spots and sparse in others) and fall back to a tolerance set by the overall scene scale.

RANSAC is random, but a triage tool cannot be. The random number generator is seeded, so running the extraction twice on the same scan gives the same planes and the same debris volume every time. A regression test builds a floor and a wall leaning exactly 3 percent and checks that the extractor reads back a drift near 0.03 in the severe band.

We do not turn the result into a mesh on purpose. A smooth surface would hide the leftover points that no plane could explain, and that leftover is exactly what tells an assessor how much to trust the numbers.

The one structural rule: core/ never imports Three.js and never touches the DOM. The geometry functions take plain arrays and a matrix instead of a THREE.Points object, which is why the measurement code can be unit-tested without a renderer.

Ranking

The index is rho = (n * q * r * lambda) / max(0.1, tau), which works out to expected lives saved per crew-hour:

  • n is how many people are believed to be inside. This is an operator estimate, not something read off the scan.
  • q is the probability they are still trapped alive.
  • r is the probability of a successful extraction.
  • lambda is how fast survival odds drop for that kind of collapse (a pancake collapse drops far faster than a lean-to).
  • tau is crew-hours, floored at 0.1 so a slider at the minimum cannot divide by zero.

Sites sort from highest rho to lowest. Confidence is kept out of the formula on purpose. It is a flag on how good the evidence is, not a multiplier, so it cannot quietly inflate or shrink a ranking.

The Agent Swarm

There are five agents, one for each input the ranking needs, and each has its own evidence source and its own check:

Agent Input Reads Check
RECORDS n rosters, shift patterns, time of day two independent sources; if they disagree it widens the range instead of averaging it
ACCESS r plane extents, debris columns, route clearance re-run the route with the biggest debris pile removed; r must not jump a band
VOLUME tau debris volume, plane fill, shoring load column volume against a convex-hull bound; a big gap flags an overhang
MORPHOLOGY collapse type plane classes, slab tilt, lean angles, drift re-fit on half the cloud; the collapse type must agree
CORROBORATION confidence how well the other four agreed flags low whenever any upstream agent failed its own check

Three rules keep this safe to put near a real crew:

  • Proposals are rejected, not clamped. If an agent returns n = 500, that is a reasoning failure. Clamping it to 50 would hide the failure inside the ranking, so instead the value is thrown out and the error names the field and the number so the caller can try again.
  • The swarm proposes and the operator applies. Nothing moves a slider on its own, and every application is logged with the value it replaced.
  • The trapped-alive probability has no agent. An exterior scan carries no evidence that someone inside is alive, so we leave that to the operator instead of handing them a made-up number.

Frontend (Vite + React)

The whole thing is one page: a panel UI on the side and a 3D scene in the middle. Three.js does the camera, a hand-written orbit control, the site markers, and the plane and drift overlays, with the scan drawn as THREE.Points rather than a solid surface. State lives in a small useSyncExternalStore store where the site records sit, while the heavier 3D handles live in the viewer keyed by id, so the site list stays plain enough to export and diff. You place a site by pressing m and clicking a structure, and the ranked list updates as you go.

Secrets and the server boundary

No secret ever gets a VITE_ prefix, because Vite would inline it into the code shipped to every visitor. Reasoner keys live only in server/swarm/ and are read from process.env. The browser only ever calls /api/swarm/*, and the same handlers run on the Vite dev server, the Vite preview, and as Vercel functions. Athena is the default when Stripe Projects provisions it, but adding any other provider is one file plus a line in the index.

🚧 Challenges we ran into

Getting a measurement out of a splat without turning it into a mesh. Meshing is the obvious move, and it is the wrong one here, because it smooths over the uncertainty we need to show. Building a RANSAC that keeps the leftover points instead of hiding them took the most thought.

Making a random algorithm repeatable. A crew cannot get a different debris volume each time they re-run the scan. Seeding the RNG and pinning a known-answer test fixed that, and let us check the extractor against a wall we already knew the lean of.

Guarding the input instead of the output. It is tempting to clamp a bad number into range. We reject it instead, because a quietly corrected value is more dangerous than a missing one when the ranking decides who a crew reaches first.

Keeping core/ free of the renderer. The first build hung Three.js objects off each site record, which made the list impossible to export or diff. Splitting the geometry and logic away from the viewer is what makes the numbers testable.

🏆 Accomplishments that we're proud of

✅ A splat-to-measurements pipeline that keeps the leftover instead of hiding it ✅ Opacity- and covariance-weighted RANSAC that is seeded and reproducible, with a known-answer test ✅ Wall lean as a real number (drift is the tan of the tilt), banded and drawn on the scan ✅ A five-agent swarm that proposes and checks but can never move a slider on its own ✅ Zod validation that rejects bad values instead of clamping them ✅ A clean client and server split: the browser only ever calls /api/swarm/*, and no secret is VITE_ prefixed

📚 What we learned

  • The leftover, the part the model could not explain, is often the most useful thing to show, so we stopped hiding it.
  • Rejecting a bad value beats silently fixing it when lives are on the line.
  • A seeded RNG turns "trust me" into "run it yourself and check."
  • Framing every agent as a suggestion a human has to accept is what makes it usable near a real crew.

What's next for save-splat 🚀

Feature Status Description
Multi-scan change detection 🔜 Planned Line up two scans of the same site and see if a wall moved between them
Append-only run log ✅ Shipped A Postgres swarm_runs table records every proposal and check; a failed write never blocks a run
Confidence in the 3D view 💡 Concept Show the leftover fraction per plane right on the overlay
Handoff export 💡 Concept One export of the planes, drift, volume, ranking, and logs for command

The Dream: a crew walks up to a collapsed building, scans it with the phone in their pocket, and within seconds has real structural numbers and a place in the queue that command can defend, with a human making every call and a log they can read back later. The tool reads the rubble. The people decide. 🎉

A note on framing. save-splat maps against the Sendai Framework for Disaster Risk Reduction 2015 to 2030 and the related SDG targets (see PROJECT-BRIEF.md). It is a decision aid. The output is a ranked list for incident command to review, not an automatic dispatch order, and the UI copy is kept honest about that.

Built With

Share this project:

Updates

Submission history