save-splat

Reads the rubble so the crew can decide where to dig first.

Comment

https://projects.dev/leaderboard

save-splat

One scan. Every measurement. Zero guesswork about who to reach first.

savesplat is a engineer-in-the-loop, AI-assisted post-disaster triage viewer built for the people standing in front of a collapsed building. For urban search-and-rescue (USAR) crews, it turns a phone LiDAR scan into hard structural facts — wall lean, slab angles, debris volume — and a ranked order of which sites buy the most lives per crew-hour. For incident command, it replaces gut-feel triage and scribbled clipboards with a defensible, logged, reproducible prior.

It reads the rubble so the crew can decide where to dig — and it never pretends to be the one making the call. (the append-only field run-log just landed; multi-scan change detection is next!)

Inspiration 💡

After an earthquake, a USAR team can face dozens of collapsed structures and a few crews. The order you visit them in is, bluntly, who lives. And that decision today is made with disconnected tools that don't remember anything: a handheld scanner here, a paper recon form there, a structural engineer's eyeball, a whiteboard with site names and arrows. None of them talk to each other, and none of them turn a scan into a number you can argue about.

There's a second problem nobody says out loud: "that wall looks bad" is not a measurement. Someone points at a leaning façade and everyone nods — but is it a 1% drift or a 6% drift about to pancake onto the crew? Without a number, you can't rank, you can't hand off, and you can't defend the decision afterward.

For the crew: Drop in a Scaniverse/Polycam .ply and save-splat extracts the structural facts — walls, slabs, inclines, wall drift in severity bands, debris volume above the detected ground plane — and lights up a ranked dispatch list. It handles the geometry so the engineer can focus on judgment.

For incident command: A ranked prior with every input visible and every override logged. Not a black box. Not an autopilot. A number you can question. 🧭

What it does ✨

Feature Description
Splat Ingestion Streams a gaussian-splat .ply (Scaniverse/Polycam) or a .glb/.gltf mesh, sniffs the header for per-Gaussian covariance + SH colour, guards against multi-hundred-MB files, and renders it as a live point cloud.
Structural Extraction An opacity-weighted RANSAC plane fitter recovers walls, slabs and inclines with a per-Gaussian tolerance ε_i = K·√(nᵀΣ_i n) — each splat's own covariance sets how tightly it must fit.
Wall Drift, Banded Verticality as the drift ratio δ = tan θ — the number a structural engineer actually reads — sorted into severity bands (…SEVERE) with a colour overlay.
Debris Volume Column-method volume above a detected ground plane, clustered and ranked. Deliberately not a mesh — a surface hides the residual an assessor needs to see.
Triage Ranking ρ = (n · q · r · λ) / max(0.1, τ) — expected lives per crew-hour, weighted by collapse-type urgency. Sites sort descending. This is the dispatch order.
Agent Swarm 🧠 Five server-side agents, one per ranking parameter, each with its own evidence source and a deterministic verifier. They propose; the operator applies.
A/B Scan Slots Two scan slots for a visual before/after. Honest about being a toggle — no registration or change detection (yet).

Example fieldflow!!!

A crew arrives at three flagged structures and scans each.

save-splat:  Site 2 — WALL DRIFT 0.031 (SEVERE), one debris column 14.2 m³
             above ground plane, morphology reads MIXED.
             ρ ranks it above Sites 1 and 3.

*Site 2 jumps to the top of the dispatch list*
*The leaning wall lights up in the SEVERE band on the overlay*
*The engineer sees 0.031 — not "looks bad" — and decides.*

Incident command reviews the ranking before committing a crew.

They open Site 2. Every input is on screen: n (occupancy, an operator
estimate), q (P trapped alive — no agent, by design), r, τ, collapse type.
The ACCESS agent proposed r = 0.6; the log shows the operator accepted it,
and the value it replaced. Nothing moved a slider on its own.
(this is the whole point — a prior you can defend, not an order you obey)

How we built it 🛠️

┌─────────────────────────────────────────────────────────────────────────────┐
│                          SAVE-SPLAT PIPELINE                                │
└─────────────────────────────────────────────────────────────────────────────┘

  🛰️  Input (Gaussian-splat .ply  /  .glb·.gltf mesh)
        │
        ▼
  ┌──────────────┐     ┌──────────────┐     ┌──────────────────────┐
  │  Vite +      │────▶│  /api/swarm  │────▶│  Reasoner (server)   │
  │  React + TS  │     │  (one URL)   │     │  Athena·OR·OAI·Anth  │
  └──────────────┘     └──────────────┘     └──────────────────────┘
        │                                            │
        ▼                                            ▼
  ┌──────────────────────┐                  ┌──────────────────────┐
  │  core/  (pure logic) │                  │  Zod validation gate │
  │  NO three.js, NO DOM │                  │  reject, don't clamp │
  └──────────┬───────────┘                  └──────────┬───────────┘
             │                                         │
   ┌─────────┼───────────────┐              ┌──────────▼───────────┐
   ▼         ▼               ▼              │  Five agents, one    │
┌──────┐ ┌────────┐ ┌────────────────┐      │  per ρ parameter:    │
│ ply/ │ │geometry│ │  orientation   │      │  RECORDS  → n        │
│parse │ │extract │ │  up-axis sniff │      │  ACCESS   → r        │
└──┬───┘ └───┬────┘ └────────────────┘      │  VOLUME   → τ        │
   │         │                              │  MORPHOLOGY → type   │
   │         ▼                              │  CORROBORATION→ conf │
   │  ┌───────────────────┐                 │  (q has NO agent)    │
   │  │ opacity-weighted  │                 └──────────┬───────────┘
   │  │ RANSAC, seeded RNG│                            │
   │  │ ε_i = K·√(nᵀΣ_i n)│                     each proposal has
   │  │  ↓                │                     a deterministic
   │  │ walls·slabs·incl. │                     verifier
   │  │  ↓                │                            │
   │  │ drift δ = tan θ   │                            ▼
   │  │  ↓                │                  ┌──────────────────────┐
   │  │ debris columns    │                  │ operator applies →   │
   │  │ over ground plane │                  │ logged w/ old value  │
   │  └─────────┬─────────┘                  └──────────┬───────────┘
   │            │                                       │
   └────────────┴───────────────────┬───────────────────┘
                                     ▼
                          ┌────────────────────┐
                          │  ranking.ts        │
                          │  ρ = nqrλ / max(.1,τ)│
                          └─────────┬──────────┘
                                    ▼
                              🧭 Visual Output
        (THREE.Points cloud + plane/drift overlay + markers +
              ranked site panel + JSON/CSV export)

Architecture Overview

Layer Technology Purpose
Frontend Vite, React 18, TypeScript Panel UI + scene, single-page workspace
State useSyncExternalStore store Sites live in state; their meshes live in the viewer, keyed by id
Renderer Three.js Camera, hand-written orbit, markers, plane/drift overlay — splats as THREE.Points
Core Logic Pure TS (no three.js, no DOM) Parsing, RANSAC, ranking, export — unit-testable without a browser
Parsing Custom .ply + gaussian-splat reader Per-Gaussian covariance from scale_*/rot_*, SH f_dc_* colour, streaming w/ progress
Geometry Opacity-weighted RANSAC (seeded mulberry32) Planes, wall drift bands, column-method debris volume
Validation Zod (core/swarm/schema.ts) The boundary: anything from outside is rejected, not clamped
Reasoner Node-only server/swarm/ Prompts → provider call → Zod gate → verifiers
Providers Athena (Stripe-provisioned default) · OpenRouter · OpenAI · Anthropic One Reasoner interface; whichever key is present wins, Athena checked first; model resolved from the account, never hardcoded
Hosting Vercel functions + Vite dev/preview Same handlers mount in all three; browser only ever calls /api/swarm/*
Run Log Supabase Postgres pooler (optional, append-only) Every run appended to swarm_runs; a write failure warns, never blocks a run; the app never reads it back

Geometry & Core (pure TypeScript)

Opacity-weighted RANSAC that respects the splat. A gaussian splat isn't a point cloud — every splat carries a covariance Σ describing how fuzzy it is. So instead of one global inlier threshold, each point gets its own: ε_i = K·√(nᵀΣ_i n). A crisp, confident splat must fit the plane tightly; a diffuse one is allowed slack. Opacity weights the fit so faint noise can't outvote solid structure. The RNG is seeded (mulberry32) — re-running extraction on the same cloud produces byte-identical planes, and a regression test builds a floor + a wall leaning exactly 3% and asserts we recover drift ≈ 0.03 in the SEVERE band.

Wall verticality as a number engineers already use. Tilt comes out as the drift ratio δ = tan θ — rise over run — banded by severity with a live colour overlay. "That wall looks bad" becomes 0.031.

Debris volume by the column method — and deliberately not a mesh. We detect a ground plane, then integrate rubble volume in columns above it and cluster the result. We refuse to hand back a triangulated surface: a mesh hides the residual, and the residual (how much of the scan the planes failed to explain) is exactly the uncertainty an assessor must see before trusting the number.

The core/ purity rule. None of this touches three.js or the DOM. buildWorkingSet takes vertex data + a world matrix, not a THREE.Points. That single boundary is why the whole geometry pass is testable without a renderer — and why the numbers can be checked against the original single-file reference build they were ported from.

The Ranking Index

ρ = (n · q · r · λ) / max(0.1, τ) — expected lives saved per crew-hour:

  • n — persons believed inside (an operator estimate, never derived from geometry)
  • q — P(trapped alive)
  • r — P(extraction)
  • λ — survival decay by collapse morphology (pancake 0.35 ≫ lean 0.06 per hour)
  • τ — crew-hours, guarded at 0.1 so a slider at minimum can't divide by zero

Sites sort descending by ρ. Confidence is deliberately kept out of ρ — it's an evidence-quality flag for the operator, not a multiplier that would quietly launder uncertainty into the ranking.

The Agent Swarm — assisted assessment, never autonomy

Five agents, one per ranking parameter, each with its own evidence source and a deterministic verifier:

Agent Param Reads Verifier
RECORDS n rosters, shift patterns, time-of-day occupancy two independent sources; disagreement widens the band instead of averaging it away
ACCESS r plane extents, debris columns, route clearance re-run the route with the largest debris cluster removed; r must not jump a band
VOLUME τ debris volume, plane fill %, shoring burden column volume vs a convex-hull bound; a gap past tolerance flags an overhang
MORPHOLOGY type plane classes, slab tilt, lean-to angles, drift re-fit on a held-out half of the cloud; the λ band must agree
CORROBORATION conf agreement across the other four flags LOW whenever any upstream agent failed its verifier

Three invariants make this safe to put in front of a crew:

  • Proposals are rejected, not clamped. n = 500 is a reasoning failure; silently turning it into 50 would launder a bad number into the ranking as if it had been considered. The error names the field and the value so the caller can retry.
  • The swarm proposes; the operator applies. Nothing writes a slider without an explicit operator action, and every application is logged with the value it replaced.
  • q (P trapped alive) has no agent — on purpose. An exterior scan carries no evidence an occupant is alive. The gap is the honest answer, not an oversight.

Secrets & the server boundary

No secret is ever VITE_-prefixed (Vite would inline it into the client bundle). Reasoner keys live only in server/swarm/, read from process.env. The browser talks to /api/swarm/* only — the same handlers mount on the Vite dev server, the Vite preview (the on-stage fallback) and as Vercel functions. The default reasoner is Athena, provisioned by the Stripe Projects CLI so stripe projects env --pull is the whole setup step — but adding any other vendor is one file implementing the Reasoner interface plus a line in the index, and whichever key is present wins. The model is resolved from the account, never hardcoded.

🚧 Challenges we ran into

Making a splat measurable without turning it into a mesh. The reflex answer to "3D scan → geometry" is meshing, and it's the wrong one — it papers over exactly the uncertainty we need to show. Building an opacity- and covariance-weighted RANSAC that produces planes with a residual instead of a smooth lie was the core design fight.

Reproducibility of a randomized algorithm. RANSAC is random; a triage tool can't be. Seeding with mulberry32 and pinning a regression fixture (a wall leaning exactly 3%) means the same scan always yields the same planes — and a reviewer can verify the extractor against a known answer.

Guarding the input, not the output. The temptation with a bad model proposal is to clamp it into range. We do the opposite — reject it — because a laundered number is more dangerous than a visibly missing one in a life-safety ranking.

Keeping core/ renderer-free. The original build hung three.js meshes off each site record, which made the list impossible to export or diff. Splitting pure geometry/logic from the viewer — data in, data out — is what makes the numbers testable and the sites serializable.

Two audiences, one honest story. Crews want δ = 0.031 and a residual fraction. Command wants a ranked list they can defend. Both had to see the same underlying numbers with nothing hidden between them.

🏆 Accomplishments that we're proud of

✅ A gaussian-splat → structural-facts pipeline that stays honest about its residual ✅ Per-Gaussian covariance-weighted RANSAC with seeded, reproducible output + a regression fixture ✅ Wall drift as a real engineering number (δ = tan θ), banded and overlaid ✅ A five-agent swarm that proposes and verifies but can't move a slider on its own ✅ Zod validation that rejects rather than clamps — bad numbers can't sneak into the ranking ✅ Strict client/server secret boundary; the browser only ever calls /api/swarm/* ✅ Honest UI copy: a ranked prior for incident command review, never an autonomous dispatch order

📚 What we learned

  • The residual is the product. The most useful thing a triage tool can show isn't the clean answer — it's how much of the scan it couldn't explain. Don't hide it.
  • Reject, don't clamp. In a life-safety ranking, a silently corrected input is worse than a loud failure.
  • Determinism is a feature. A seeded RNG turned "trust me" into "re-run it yourself and check."
  • The operator is the algorithm's boss, not its output. Framing every agent as assisted assessment — propose, verify, log, wait for a human — is what makes this deployable near a real crew.
  • A clean boundary buys testability. Keeping core/ free of three.js and the DOM is why the numbers can be checked without a browser.

What's next for save-splat 🚀

Feature Status Description
Multi-scan change detection 🔜 Planned Real registration between A/B slots — is that wall moving between scans?
Append-only field run-log ✅ Shipped Postgres swarm_runs table records every proposal and verifier result; write failures never block a run
Confidence-aware overlays 💡 Concept Surface the residual fraction per plane directly in the 3D view
Handoff export pack 💡 Concept One-click structural brief (planes, drift, volume, ranking, logs) for incident command

The Dream: a crew walks up to a collapsed structure, scans it with the phone in their pocket, and within seconds has defensible structural numbers and a ranked place in the queue — with a human making every call and a log that can be read back afterward. The tool reads the rubble. The people decide. 🧭

Framing note. save-splat maps against the Sendai Framework for Disaster Risk Reduction 2015–2030 and the relevant SDG targets (see PROJECT-BRIEF.md). It is a decision aid: output is a ranked prior for incident command review, not an autonomous dispatch order — and the UI copy is kept honest about that on purpose.

Built With

Share this project:

Updates

Submission history