AirTools | Devpost

Tape Measure
Area and Angles
Personalized Catalog
Place Part
Remove Item
Valid Fit Check
Invalid Fit Check
Compare Sellers
Hold to Pay
Switch Site
Drone Scanned Site of GT Challenge Course
End to End pipeline

Inspiration

Falls are the leading cause of death for roofers and gutter installers.
Current drone/satellite inspection tools only give photos or static models.
Buying new parts requires searching for listings, spec sheets, and guessing if the dimensions match the measurement.

Expert Testimonials on AirTools:

"Our software measurements can't do complex projects, especially with higher and lower gutters. We try to get a visual from a drone. [AirTools] would be valuable for those jobs." - Gutters Plus, Atlanta

"I think this could be a super cool and innovative use of a drone in our industry" - Kyle Harmeyer (Pro Edge Roofing)

"Satellite and drone only measurements take a lot more time and a lot more effort than we think it'll be." - Urban Seamless Gutters, Atlanta

"Drone only measurements are the past, we are way beyond that when it comes to measuring in the industry." - Ernie (Trident Public Adjusting)

"Using drones for dimensioning is only half the value. The bigger win is getting those measurements into measuring software." - LRG CLAIMS, Tyler, Texas

What it does

AirTools turns drone footage into a true-size 3D copy of a building that you walk around in a Meta Quest 3S. You can measure length, level, area, and scale.
If an item needs replacement, you can consult Grok to find compatible parts and view their models in VR at true size using Grok Imagine. Existing parts can also be hidden to see the empty space.
Grok can also run a measurement survey for you, lay out placement plans, point out missing coverage, answer manual and permit questions, and coach you through the install step by step.
Hazardous edges and ladder placement areas are highlighted.
Grok lets users browse personalized results or sort by price, delivery speed, and reviews. Payment via Visa includes a 1-second security hold. The headset can't access card info, and voice commands alone can't authorize payment.

How we built it

A Python pipeline turns DJI footage, stills, and SRT telemetry into a real-world-scaled, textured 3D scene. It extracts frames and telemetry, runs pycolmap SfM with VGGT as a fallback, uses caption altitude to establish scale within ±5%, and feeds the reconstruction through OpenMVS PatchMatch, MoGe-2 TSDF filling, ReconstructMesh, auto-cropping, and TextureMesh. The resulting scene includes camera poses, dense geometry, planes, edges, semantic structure, and measurable cavities for removable parts.
LIMAP & PxwPlanar provide geometry-aware structure and snapping, while plane fitting, gravity alignment, face votes, and QA keep measurements and surfaces spatially consistent.
SAM 2.1/3 identifies parts and their boundaries from video, while Grok names components, finds replacement parts, validates published dimensions, and generates searchable part geometry that can be checked against the building.
Unity 6 + Meta XR turns the spatial model into an interactive Quest 3S experience for measurement, leveling, ladder planning, safety warnings, annotations, and full-scale part visualization.
Every measurement, hazard, photo, part, annotation, and reconstruction element is tied back to the same spatial model, so inspection, planning, and installation happen in one environment.
We used Cursor with parallel feature lanes and automated test gates, while Grokbot coordinated tasks, handoffs, missing measurements, conflicts, and installation planning across the pipeline.

Challenges we ran into

Gaussian splats dropped to 36 FPS at 400k splats on Quest 3S, so we switched to textured meshes and maintained 72 FPS.
Drone GPS/altitude wasn't reliable indoors, so we built known-dimension scaling. A 24″ dishwasher corrected our kitchen model by 1.49×.
Photogrammetry blurred edges by ~16 mm. Snapping to fitted geometry reduced corner error from 6.9 mm to 2.7–3.6 mm.
Occluded surfaces and mesh holes required geometric fallbacks that distinguish measured vs. estimated geometry.
Models could hallucinate dimensions, locations, or fail under rate limits, so every AI output gets validation, fallbacks, or explicit labeling.
We designed checkout so neither voice nor accidental input can authorize a purchase, with zero payment credentials on the headset.

Accomplishments that we're proud of

Our end-to-end loop works: drone scan → measure → find a part → view it at true size → verify fit → compare sellers → purchase → receipt saved.
We're able to get 72 FPS on Quest 3S with a 200k-triangle, 4K-textured scan.
A spoken request was able to replace a fridge with a more modern version.
A live scan updates preserve measurements, annotations, and placed parts.

What we learned

AI can search and summarize but every physical dimension needs to be validated against published specs.
Short scripted sessions caught issues the Simulator couldn't.
Contractors confirmed the value is getting accurate measurements into the workflow they already use.

What's next for AirTools

Plug into contractor workflows.
Automate outdoor scaling.
Cleaner scans, richer roof/gutter geometry, and manufacturer CAD where available.
Go real-time.
Track change over time.