Waterloo Whiteout

Inspiration

Arctic search requires platforms with different strengths. Fixed-wing aircraft cover large areas, quadcopters provide precise local observation, and fixed towers watch continuously. WHITEOUT challenged us to combine those capabilities into shared intelligence for finding and tracking a moving vessel.

Our approach assigns each asset a specialized role while combining its observations into a common mission picture.

What it does

Waterloo Whiteout is a safety-gated search-and-track controller for ArcticSim. It includes:

A trained YOLOv8 boat detector.
Fixed-wing and quadcopter search missions.
MAVLink telemetry and typed MAVProxy vehicle controls.
Camera-based target geolocation.
Constant-velocity target tracking.
Search, confirmation, tracking, reacquisition, return, and abort recommendations.
A dashboard for camera feeds, fleet state, routes, detections, and target tracks.
Recording and replay for repeatable testing.
Explicit safety controls for flight and track submission.

The fixed-wing aircraft searches the broad target corridor. The quadcopter performs detailed local coverage and follow-up observation, while two towers provide persistent views.

How we built it

We built the controller in Python using typed interfaces for cameras, detections, telemetry, missions, tracks, and vehicle commands.

MJPEG camera streams feed a custom YOLOv8 detector that returns boat bounding boxes, confidence values, and timestamps. An automatic labelling tool supported creation of the training dataset.

MAVLink provides vehicle position and attitude. A geolocation component projects detections onto the water surface, and an alpha-beta tracker estimates the target’s position and velocity over time. The coordinator converts that state into recommendations without automatically moving vehicles.

We analyzed the generated Fort Ross terrain and the vessel’s approximately 6.1 km route. The fixed-wing mission performs broad coverage at approximately 200 m altitude over a 13.9 km mission. The quadcopter flies a tighter local pattern at approximately 90 m, covering about 2.5 km including transit. We checked complete mission segments against terrain, not only individual waypoints.

Missions use the QGroundControl WPL format. MAVProxy validates and uploads them, but arming and starting the mission remain separate operator actions.

The FastAPI and MapLibre dashboard combines four camera feeds with vehicle state, routes, detections, target history, and predicted movement. It also supports recorded-session replay.

The four team workstreams covered controller and search architecture, MAVLink and flight control, dashboard development, and YOLO detection.

Challenges

Unknown target position

The vessel can begin anywhere along its route. We used the fixed-wing aircraft for broad coverage and reserved the quadcopter for local search and target confirmation.

Terrain-safe routes

Waypoints can be over water while the path between them crosses land. We validated complete route segments against the terrain.

IMU drift handling

The mobile drones limit heading drift through EKF fusion of IMU, GPS, and magnetometer measurements. The fixed towers use pan-and-tilt encoder feedback as their stable orientation reference, avoiding reliance on integrating IMU measurements over time. This prevents accumulated gyro error from degrading camera pointing accuracy.

Multi-person integration

Detection, control, tracking, and visualization were developed concurrently during a one-day hackathon. Typed models and clear interfaces allowed the components to be merged without tightly coupling each workstream.

Safe vehicle control

Accidental commands could disrupt the simulation. We separated read-only telemetry from control, validated commands before transmission, and required explicit confirmation for flight and submission.

Coordinate conversion

We had to connect image pixels, camera orientation, latitude and longitude, and the simulator’s EPSG:3413 Arctic projection.

Accomplishments

Trained and integrated a YOLOv8 boat detector.
Built complementary search missions for two aircraft types.
Implemented managed, typed MAVProxy control.
Added geolocation, tracking, and coordination components.
Built a unified live and replay dashboard.
Added terrain-aware mission validation.
Kept recommendations separate from automatic execution.
Discovered 79 offline tests, with 73 passing and 6 intentionally skipped because they require optional dependencies or a live simulator.

What we learned

We learned that multi-agent coordination depends on clear information sharing more than identical vehicle behavior. Each asset can remain specialized if detections, telemetry, tracks, and assignments use consistent data structures.

We also learned that realistic search planning must account for target uncertainty, terrain, camera footprint, turning constraints, endurance, and recovery procedures.