PATIENT DIRECTORY
AUTH
MAIN 3D DIGITAL TWIN VIEW
MEASUREMENTS AND DIFFERENTIAL DIAGNOSIS ON CARDIOGRAMS
DIFFERENT SLICING PANES
DIFFERENT SECTIONS OF THE HEART
DIFFERENT TYPES OF SCANS TO VIEW THE DIGITAL TWIN

CAIRO: Turning Echocardiograms into Clearer Clinical Decisions

A clinician-in-the-loop cardiac intelligence workspace

Inspiration

An echocardiogram can be acquired in minutes, but turning it into a reliable clinical decision requires far more work. Sonographers and cardiologists must select usable views, reconcile noisy cine loops, trace anatomy, repeat measurements, and communicate findings. That burden is especially acute when specialist capacity is limited and patients are far from specialist services.

We were inspired by a simple question: what if every echo study arrived with a coherent, inspectable starting point rather than a stack of raw clips? CAIRO is designed to make cardiac imaging easier to review, not to replace the clinician who is accountable for the final interpretation.

What it does

CAIRO turns echocardiography cine loops into an interactive review workspace. It runs temporal AI inference over video clips to estimate measurements including ejection fraction (EF), left-ventricular end-diastolic volume (LVEDV), and left-ventricular end-systolic volume (LVESV). Results are stored per video and aggregated at the study level so a reviewer can inspect both the outputs and the underlying study.

The clinician-facing interface combines the source echo video with a synchronised 3D cardiac digital twin. Users can animate the cardiac cycle, inspect anatomy from multiple viewpoints, toggle valve and tracing overlays, explore an AHA 17-segment heatmap, and review volume and Wiggers-style telemetry curves. A PanEcho AI panel exposes the model outputs alongside the visual evidence rather than hiding them behind a single score.

How we built it

We built CAIRO as an end-to-end prototype with a deliberately modular architecture:

Inference: a Python pipeline loads PanEcho through PyTorch, samples multiple clips from an echo video, and produces regression and classification outputs. The initial workflow focuses on EF, LVEDV, and LVESV, while retaining the ability to run the broader PanEcho task set.
Backend: FastAPI exposes study, inference, and browser-friendly video-streaming endpoints. SQLAlchemy and Alembic provide an auditable PostgreSQL data model for patients, studies, videos, predictions, labels, and processing state.
Data flow: local files and Google Cloud Storage URIs are supported; cached video transcoding makes clips playable in the browser. The project is structured for GCP-hosted inference and Cloud SQL, with the larger production pipeline intended to use Pub/Sub workers.
Clinical workspace: Astro and React power the application shell, while React Three Fiber and Three.js render the interactive heart model. The frontend synchronises the video, cardiac phase, measurements, and visual overlays in one review surface.

Challenges we ran into

Echocardiography is a difficult modality: image quality varies by acoustic window, patient anatomy, motion, acquisition technique, and view. Converting a model output into something clinicians can trust therefore required more than simply displaying a number. We had to keep source video accessible, present model-derived visualisations as aids to review, and preserve per-video outputs so results remain traceable.

There was also a practical systems challenge. ML inference, clinical data, video streaming, and real-time 3D rendering have very different runtime needs. We addressed this by separating the inference service, database-backed API, and browser experience, while keeping the interfaces between them simple.

Finally, clinical AI must be built with restraint. The current prototype is a decision-support workspace, not an autonomous diagnostic system. Its outputs require clinician review, and any real-world deployment would require rigorous validation, privacy controls, workflow testing, and appropriate regulatory assessment.

Accomplishments that we're proud of

Built a working path from echo video ingestion to stored AI predictions and a clinician-facing study view.
Connected temporal cine-loop inference to a rich, interactive visual review experience instead of a static results dashboard.
Made the cardiac cycle explorable through synchronised video playback, a 3D twin, anatomical overlays, and physiological telemetry.
Designed the backend around idempotent prediction writes and explicit video status transitions, making repeated runs safer and easier to audit.
Kept the product direction vendor-neutral, so the workflow can fit mixed imaging environments rather than depend on a single ultrasound platform.

What we learned

The strongest insight was that clinical value comes from workflow, not model accuracy alone. A useful measurement must be easy to inspect, compare with the source study, and act on within a clinician's existing review process. We also learned that the 3D experience is most compelling when it is synchronised to real clinical context: the moving echo clip, phase of the cycle, volumes, and segment-level information all reinforce one another.

On the technical side, we learned to treat inference as a product pipeline. Reliable video handling, explicit processing states, repeatable database writes, and clear API boundaries are as important as the model call itself. Building these foundations early makes later validation and deployment more realistic.

What's next for CAIRO

Validate the workflow on a larger, diverse, multi-site set of echocardiography studies and measure accuracy, calibration, and failure modes by subgroup and image quality.
Add clinician controls for accepting, editing, rejecting, and annotating AI outputs, with a complete audit trail.
Expand from core ventricular measurements to quality checks, view-awareness, and carefully bounded, explainable differential-support features.
Integrate with DICOM/PACS and reporting workflows, then evaluate whether CAIRO improves reporting turnaround, rework, and capacity in real clinics.
Establish the evidence, quality-management, privacy, and regulatory pathway required for a safe clinical deployment.

CAIRO is a prototype for clinician-reviewed decision support. It is not intended for autonomous diagnosis or clinical use without validation and appropriate regulatory clearance.