CortexTrace

Explore brain response. Trace every insight.

Comment

CortexTrace

Explore brain response. Trace every insight.

Launch the live Cohort Explorer · Agent health · Private DataHub operations

CortexTrace is a governed brain-response intelligence product. It turns predicted cortical-response matrices for short-form videos into an interactive 3D cohort experience, then uses DataHub to make the result discoverable, traceable, and explainable.

The public visualization answers what patterns are present. The DataHub-backed context agent answers where the result came from, who owns it, which pipeline built it, and which quality contracts apply.

[!IMPORTANT] CortexTrace is an exploratory research and visualization project—not a medical device, diagnostic system, or causal model of human behavior. Predicted cortical maps and correlations support hypothesis generation, not clinical conclusions.

Project at a glance

Dimension Current deployment
Unique reels in the combined cohort 292
Full cortical matrices retained 227 / 292
Response values per cortical map 20,484 vertices
High-resolution rendered surface 327,684 mesh vertices
Named network and timing features 96
DataHub assets 8
DataHub pipeline jobs 3
DataHub model entities 2
DataHub quality assertions 5
Recorded lineage edges 8
Published metadata objects 20
Automated tests 23 passing

Inspiration

Short-form video platforms are exceptionally good at measuring clicks, likes, comments, and watch time. Those metrics describe behavior, but they do not make the underlying analysis transparent—and a compelling brain visualization by itself is still difficult to trust.

We wanted to connect three experiences that are usually separated:

  1. Scientific exploration — see cohort-level cortical-response patterns on a real, interactive cortical surface.
  2. Responsible evidence — distinguish validated feature-level results from exploratory vertex maps and keep missing evidence visibly missing.
  3. Data governance — trace every public output through datasets, jobs, models, schemas, owners, and quality contracts.

That led to CortexTrace: not simply a brain viewer, but a governed data product whose visual claims can be interrogated.

What it does

Explore the cohort

The hosted Cohort Explorer combines two source exports, deduplicates them by reel ID, joins available public metadata, and renders the resulting brain-response maps in the browser.

Experience What the user can do Why it matters
Cohort Compare all-reel and genre-level averages and difference maps Moves from individual examples to population-level patterns
Reels Filter 292 reels and paint mean, standard deviation, maximum, or peak maps Keeps each visual result connected to an individual record
Overlaps Explore reel similarity, PCA structure, and region-by-genre overlap Reveals clusters and shared response structure
Signature Inspect cohort consistency and prototypical or unusual reels Describes the common pattern without pretending there is a non-viral control
Group Maps Compare baked averages without loading every raw matrix Makes a large cohort shareable as one public artifact
DataHub Context Ask lineage, schema, ownership, and assertion questions Turns the visualization into an explainable governed product

The WebGL cortex supports rotation, zoom, folded/inflated surfaces, cohort filters, and warm/cool difference maps:

  • Warm and cool regions indicate divergence between selected groups.
  • Neutral cortex indicates overlap.
  • Exploratory vertex maps are not presented as stable localization results.
  • Missing engagement or model evidence is shown as unavailable and is never inferred.

Ask the governed context agent

The public drawer can answer questions such as:

Trace the Cohort Explorer back to its source datasets.
Which quality checks govern the cortical response matrices?
What schema and owner are recorded for the network feature table?

Each answer is generated from DataHub metadata using read-only tools. The API returns the tools used and the exact source URNs alongside the answer.

{
  "answer": "...",
  "tools_used": ["search", "get_lineage"],
  "source_urns": ["urn:li:dataset:(...)"],
  "grounded_in": "DataHub"
}

Impact

CortexTrace demonstrates that scientific AI products can be visual, shareable, and governed at the same time.

Stakeholder Common problem CortexTrace impact
Researchers High-dimensional surface arrays are difficult to inspect or communicate Converts cortical matrices into an interactive, anatomically aligned experience
Data scientists Model reports become disconnected from source features and study constraints Records features, evaluation jobs, models, ownership, and lineage in DataHub
Reviewers and judges A polished visualization can hide missing evidence or overclaim results Surfaces limitations, missing values, exploratory status, and quality contracts directly
Data stewards Documentation becomes stale as pipelines change Declares stable catalog entities and lineage in code and republishes them programmatically
Non-technical users Raw NumPy arrays and pipeline logs are inaccessible Provides a public browser experience plus plain-language governed questions
AI application teams General chatbots can invent provenance or use unsafe tools Restricts the agent to read-only DataHub context and returns evidence URNs

The project’s central impact is trust at the point of interpretation. Users do not need to leave the product, find a separate catalog, or ask an engineer which file produced a chart. The answer is available next to the visualization and is grounded in the metadata graph.

How we built it

Brain-response pipeline

Stage Implementation Output
Video inference External video-to-cortical-response model T × 20,484 predicted response matrix
Cohort assembly analyze_cohort.py Deduplicated records, similarity, PCA, group and difference maps
Anatomical mapping FreeSurfer fsaverage5, Schaefer-200/Yeo-17, Destrieux atlas Named parcels, networks, and surface-aligned values
Feature extraction extract_features.py ROI features and 96 network/timing features
Evaluation model_features.py Nested cross-validation and label-permutation report
Browser build build_cohort.py + cohort_template.html Self-contained cohort.html with embedded data and geometry
Visualization Three.js/WebGL Folded and inflated interactive cortical surfaces

The original response data lives on the fsaverage5 cortical surface: 10,242 vertices per hemisphere, or 20,484 values per map. Registration-sphere nearest neighbor mapping projects those values onto a higher-resolution display mesh without inventing new measurements.

GCP system architecture

Layer GCP service Responsibility
Public product Cloud Storage Hosts the shareable static Cohort Explorer
Context API Cloud Run Scale-to-zero FastAPI service for catalog and governed questions
Reasoning Vertex AI Runs Gemini through Google ADK
Metadata platform Compute Engine Runs DataHub v1.6.0 for the hackathon deployment
Private connectivity Custom VPC / Direct VPC egress Connects Cloud Run to the private DataHub endpoint
Credentials Secret Manager Stores the DataHub PAT and administrator credential
Administration Identity-Aware Proxy Provides maintenance access without public SSH or UI ports

The browser only knows the public Cloud Run API. It never receives the private GMS address or DataHub token.

How DataHub is used

DataHub is the central trust layer, not a decorative integration and not the storage location for raw brain matrices. It catalogs the assets and relationships that explain how the product was built.

DataHub capability CortexTrace implementation Product value
Metadata catalog Eight stable dataset/data-product entities declared in brain_datahub/catalog.py Makes every major input and output discoverable
Schemas Records reel identifiers, nullable engagement fields, cortical matrix structure, feature counts, and report fields Lets users understand the shape and meaning of an asset before using it
Ownership and tags Assigns technical ownership and tags such as brain-response, python-pipeline, and public-data-product Establishes accountability and searchable context
Dataset lineage Connects source matrices to features, features to evaluation, and source exports to the public explorer Answers “where did this come from?” without manual archaeology
Data jobs and flows Publishes the extract, evaluate, and build jobs inside a Python data flow Represents transformation logic rather than only endpoints
ML model metadata Registers the external cortical predictor and the network-feature evaluation harness Separates the producing model from the evaluation model
Custom assertions Publishes five stable quality-contract definitions Makes expectations explicit without inventing execution results
Agent Context Builds Google ADK tools from the DataHub client with include_mutations=False Grounds Gemini in governed metadata while preventing catalog writes
Evidence URNs Captures URNs from tool calls, responses, and final text Makes generated answers inspectable and auditable
Metadata-service authentication Uses a PAT through the frontend-proxied /api/gms endpoint Protects programmatic access while minimizing exposed services

Governed metadata inventory

Type Count Examples
Assets 8 Reel metadata, two feature exports, cortical matrices, ROI features, network features, model report, Cohort Explorer
Jobs 3 Extract features, evaluate models, build explorer
Models 2 Video-to-cortical-response predictor, network-feature evaluator
Assertions 5 Unique IDs, matrix shape, matrix coverage, feature schema, frozen study contract
Lineage edges 8 Input/output relationships across the three jobs
Total published objects 20 15 catalog/flow/job/model entities plus 5 assertions

Recorded lineage

Per-reel cortical matrices
        └── Extract interpretable features
              ├── Schaefer-200 ROI features
              └── 96 network + timing features
                         └── Evaluate predictive models
                                  └── Nested-CV model report

Feature export A ─┐
Feature export B ─┼── Build Cohort Explorer ──> Public GCS data product
Reel metadata ────┤
Model report ─────┘

Quality contracts

Assertion Governed expectation
unique-reel-ids The explorer contains one row per reel ID
matrix-shape Every retained cortical matrix has exactly 20,484 vertices
matrix-coverage At least 75% of cohort reels retain a full matrix
feature-schema The validated network table exposes exactly 96 named features
study-contract Inferential results reference the frozen study specification

Definitions are published without synthetic pass/fail history. Assertion results should only be emitted by the pipeline that actually executed each check.

Read-only agent service

The hosted context API is a small FastAPI service built with DataHub Agent Context, Google ADK, and Vertex AI.

Endpoint Behavior
GET /health Reports whether DataHub and Vertex AI are configured; exposes no credentials
GET /api/catalog Returns a safe public snapshot of assets, lineage, and quality contracts
POST /api/ask Runs a read-only governed question and returns evidence URNs

The agent can use DataHub search, entities, schemas, lineage, lineage paths, dataset queries, and assertion context. Mutation tools are never registered. Requests are length-limited, time-limited, sanitized on failure, and protected by a CORS policy restricted to the hosted frontend origin.

Security boundary

Control Deployed behavior
Public frontend Static files only; contains no GCP or DataHub credential
Cloud Run identity Dedicated service account with only required Vertex AI and secret access
DataHub GMS Authenticated and reachable through the private VPC
VM ingress No public access to SSH, frontend, GMS, MySQL, Kafka, or OpenSearch
Admin access IAP tunnel only
Secrets Stored in Secret Manager; not baked into images or HTML
Agent permissions Read-only DataHub tool registration
Runtime resilience Systemd recovery starts databases before GMS/frontend after VM reboot

[!NOTE] The DataHub VM is an intentionally cost-conscious, single-VM hackathon deployment. A production rollout should use DataHub’s supported production topology, managed backing services, backups, monitoring, and organization-wide identity controls.

Challenges we ran into

Challenge Why it was difficult What we did
Mapping response values to a real cortex Arbitrary 3D brain models have no correspondence to fsaverage5 vertices Used FreeSurfer surfaces and registration-sphere nearest-neighbor mapping
Rendering a large cohort in one shareable page The generated explorer contains geometry and many baked maps Serialized compact arrays, used float16 where appropriate, and hosted the artifact on GCS
Combining heterogeneous exports Source roots overlap and have incomplete matrices or metadata Applied deterministic reel-ID deduplication and surfaced coverage/skipped counts
Avoiding misleading modeling claims Every reel is curated as viral and engagement labels are sparse Kept the combined analysis descriptive and separated nested-CV feature evaluation from exploratory maps
Making an LLM trustworthy A generic assistant can invent provenance or modify systems Grounded it in DataHub, disabled mutation tools, and returned source URNs
Self-hosting DataHub securely on a budget Quickstart includes several stateful services and permissive development defaults Added swap, authenticated GMS, hardened credentials, private VPC rules, IAP, and reboot recovery
Connecting serverless compute to private GMS Cloud Run needed DataHub access without exposing the VM Used Direct VPC egress and allowed GMS traffic only from the private subnet
Keeping secrets out of the browser A public static page cannot safely carry service credentials Kept all secrets in Secret Manager and routed governed requests through Cloud Run

Accomplishments that we're proud of

  • Built and hosted a working 292-reel interactive cortical explorer with a visible high-resolution WebGL brain.
  • Made DataHub a first-class product surface: 8 assets, 3 jobs, 2 models, 5 assertions, and 8 lineage edges are available to the agent.
  • Published 20 real metadata objects to the self-hosted DataHub instance.
  • Delivered a live read-only question that traced the Cohort Explorer through its build job back to seven source and derived datasets.
  • Preserved scientific honesty: missing engagement stays missing, assertion results are not fabricated, and exploratory localization is labeled clearly.
  • Deployed an authenticated private GMS path without publicly exposing DataHub, its databases, Kafka, or OpenSearch.
  • Verified the hosted browser experience with a 1010 × 828 WebGL canvas, an online governed context panel, and zero JavaScript errors or failed requests.
  • Maintained an automated suite with 23 passing tests across the builder, frontend integration, catalog, agent API, and VM infrastructure.

What we learned

  1. A visualization becomes a data product when its provenance is part of the experience. DataHub transformed the demo from “look at this brain” into “inspect this result and trace its evidence.”
  2. Metadata-grounded AI is more useful than a larger unrestricted prompt. Search, lineage, schema, and assertions gave the agent a concrete world model; removing mutation tools made the public integration safer.
  3. Interpretability often improves when dimensionality decreases. The 96 named network/time features are easier to evaluate and explain than tens of thousands of anonymous vertex summaries.
  4. Missing data is itself important metadata. Nullable engagement fields, matrix coverage, and skipped records should remain visible rather than being silently imputed for a polished demo.
  5. Quality definitions and quality results are different things. A contract can be governed today, but only the executing pipeline should claim that it passed.
  6. Deployment architecture is part of trust. Private networking, service identities, secret handling, authentication, and reboot behavior matter as much as the model when a product becomes publicly shareable.

Scientific limitations

Limitation Consequence
No clean non-viral comparison cohort CortexTrace does not claim a viral-vs-non-viral classifier
Sparse public engagement metadata Likes/comments are shown only when observed and are never inferred
Predicted rather than measured cortical responses Maps illustrate model outputs, not an individual’s clinical brain activity
Exploratory vertex-wise analysis Surface patches are hypotheses, not stable anatomical localization
Single-VM DataHub deployment Appropriate for a hackathon demonstration, not production scale or availability

Run locally

Prerequisites

  • Python 3.10+
  • A modern WebGL browser
  • numpy, scipy, and nilearn for the visualization pipeline
  • Local source exports under train_data/ and/or train_data2/

Large source arrays, generated feature tables, reel media, credentials, and generated HTML are intentionally excluded from Git.

Build the Cohort Explorer

pip install numpy scipy nilearn

python build_cohort.py \
  --root train_data \
  --root train_data2 \
  --agent-api https://brain-data-context-635367932686.asia-south1.run.app

python view_brain.py --page cohort.html

Useful build options:

Option Purpose
--reso fsaverage6 Build a smaller but still detailed cortex
--stats mean,peak Bake fewer per-reel maps into the HTML
--meta-root instagram_reels Join public reel metadata
--agent-api URL Inject the hosted read-only context API
--out FILE Choose the generated HTML path

Validate or publish the DataHub catalog

pip install -r requirements-agent.txt

# Validate all entities without contacting DataHub.
python -m brain_datahub.publish --dry-run

# Publish to an authenticated DataHub instance.
export DATAHUB_GMS_URL="https://your-datahub.example/api/gms"
export DATAHUB_GMS_TOKEN="..."
python -m brain_datahub.publish

Never put a DataHub token in browser code, a committed environment file, or a container image.

Run the context API

export DATAHUB_GMS_URL="https://your-datahub.example/api/gms"
export DATAHUB_GMS_TOKEN="..."
export GOOGLE_CLOUD_PROJECT="your-project"
export GOOGLE_CLOUD_LOCATION="us-central1"
export BRAIN_ALLOWED_ORIGINS="http://127.0.0.1:8777"

uvicorn agent_service.app:app --host 127.0.0.1 --port 8080

See agent_service/README.md for all runtime settings.

Run the tests

python -m pytest \
  tests/test_build_cohort.py \
  tests/test_cohort_agent_ui.py \
  tests/test_datahub_catalog.py \
  tests/test_agent_service.py \
  tests/test_datahub_vm_infra.py \
  -q

Repository map

Path Responsibility
analyze_cohort.py Deduplication, metadata joins, PCA, similarity, overlaps, and group maps
build_cohort.py Builds the self-contained Cohort Explorer artifact
cohort_template.html Three.js UI and DataHub context drawer
brain_common.py Shared cortex geometry and anatomical helpers
atlas_schaefer.py Schaefer-200/Yeo-17 mapping onto fsaverage5
extract_features.py ROI and network/timing feature extraction
model_features.py Nested-CV and permutation evaluation harness
brain_datahub/catalog.py Canonical governed asset, job, model, and quality specification
brain_datahub/publish.py DataHub SDK publisher and custom assertions
agent_service/ Public read-only DataHub + Vertex AI context API
infra/datahub-vm/ Private DataHub VM bootstrap, hardening, and recovery
docs/visuals/ Pitch, architecture, brain-model, and DataHub visuals
tests/ Builder, UI, metadata, agent, and infrastructure tests

What's next for CortexTrace

Priority Next step Expected impact
1. Executed quality results Run the five DataHub assertions in the analysis pipeline and publish real result history Turns static contracts into continuous trust signals
2. A valid contrast cohort Add licensed low- and non-viral examples with creator-aware splits Enables an honest viral-vs-non-viral research question
3. Better outcome labels Collect normalized engagement, exposure, and creator context Reduces the current label bottleneck and confounding
4. Measured-response validation Evaluate against appropriately licensed measured fMRI benchmarks Tests whether predicted response patterns generalize
5. Richer DataHub governance Add domains, glossary terms, structured properties, policies, and automated ownership workflows Makes the catalog useful to a larger research organization
6. Production DataHub topology Move stateful services to a supported resilient deployment with backups and monitoring Improves availability, recovery, and scale
7. Saved investigations Let users preserve a governed question, selected cohort, and evidence URNs as a report Makes analyses reproducible and shareable
8. Longitudinal monitoring Track dataset, feature, model, and assertion changes across releases Shows how the product and its evidence evolve over time

The long-term goal is for every visual insight in CortexTrace to carry its own lineage, quality state, methodological limits, and reproducible evidence trail.

Built With

Share this project:

Updates