Transitguard

A lineage-grounded agent that turns a transit agency's GTFS feeds into a governed DataHub graph, then tells you whether a feed change is safe to ship and which federal report goes down with it.

Comment

Inspiration

The FTA now requires fixed-route agencies that take federal money to publish a valid, current GTFS feed, tied to National Transit Database (NTD) reporting and grant eligibility. Free validators answer one narrow question: is this feed well-formed against the spec. None of them map that posture to the funding mandate, and none can tell you what breaks downstream when a feed changes. A context-blind LLM asked "is this change safe" simply guesses, because it cannot see what is connected to what. DataHub already models lineage and compliance as a graph, which is the context that agent lacks. TransitGuard gives the agent that graph, so it can answer the questions that decide an agency's funding eligibility.

What it does

Given a proposed feed change (say, renaming the stop_id column), TransitGuard answers two questions a plain validator cannot. First, is the feed compliant with the federal funding mandate right now? It runs the canonical GTFS validator, maps posture to NTD compliance, and records the result as DataHub assertions and tags. Second, if the feed changes or breaks, what goes down with it? It traverses real column-level lineage to name the downstream datasets, the departures dashboard, and the federal NTD export that the change affects.

On a confirmed "not safe to ship" finding, and only after a human confirms, it writes a governance record back into DataHub so a breaking change cannot pass silently.

How it works

Public GTFS feeds become a governed DataHub context graph, an agent answers off that graph, a context-blind baseline proves the graph is load-bearing, and a human-confirmed write-back closes the loop. See docs/architecture.md for the full diagram.

  • The GTFS-to-DataHub connector registers each GTFS table as a DataHub dataset on a custom gtfs platform, with a real schema and column-level foreign-key lineage (stops.stop_id to stop_times.stop_id, and so on). Only spec-defined keys are emitted, so nothing is manufactured, and because no such connector existed before, it doubles as an open-source contribution.
  • Validation maps feed posture to NTD compliance: MobilityData's canonical gtfs-validator runs, and two assertions per dataset (validation and freshness) render as green and red badges, alongside compliance tags and properties.
  • A dbt-duckdb project (staging models, a departures mart, an NTD stop export) emits its own column-level lineage back onto the graph, so a stop_id rename provably propagates from the raw feed to the federal NTD export.
  • The impact agent is Claude Code grounded by the official DataHub MCP server. It traverses the lineage, reads compliance from the graph, and returns a plain-language impact statement plus a structured verdict (safe_to_ship, ntd_certification, affected_datasets, breaks_ntd_export).
  • The context-moat benchmark runs the same agent twice, once with the DataHub graph and once against an isolated raw-feed-only baseline, over a fixed question set, and reports the per-bucket accuracy delta.
  • The governed write-back runs on a confirmed finding, after a human [y/N] confirm: it tags the changed table and its affected downstream datasets pending-breaking-change, records the finding as properties, and fails a safe to ship assertion. The tag write is a PATCH merge, so existing compliance tags survive, and --resolve reverses the whole record.

The result (the headline)

We measured the value of the graph directly. The same agent answered the same fixed question set twice: once grounded by the DataHub graph, once with only a raw-feed summary in an isolated working directory, so the only difference is the DataHub context.

bucket n grounded blind delta
spec_validity 2 1.00 0.00 +1.00
freshness 2 1.00 1.00 +0.00
downstream_impact 3 1.00 0.00 +1.00
compliance_consequence 2 1.00 0.00 +1.00
overall 9 1.00 0.22 +0.78

The blind agent returned unknown on every question in the three buckets the graph makes answerable (spec validity, downstream impact, compliance consequence), because it has no validator, no lineage, and no compliance metadata to work from. Freshness is a deliberate +0.00: staleness is computable from the feed's own service dates, which the blind agent gets, so the graph earns nothing there. Reporting that honestly (instead of hiding it) is what keeps the benchmark from being a strawman.

On the headline scenario, renaming stop_id in the failing clallam feed:

  • Grounded: affected_datasets lists stg_stop_times, mart_departures, and ntd_stop_export; breaks_ntd_export: true; ntd_certification: AT RISK; safe_to_ship: false.
  • Blind: affected_datasets: []; downstream impact, NTD reach, and compliance all unknown.

What makes it original

Detection of a downstream blast radius is a built-in DataHub capability, and free GTFS validators are common. The original part is the combination none of them close: mapping GTFS posture to the FTA funding mandate, extending real column-level lineage from a raw transit feed to a federal report, proving the graph is load-bearing with a quantified context-blind control, and closing the loop with a reversible, human-confirmed governance write-back. It is also a new vertical for DataHub (transit and govtech) and contributes a connector for a domain the catalog did not cover.

How we built it

Python 3.11 (pinned via uv), the DataHub Python SDK for the connector and write-back, the official DataHub MCP server for the agent, dbt-duckdb for the derived chain, and MobilityData's gtfs-validator for the deterministic validation. The agent is Claude Code running headless against the MCP server, so no separate API key is needed. Everything runs against a local DataHub (datahub docker quickstart).

Reused code (disclosure)

Per the hackathon's pre-existing-work policy: src/features/validation/validate.py adapts a canonical-validator wrapper from a pre-existing script (subprocess invocation, report parsing, service-date extraction). The validator itself is MobilityData's Apache-2.0 gtfs-validator, provided as a jar (not committed). Everything else is new work for this submission: the DataHub connector, the derived-chain lineage emission, the compliance and assertion mapping, the impact agent, the benchmark, and the governed write-back.

Challenges we ran into

  • Custom-platform lineage: verifying that the DataHub MCP server traverses lineage on a custom gtfs platform (not a built-in warehouse connector) and that the UI renders it.
  • Assertion run events are timeseries aspects, so they read differently from normal aspects; the compliance badges depend on getting that right.
  • Keeping the benchmark honest: the blind baseline is deliberately isolated (no MCP, no repo access) and the freshness bucket is reported as a tie rather than buried, so the delta reflects only the DataHub context.

What we learned

For these questions, the graph carries the answer. Without lineage and compliance metadata, a capable agent correctly refuses to guess and returns unknown; with them, it returns a definite, compliance-grounded verdict. That gap is measurable and large.

What's next for TransitGuard

  • Contribute the GTFS connector upstream as a DataHub ingestion source so any agency can catalog its feeds.
  • Broaden the demo to a full region's feeds and wire the write-back into a CI merge-gate.

Built With

  • datahub
  • datahub-mcp-server
  • dbt
  • duckdb
  • python
Share this project:

Updates