Firehose Documentation
Real-time web monitoring. Create rules, stream every matching page as it's crawled, and watch specific URLs for changes.
Firehose monitors the web in real time. You write rules, and every crawled page that matches is delivered to you over a Server-Sent Events stream — typically within moments of the page being crawled.
Need to track a handful of specific pages instead of the whole web? URL Watch crawls them on a schedule and surfaces the diffs.
Smart Digest →New
Hand a topic to an agent and read one short report that keeps itself current. Free during beta, and it uses none of your matches or credit.
Quickstart →
Go from zero to a live stream of matching pages in five minutes.
Core concepts →
Organizations, taps, rules, and the crawl → match → stream pipeline.
Stream →
The core product: rules, the SSE stream, and the full query language.
URL Watch →
Track specific URLs for changes on a schedule and read the diffs.
Ways to use Firehose
Most integrations consume the Stream. Reach for Smart Digest when you want a short summary of a topic rather than every match, the dashboard feed to explore matches before you write any code, and URL Watch when you already have the exact URLs you want checked on a schedule.
| Surface | What it is | Reach for it when | Metered as |
|---|---|---|---|
| Smart Digest New | An agent tracks a topic and keeps one short report current | You want the few items that matter, summarized, rather than every match | Free during beta — no credit, no match quota |
| Stream (SSE) | Real-time push of every match as it's crawled | You're integrating code and want matches the moment they happen | Prepaid credit, per match |
| Dashboard feed | Browse and triage matches in the UI, no code | You want to watch or explore matches without building anything | Monthly reviewable-match quota |
| URL Watch | Scheduled re-crawl of specific URLs, with stored diffs | You already know the URLs and want changes on your cadence | Monthly check quota |
The Stream and the feed are two views of the same thing: rules on a tap match pages across the whole web, and you either consume them as an SSE stream or watch them arrive in the dashboard. Smart Digest sits on the same tap machinery and reads the matches for you, writing a report instead of handing over the pages. URL Watch is different — it re-crawls a fixed list of URLs you give it.
A tap only surfaces a page when the crawler reaches it, on the crawler's own schedule — so to reliably
catch changes to a specific page, reach for URL Watch, not a tap with a url: rule.
Built for agents
Every page in these docs has a Copy for LLM button and a raw-Markdown twin — just append .md
to any URL (for example, /get-started/quickstart.md). There's also
an llms.txt index and a full llms-full.txt corpus you can feed to
a model in one shot.
