Self-hosted, resumable-stream broker for LLM apps. Kill wifi mid-generation, refresh the page, the stream continues without a hiccup. Three lines to integrate.

Built on the open Durable Streams protocol. Wire-compatible, plus an LLM layer: cancel-vs-disconnect, producer leases, and a React hook.

1. Broker

cd broker && uv sync && uv run zephyr serve

2. Producer (Python)

async with produce(stream_id, broker_url="http://127.0.0.1:4437") as pub:
    async for chunk in llm_stream:
        await pub.send(chunk)

3. Client (React)

const { text, status, cancel } = useZephyrStream(streamId, {
  brokerUrl: "http://127.0.0.1:4437",
});

cd site && npm install && npm run dev
# → http://127.0.0.1:3000

Product story, comparison, integrate, download, signup waitlist, and an in-page kill/reconnect proof.

Three terminals:

# Terminal 1 — broker (SQLite)
cd broker && uv sync && uv run zephyr serve

# Terminal 2 — demo API (echo LLM)
cd demo/backend && uv sync && uv run uvicorn zephyr_demo.main:app --reload --port 8000

# Terminal 3 — chat UI
cd demo/frontend && npm install && npm run dev

Open http://127.0.0.1:5173

1. Send a message — tokens stream.
2. Click Kill connection — rendering stops; generation continues on the broker.
3. Click Reconnect or refresh the page — missed tokens appear, live tokens continue.
4. Open second tab — both tabs share the stream. Stop generation cancels in both.

cd broker
docker compose up --build
# proxy on :4437, two broker replicas, shared Redis

Or:

uv run zephyr serve --backend redis --redis-url redis://127.0.0.1:6379/0

LLM / echo producer  →  Zephyr broker (durable log)  →  React tabs (resume by offset)
                              ↑
                     explicit cancel only
                     (disconnect never cancels)

Offsets live in sessionStorage. Refresh is just another resume. See docs/ for product, architecture, protocol, and roadmap.

cd broker && uv run pytest
cd producer-py && uv run pytest
cd client-ts && npm test
# optional Redis: start Redis, then
# ZEPHYR_TEST_REDIS_URL=redis://127.0.0.1:6379/15 uv run pytest tests/test_redis_backend.py

Apache-2.0