Context Awareness

How the agent knows what the user is looking at — and how the agent can control what the user sees.

Overview

Without context awareness, the agent is blind. It asks "which email?" when the user is staring at one. It cannot act on the current selection, cannot provide relevant suggestions, and cannot modify what the user sees. With context awareness, the user can click a row, highlight a paragraph, select a slide element, or press Cmd+I, then say "summarize this" and the agent already knows what "this" means.

To understand what to put in which surface (AGENTS.md vs. skills vs. application_state), see Writing Agent Instructions — The four surfaces the agent sees.

Six patterns solve this:

1. Navigation state — the UI writes a navigation key to application-state on every route change
2. Current URL — the framework writes __url__ so query params are visible and editable by the agent
3. Selection state — the UI writes a selection key when the user focuses, selects, or multi-selects something meaningful
4. view-screen — an action that reads application state, fetches contextual data, and returns a snapshot of what the user sees
5. Prompt handoff — UI controls call sendToAgentChat() when a click should become an agent turn
6. navigate — a one-shot command from the agent that tells the UI where to go

Context layers

Use different context channels for different jobs:

The short version: URL query params are the source of truth for shareable filters, navigation stores semantic IDs and view names, view-screen turns those state layers into useful data, and sendToAgentChat() turns UI intent into a chat message when the user clicks a command.

Navigation state

The UI writes a navigation key to application-state on every route change. This tells the agent what view the user is on, what item is open, and which semantic UI state matters.

{
  "view": "inbox",
  "threadId": "thread-123",
  "focusedEmailId": "msg-456",
  "label": "important"
}

What to include in navigation state:

• view — the current page/section, such as "inbox", "form-builder", or "dashboard"
• Item IDs — the selected/open item, such as threadId or formId
• Semantic aliases — active tab, label name, or other stable app concepts that help the agent reason
• Light focus state — focused row, active tab, current panel

Keep navigation small and semantic. It should identify the current screen, not duplicate whole records or mirror every query param. Fetch records in view-screen so the agent always gets fresh data.

The agent reads this before acting:

import { readAppState } from "@agent-native/core/application-state";

const navigation = await readAppState("navigation");
// { view: "inbox", threadId: "thread-123", label: "important" }

Current URL and filters

AgentPanel automatically syncs the current React Router URL into the __url__ application-state key. The built-in agent includes it in every turn as a <current-url> block:

<current-url>
pathname: /adhoc/revenue
search: ?f_region=west&q=renewal
searchParams:
  f_region: west
  q: renewal
</current-url>

This is the canonical layer for shareable filter state. If the user can copy a URL and come back to the same filtered list, the filter belongs in the query string. The agent can change those filters with the built-in set-search-params tool:

set-search-params({ "params": { "f_region": "east", "q": null } })

Use navigation only for semantic aliases that help view-screen fetch or summarize the right data. A dashboard might keep navigation.dashboardId while __url__.searchParams owns f_region, f_dateStart, and q.

When view-screen returns a richer snapshot, it can copy important URL filters into a friendly activeFilters object:

const url = (await readAppState("__url__")) as {
  searchParams?: Record<string, string>;
} | null;

if (url?.searchParams) {
  screen.activeFilters = Object.fromEntries(
    Object.entries(url.searchParams).filter(
      ([key, value]) => key.startsWith("f_") && value,
    ),
  );
}

Selection state

Selection is semantic UI state. It is how "the chart I clicked", "these three rows", "this slide title", or "the current email draft range" becomes model-visible context.

Use the selection app-state key for durable selection that should survive a moment of navigation, empty-chat suggestions, or a later view-screen call:

{
  "kind": "slide.elements",
  "deckId": "deck-123",
  "slideId": "slide-4",
  "items": [
    {
      "id": "hero-title",
      "selector": "[data-block-id='hero-title']",
      "label": "Hero title",
      "text": "Q3 launch plan"
    }
  ],
  "capturedAt": 1780332977027
}

Write it from the UI when the user selects, focuses, or multi-selects meaningful objects:

import { setClientAppState } from "@agent-native/core/client/hooks";
async function syncSelection(selection: unknown | null) {
  await setClientAppState("selection", selection, { keepalive: true });
}

Good selection state includes:

• Stable IDs the agent can use in actions, such as threadId, slideId, or assetId
• A short human label so prompts and suggestions are readable
• Enough text or metadata to disambiguate the object
• Optional UI locators such as selectors or coordinates when the agent needs to refer back to a visual element
• capturedAt when stale selection would be harmful

Avoid storing secrets, full documents, large binary payloads, or whole API responses in selection. Store IDs plus short excerpts, then let view-screen fetch the current source of truth.

One-shot selected text

AgentPanel already handles the common text-selection flow. When the user presses Cmd+I (or Ctrl+I) with text selected on the page, it:

1. Reads window.getSelection()
2. Writes { text, capturedAt } to pending-selection-context
3. Focuses the agent chat

The production agent injects that key into the next turn as immediate selection context and ignores it once it is stale. This is the path that makes "select text, press Cmd+I, ask 'make this punchier'" work without the user copying the selection into the prompt.

Custom editors can write the same key when their selection is not represented by native browser selection:

import { setClientAppState } from "@agent-native/core/client/hooks";
await setClientAppState(
  "pending-selection-context",
  {
    text: selectedMarkdown,
    capturedAt: Date.now(),
  },
  { keepalive: true },
);

Use pending-selection-context for one-shot "act on this exact highlighted text" flows. Use selection for durable object selection that view-screen and dynamic suggestions should keep seeing.

The view-screen action

Every template should have a view-screen action. It reads navigation and selection state, fetches the relevant data, and returns a snapshot of what the user sees. This is the agent's eyes.

The agent should call pnpm action view-screen before acting on the current UI. This is a hard convention across all templates. When adding new features, update view-screen to return data for the new view and any new selection shape.

Prompt handoff with sendToAgentChat()

Sometimes context should not just sit in app state. A user clicks a button, drops a comment pin, selects an item and chooses "Ask agent", or presses an AI command in a toolbar. That click is an instruction. In browser UI, hand it to the agent with sendToAgentChat().

import { sendToAgentChat } from "@agent-native/core/client/agent-chat";
function askAgentAboutSelection(selection: {
  documentId: string;
  blockId: string;
  label: string;
  text: string;
}) {
  sendToAgentChat({
    message: `Improve the selected block: ${selection.label}`,
    context: [
      `Document id: ${selection.documentId}`,
      `Block id: ${selection.blockId}`,
      "Current selected text:",
      selection.text,
    ].join("\n"),
    submit: false,
    openSidebar: true,
  });
}

Use the fields deliberately:

sendToAgentChat() is the supported browser wrapper for the submitted-chat path sometimes seen internally as agentNative.submitChat. App UI should call the wrapper instead of posting agentNative.submitChat directly because the wrapper handles local sidebars, Builder/Frame routing, MCP App host routing, tab IDs, and code-request routing.

Use agentChat.submit() or agentChat.prefill() for Node/script contexts where there is no browser sidebar. Server actions generally should not call browser-only sendToAgentChat(); if an action needs the open UI to ask the agent something, write a small request into application_state and let a UI bridge send it from the browser.

Clicked items in the prompt

For the "click items in the UI and they become part of the prompt" experience, combine selection state with prompt handoff:

1. On click or multi-select, write semantic selection state so view-screen, dynamic suggestions, and future turns can see it.
2. If the click is also a command, call sendToAgentChat() with a concise visible message and richer hidden context.
3. In view-screen, hydrate the selected IDs into current records so the agent can verify the object before mutating it.
4. Clear selection when the object is no longer selected, deleted, or no longer relevant.

That gives the user the magic "this is what I meant" behavior without stuffing every prompt with bulky visible context.

The navigate action

navigate is the mirror image of navigation. Where navigation is the UI telling the agent where the user is, navigate is the agent telling the UI where to go. The agent writes a one-shot navigate command to application-state; the UI reads it, performs the navigation, then deletes the entry.

// Agent side -- write a navigate command
import { writeAppState } from "@agent-native/core/application-state";

await writeAppState("navigate", { view: "inbox", threadId: "thread-123" });

On the UI side you never poll or delete this key by hand. Both directions — writing navigation on every route change and consuming the agent's navigate command — are handled by a single hook, useNavigationState, covered in the next section.

The navigation key belongs to the UI; the agent must never write to it directly. The agent writes navigate, the UI performs the move, and that move is what updates navigation.

When the destination has a real URL, include a same-origin path on the navigate command and have the UI prefer that path before falling back to semantic fields. Keep app navigation single-channel: do not write both navigate and __set_url__ for the same move. __set_url__ is for the framework URL tools (set-url-path, set-search-params) and URL-only filter changes. For commands that can arrive while chat is streaming, commit the route with navigate(path, { replace: true, flushSync: true }) instead of wrapping it in a view transition so the address bar and visible page stay together.

The useNavigationState hook

useNavigationState is your app's hook, not a framework import. Every template ships one at app/hooks/use-navigation-state.ts and calls it once from the app shell (root.tsx). It is the single place that wires navigation in both directions:

• Outbound (UI → agent): writes the navigation key whenever the route changes, so the agent always knows the current view.
• Inbound (agent → UI): polls the navigate command, runs the navigation, and deletes the command.

It stays short because it is a thin wrapper around the real framework primitive, useAgentRouteState (exported from @agent-native/core/client). You supply two app-specific functions and the framework does the rest:

// app/hooks/use-navigation-state.ts -- this file lives in YOUR app
import { useAgentRouteState } from "@agent-native/core/client/navigation";
import { TAB_ID } from "@/lib/tab-id";

interface NavigationState {
  view: "inbox" | "thread";
  threadId?: string;
  path?: string;
}

export function useNavigationState() {
  useAgentRouteState<NavigationState>({
    browserTabId: TAB_ID,
    requestSource: TAB_ID,

    // UI → agent: derive semantic state from the current URL.
    getNavigationState: ({ pathname }) => {
      const match = pathname.match(/^\/thread\/([^/]+)/);
      return match ? { view: "thread", threadId: match[1] } : { view: "inbox" };
    },

    // agent → UI: turn a `navigate` command into a route to push.
    getCommandPath: (command) =>
      command.path ??
      (command.view === "thread" && command.threadId
        ? `/thread/${command.threadId}`
        : "/"),
    navigateOptions: { replace: true, flushSync: true },
  });
}

The TAB_ID imported from app/lib/tab-id.ts is the framework's getBrowserTabId() value. Page-local WebMCP calls send the same id in X-Agent-Native-Browser-Tab, so the server resolves per-tab navigation/selection state to the exact tab the agent or user is driving — there is no second tab scheme to keep in sync.

Paths returned as strings are always local to this app, even when they start with another workspace app's mount. To navigate to a sibling app, return { type: "workspace-app", path: "/seo-application/settings" } with the full mounted path.

useAgentRouteState assumes React Router. When navigation does not live in the URL — a wizard step, a canvas selection, a non-router shell — drop down to the lower-level useSemanticNavigationState instead: you hand it a ready-made state value plus navigationKeys/commandKeys and an onCommand callback, and it stays completely agnostic about React Router.

Jitter prevention

When the agent writes to application-state, the sync system might cause the UI to refetch data it just wrote. This creates jitter. The solution is source tagging:

Use setClientAppState, writeClientAppState, readClientAppState, and deleteClientAppState from @agent-native/core/client for browser-side application-state access. Pass { requestSource: TAB_ID } on UI writes when pairing with useDbSync({ ignoreSource: TAB_ID }); pass { keepalive: true } for short-lived writes such as selection cleanup during unload.

// app/root.tsx
import { TAB_ID } from "@/lib/tab-id";

useDbSync({
  queryClient,
  ignoreSource: TAB_ID, // ignore events from this tab's own writes
});

How it works:

• Agent writes are tagged with requestSource: "agent" (the action helpers do this automatically)
• UI writes include the tab's unique ID via X-Request-Source header
• The server stores the source on each event
• When processing sync events, the UI filters out events matching its own ignoreSource value — so it doesn't refetch data it just wrote
• Events from agents, other tabs, and actions still come through normally

What's next

• Using Your Agent — how end users experience the context the agent sees
• Agent Mentions — @-mention files and agents into a turn instead of relying on ambient context
• Drop-in Agent — mount the panel that reads and writes this state
• Real-Time Collaboration — keep app state in sync when the agent and users edit together
• Application State — the full application-state and client helper reference
• Toolkit piece: Context & Knowledge — the Context X-Ray presentation built on this state