Conversation
kamath
added a commit
that referenced
this pull request
Jan 15, 2025
varadfromeast
added a commit
to varadfromeast/stagehand
that referenced
this pull request
May 3, 2026
…py path Updates the architecture to match what's actually true after the 2026-05-03 live smoke: - Layer 6 (Browser Session): adds 'Stagehand is configured in noLlm mode' subsection with the exact constructor flags, and 'Stagehand methods cartographer actually calls' subsection listing the narrow surface used at runtime. Notes Stagehand provides ~2,400 LOC of CDP/iframe/ locator infrastructure cartographer doesn't have to maintain. - Stagehand cache section: explicit 'Cartographer does not use Stagehand's cache at all' framing. Notes the actCache/agentCache callbacks throw if invoked under noLlm but are never consulted because act(Action) routes straight to takeDeterministicAction. - Replaced 'Current Happy Path Example' with the verified open_inbox smoke output from 2026-05-03, including the architectural claim it validates end-to-end. - 'What Comes Next': bumped postcondition tightening to browserbase#1 (silent-send trap on send-message). Login persistence is no longer pending.
shrey150
added a commit
that referenced
this pull request
Jun 5, 2026
## Summary Migrates [browserbase/cli#141](browserbase/cli#141) into the monorepo now that the oclif `browse` CLI lives in `packages/cli`. Cloud commands (`cloud search`, `cloud fetch`, etc.) hard-fail when `BROWSERBASE_API_KEY` is missing — data showed this is the #1 cause of first-run failure (~74% of installs fail their first command; ~94% of those are missing-key failures on cloud commands). Local browser automation (`open`/`get`/`click`/…) needs no key and is what recurring users overwhelmingly use. This PR "leads with local" by making the missing-key error friendly and actionable: it now states that cloud commands need a key **and** that local automation needs none, pointing to `browse open <url> --local`. **No driver behavior change.** Driver commands already default to managed-local when no key is set, so `browse open <url>` already launches a local browser and succeeds. This is a message-only change. Changes: - `packages/cli/src/lib/cloud/api.ts` — enriched `resolveApiKey()` missing-key message. Still starts with `Missing Browserbase API key` (telemetry test substring + `missing_api_key` result code unchanged); clean `CommandFailure` exit preserved. - `packages/cli/src/lib/driver/remote.ts` — clearer remote-mode error distinguishing when a key is required vs. when `--local` works without one. (In the monorepo this string lives in `remote.ts`; in the standalone repo it was `session-manager.ts`.) - `.changeset/lead-with-local.md` — `"browse": patch`, consumed by the decoupled `release-cli` workflow. Linear: [STG-2184](https://linear.app/browserbase/issue/STG-2184) ## E2E Test Matrix Built the workspace locally (`pnpm install` + turbo build of `@browserbasehq/stagehand` + `pnpm --filter browse build`) and ran the built `bin/run.js` with `BROWSERBASE_API_KEY` unset (`env -u BROWSERBASE_API_KEY`). | Command / flow | Observed output | Confidence / sufficiency | | --- | --- | --- | | `browse cloud search "test query"` (no key) | `Missing Browserbase API key. Cloud commands (search, fetch, sessions, functions, ...) need one.` / `Set BROWSERBASE_API_KEY or pass --api-key.` / `Get a key at https://browserbase.com/settings.` / blank / `No key? Local browser automation needs none. Try: browse open <url> --local` — exit 1 | Proves the enriched, actionable message ships on the highest-volume failure path. Clean exit-1, no stack trace. | | `browse cloud fetch https://example.com` (no key) | Same enriched message | Confirms all cloud commands share the improved guidance via `resolveApiKey`. | | `browse open https://example.com --local` (no key) | `{"mode":"managed-local", ...}` — exit 0 | Proves local automation works with zero config; no code change needed here. | | `browse open https://example.com --remote` (no key) | `Error: Missing BROWSERBASE_API_KEY for remote mode. Pass --local to run a managed local browser (no key needed), or set BROWSERBASE_API_KEY for cloud sessions.` | Proves the enriched remote-mode driver error ships. | | `browse stop` | `{"stopped":true,...}` | Local daemon cleans up. | | `pnpm lint` (prettier + eslint + tsc --noEmit) | All pass | Formatting, lint, and typecheck clean. | | `pnpm test` (vitest, 15 files) | `Tests 209 passed (209)` | Full suite green, including `cli-telemetry` asserting stderr contains `Missing Browserbase API key` and `result_code === "missing_api_key"`. | 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Improves onboarding for the `browse` CLI by making missing-key errors lead with local mode: cloud commands now say they require a key and suggest `browse open <url> --local`, and the remote-mode error explains when a key is required. Message-only change that preserves telemetry ("Missing Browserbase API key", result_code `missing_api_key`) and leaves driver behavior unchanged; addresses Linear STG-2184. <sup>Written for commit ff579aa. Summary will update on new commits.</sup> <a href="https://cubic.dev/pr/browserbase/stagehand/pull/2192?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
shrey150
added a commit
that referenced
this pull request
Jun 6, 2026
Addresses Cubic review: stray </content>/</invoke> tags leaked into the README footer during authoring.
shrey150
added a commit
that referenced
this pull request
Jun 27, 2026
Closes the gap from @shubh24's review #1: you can now name a context you already have (an id a teammate shared, or one created on another device / outside the CLI) instead of only at create time. Local-only, mirrors the alias-save path; --force repoints an existing name. Validates the name (same rule as create --name) and rejects an empty id. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
shrey150
added a commit
that referenced
this pull request
Jun 29, 2026
Closes the gap from @shubh24's review #1: you can now name a context you already have (an id a teammate shared, or one created on another device / outside the CLI) instead of only at create time. Local-only, mirrors the alias-save path; --force repoints an existing name. Validates the name (same rule as create --name) and rejects an empty id. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
shriyatheunicorn
added a commit
that referenced
this pull request
Jul 27, 2026
Part of STG-2671 (deterministic suite).
Completes the deterministic a/e/o suite on the v4 API: **act 40/40,
extract 25/25, observe 12/12** under `tasks/bench-v4/`, all on
`stagehand.act/extract/observe` with async page accessors and the new
config shape.
## Porting rules applied uniformly
- Behavior preserved byte-for-byte where the API allows — same URLs,
instructions, success criteria, timings
- Schemaless v3 extracts → explicit single-key schemas (`z.object({
extraction: z.string() })`, V4_API_LOGS #2); snake_case schemas ported
verbatim so wire-casing stays comparable (#14)
- `framework/observeReplay.ts`: shared observe→act replay standing in
for the missing `act(observeResult)` (#1) — delete when the SDK ships
`ReplayActionSchema`
- `heal_*` tasks fail loudly (`V4 GAP`) rather than fake self-healing v4
doesn't have
- `combination/` removed — both tasks depended on the deprecated
agent-era surface
## Verification
- Full-suite sweeps on LOCAL and Browserbase vs v4-spike `93af925`:
53/74 pass; every failure classified (frames/OOPIF, observe ranking #15,
targeting gaps) in V4_API_LOGS
- v3 baseline on identical tasks: 65/74 — regression set is concentrated
and nameable
- Schema audit: 59 tasks byte-equivalent to v3, 13 documented wrapper
conversions, 0 drift
- Typecheck, eslint, prettier clean
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Ports the full deterministic bench suite to the v4 SDK for STG-2671,
completing act 40/40, extract 25/25, and observe 12/12 under
`tasks/bench-v4`. Preserves v3 behavior where possible and documents v4
gaps; adds an observe→act replay helper and minor stability updates.
- **New Features**
- Complete v4 bench on `stagehand.act/extract/observe`, with async page
accessors and the new config shape.
- Added `framework/observeReplay.ts` to replay observe→act until v4
ships native replay.
- **Refactors**
- Schemaless extracts now use explicit schemas (e.g., `z.object({
extraction: z.string() })`); snake_case schemas kept to compare wire
casing.
- Handled v4 gaps: `heal_*` tasks fail with clear V4 GAP messages;
iframe/node-id checks re-expressed in-page; removed `combination/` tasks
tied to the deprecated agent surface; renamed `act/nonsense_action` to
assert failure handling.
- Verification: local and Browserbase sweeps pass 53/74; failures
classified; typecheck/lint clean.
<sup>Written for commit a2db76f.
Summary will update on new commits.</sup>
<a
href="https://cubic.dev/pr/browserbase/stagehand/pull/2387?utm_source=github"
target="_blank" rel="noopener noreferrer"
data-no-image-dialog="true"><picture><source
media="(prefers-color-scheme: dark)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source
media="(prefers-color-scheme: light)"
srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img
alt="Review in cubic"
src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a>
<!-- End of auto-generated description by cubic. -->
---------
Co-authored-by: cubic-dev-ai[bot] <191113872+cubic-dev-ai[bot]@users.noreply.github.com>
This was referenced Jul 28, 2026
shriyatheunicorn
added a commit
that referenced
this pull request
Jul 28, 2026
…preflight validation, dedup
Task error contract: every bench-v4 catch block now returns the canonical
TaskResult `error` key with a sanitized string (message only — raw Errors
serialize as {} and `reason:`/`message:` keys are invisible to the runner's
progress reporting). extract_jfk_links additionally logs structured failure
data before returning.
Eval validity:
- extract_public_notices gated on compareStrings().meetsThreshold in BOTH
the v3 and v4 copies (the bare object is always truthy, so both suites
reported success regardless of extracted content); fixed on both sides to
keep cross-SDK scoring comparable
- google_flights converted to an explicit V4 GAP stub: v4 locator.click has
no force option and no act(observeResult) replay, so the force-click
benchmark cannot run on v4
- extract_recipe error messages say "expected" where they compared against
the expected constants
- stale "no { page } option" comments in tab_handling/multi_tab corrected
(the option exists; active-page tracking is exercised deliberately)
v4 init parity and diagnostics: initV4 sets selfHeal: true (matching
initV3) and bridges SDK logs into the EvalLogger via the nested
logging.onLog config, feature-detected against the loaded SDK schema.
Preflight validation: explicit --sdk now requires --harness stagehand;
--sdk v4 with --api or agent modes fails once at command validation instead
of per-row; explicit --sdk with core-tier tasks is rejected before runs are
misfiled into SDK-comparison projects.
Dedup: shared framework/textScoring.ts (normalizeString/compareStrings,
byte-identical to the 7 inlined copies) and framework/observeSelectors.ts
(in-page selector resolution, 6 identical copies unified);
observe_simple_google_search uses the shared observeReplay;
observeReplay.click now forwards the observed mouse button like v3.
Tests: focused coverage for observeReplay dispatch (14), the v4 bench
harness startup/guards (7), planner sdk propagation (5), and initV4 param
building (13). V4_API_LOGS #1/#2/#7/#8 examples updated to the stagehand.*
surface. Suite: 55 files / 442 tests green; typecheck clean.
Scoring-semantics findings intentionally not "fixed" where the v4 task is a
byte-for-byte port of the same weakness in v3 (staff_members, area_codes,
jstor ordering, next_chunk modal fallback, observe_taxes innerText gate) —
strengthening only the v4 side would break cross-SDK comparability.
shriyatheunicorn
added a commit
that referenced
this pull request
Jul 28, 2026
…preflight validation, dedup
Task error contract: every bench-v4 catch block now returns the canonical
TaskResult `error` key with a sanitized string (message only — raw Errors
serialize as {} and `reason:`/`message:` keys are invisible to the runner's
progress reporting). extract_jfk_links additionally logs structured failure
data before returning.
Eval validity:
- extract_public_notices gated on compareStrings().meetsThreshold in BOTH
the v3 and v4 copies (the bare object is always truthy, so both suites
reported success regardless of extracted content); fixed on both sides to
keep cross-SDK scoring comparable
- google_flights converted to an explicit V4 GAP stub: v4 locator.click has
no force option and no act(observeResult) replay, so the force-click
benchmark cannot run on v4
- extract_recipe error messages say "expected" where they compared against
the expected constants
- stale "no { page } option" comments in tab_handling/multi_tab corrected
(the option exists; active-page tracking is exercised deliberately)
v4 init parity and diagnostics: initV4 sets selfHeal: true (matching
initV3) and bridges SDK logs into the EvalLogger via the nested
logging.onLog config, feature-detected against the loaded SDK schema.
Preflight validation: explicit --sdk now requires --harness stagehand;
--sdk v4 with --api or agent modes fails once at command validation instead
of per-row; explicit --sdk with core-tier tasks is rejected before runs are
misfiled into SDK-comparison projects.
Dedup: shared framework/textScoring.ts (normalizeString/compareStrings,
byte-identical to the 7 inlined copies) and framework/observeSelectors.ts
(in-page selector resolution, 6 identical copies unified);
observe_simple_google_search uses the shared observeReplay;
observeReplay.click now forwards the observed mouse button like v3.
V4_API_LOGS #1/#2/#7/#8 examples updated to the stagehand.* surface.
Suite green; typecheck clean.
Scoring-semantics findings intentionally not "fixed" where the v4 task is a
byte-for-byte port of the same weakness in v3 (staff_members, area_codes,
jstor ordering, next_chunk modal fallback, observe_taxes innerText gate) —
strengthening only the v4 side would break cross-SDK comparability.
felipeofdev-ai
pushed a commit
to felipeofdev-ai/stagehand
that referenced
this pull request
Aug 4, 2026
felipeofdev-ai
pushed a commit
to felipeofdev-ai/stagehand
that referenced
this pull request
Aug 4, 2026
* add browserbase#1 trending badge * extra line lol
felipeofdev-ai
pushed a commit
to felipeofdev-ai/stagehand
that referenced
this pull request
Aug 4, 2026
…e#2192) ## Summary Migrates [browserbase/cli#141](browserbase/cli#141) into the monorepo now that the oclif `browse` CLI lives in `packages/cli`. Cloud commands (`cloud search`, `cloud fetch`, etc.) hard-fail when `BROWSERBASE_API_KEY` is missing — data showed this is the browserbase#1 cause of first-run failure (~74% of installs fail their first command; ~94% of those are missing-key failures on cloud commands). Local browser automation (`open`/`get`/`click`/…) needs no key and is what recurring users overwhelmingly use. This PR "leads with local" by making the missing-key error friendly and actionable: it now states that cloud commands need a key **and** that local automation needs none, pointing to `browse open <url> --local`. **No driver behavior change.** Driver commands already default to managed-local when no key is set, so `browse open <url>` already launches a local browser and succeeds. This is a message-only change. Changes: - `packages/cli/src/lib/cloud/api.ts` — enriched `resolveApiKey()` missing-key message. Still starts with `Missing Browserbase API key` (telemetry test substring + `missing_api_key` result code unchanged); clean `CommandFailure` exit preserved. - `packages/cli/src/lib/driver/remote.ts` — clearer remote-mode error distinguishing when a key is required vs. when `--local` works without one. (In the monorepo this string lives in `remote.ts`; in the standalone repo it was `session-manager.ts`.) - `.changeset/lead-with-local.md` — `"browse": patch`, consumed by the decoupled `release-cli` workflow. Linear: [STG-2184](https://linear.app/browserbase/issue/STG-2184) ## E2E Test Matrix Built the workspace locally (`pnpm install` + turbo build of `@browserbasehq/stagehand` + `pnpm --filter browse build`) and ran the built `bin/run.js` with `BROWSERBASE_API_KEY` unset (`env -u BROWSERBASE_API_KEY`). | Command / flow | Observed output | Confidence / sufficiency | | --- | --- | --- | | `browse cloud search "test query"` (no key) | `Missing Browserbase API key. Cloud commands (search, fetch, sessions, functions, ...) need one.` / `Set BROWSERBASE_API_KEY or pass --api-key.` / `Get a key at https://browserbase.com/settings.` / blank / `No key? Local browser automation needs none. Try: browse open <url> --local` — exit 1 | Proves the enriched, actionable message ships on the highest-volume failure path. Clean exit-1, no stack trace. | | `browse cloud fetch https://example.com` (no key) | Same enriched message | Confirms all cloud commands share the improved guidance via `resolveApiKey`. | | `browse open https://example.com --local` (no key) | `{"mode":"managed-local", ...}` — exit 0 | Proves local automation works with zero config; no code change needed here. | | `browse open https://example.com --remote` (no key) | `Error: Missing BROWSERBASE_API_KEY for remote mode. Pass --local to run a managed local browser (no key needed), or set BROWSERBASE_API_KEY for cloud sessions.` | Proves the enriched remote-mode driver error ships. | | `browse stop` | `{"stopped":true,...}` | Local daemon cleans up. | | `pnpm lint` (prettier + eslint + tsc --noEmit) | All pass | Formatting, lint, and typecheck clean. | | `pnpm test` (vitest, 15 files) | `Tests 209 passed (209)` | Full suite green, including `cli-telemetry` asserting stderr contains `Missing Browserbase API key` and `result_code === "missing_api_key"`. | 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated description by cubic. --> --- ## Summary by cubic Improves onboarding for the `browse` CLI by making missing-key errors lead with local mode: cloud commands now say they require a key and suggest `browse open <url> --local`, and the remote-mode error explains when a key is required. Message-only change that preserves telemetry ("Missing Browserbase API key", result_code `missing_api_key`) and leaves driver behavior unchanged; addresses Linear STG-2184. <sup>Written for commit ff579aa. Summary will update on new commits.</sup> <a href="https://cubic.dev/pr/browserbase/stagehand/pull/2192?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
![https://cubic.dev/buttons/review-in-cubic-dark.svg"><source](/post/java-https-cubic.dev/buttons/review-in-cubic-dark.svg"><source){kind=link}
![https://cubic.dev/buttons/review-in-cubic-light.svg"><img](/post/java-https-cubic.dev/buttons/review-in-cubic-light.svg"><img){kind=link}
![https://cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a](/post/java-https-cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a){kind=link}
![https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source](/post/java-https-www.cubic.dev/buttons/review-in-cubic-dark.svg"><source){kind=link}
![https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img](/post/java-https-www.cubic.dev/buttons/review-in-cubic-light.svg"><img){kind=link}
![https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a](/post/java-https-www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a){kind=link}
No description provided.