Enable Gemini-3.5-flash cua

[miguelg719]

why

New model dropped

what changed

Added support for the Gemini 3.5 Flash Computer Use updated toolset in GoogleCUAClient.ts, with all new tool formats correctly mapped.

test plan

---

Summary by cubic

Adds support for the google/gemini-3.5-flash computer-use agent. Normalizes Gemini 3.x tool names/args to 2.5 handlers, preserves click semantics, validates coordinates (rejects missing/NaN/Infinity), always returns a fresh screenshot, and surfaces reasoning/cached tokens.

• New Features

  • Enable google/gemini-3.5-flash in agent/LLM provider maps and public types; update tests.
  • Map 3.x functions to 2.5 handlers and accept new arg shapes: coordinate-less type, keys array or single key, magnitude_in_pixels for scroll, drag start/end pairs; recognize screenshot/take_screenshot; coordinate-less scroll falls back to PageUp/PageDown; alias wait to wait_5_seconds.
  • Always return a screenshot function response even when no executable actions are produced.

• Bug Fixes

  • Track reasoning_tokens and cached_input_tokens in Google CUA usage (per step and aggregated).
  • Preserve 3.x click-family semantics (double_click, triple_click, right_click, middle_click, move) and drop calls with missing or non‑finite coordinates; add explicit click_at guard and a shared finite-number check; add unit tests for conversion/guards.
  • Guard required args and log custom‑tool collisions: reject navigate without url and type/type_text_at without text (empty allowed); log when a custom tool name conflicts with a predefined function (predefined wins).

^{Written for commit eb1e3a7. Summary will update on new commits.}

[changeset-bot]

🦋 Changeset detected

Latest commit: eb1e3a7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 3 packages

Name	Type
@browserbasehq/stagehand	Patch
@browserbasehq/stagehand-evals	Patch
@browserbasehq/stagehand-server-v3	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[cubic-dev-ai]

4 issues found across 6 files

Confidence score: 2/5

• In packages/core/lib/v3/agent/GoogleCUAClient.ts, double_click/triple_click are collapsed to click_at without preserving click count, so intended multi-click interactions execute as single clicks and can break Gemini-3.x tasks that depend on double/triple click semantics — pass through and honor the click count before merging.
• In packages/core/lib/v3/agent/GoogleCUAClient.ts, right/middle click and mouse down/up actions are still unimplemented, so model-emitted click-family calls can silently no-op and leave automation flows stuck or incorrect — implement these handlers (or explicitly gate/fail fast) before merging.
• In packages/core/lib/v3/agent/AgentProvider.ts, extending hardcoded model-to-provider mappings keeps model onboarding tied to code changes, increasing regression risk whenever new models are introduced — switch to provider-derived/dynamic resolution instead of expanding allowlists.
• In packages/core/lib/v3/llm/LLMProvider.ts, adding support via deprecated unprefixed model IDs prolongs a legacy path and can create inconsistent model resolution behavior — route new support through provider/model IDs and avoid expanding the deprecated mapping.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/lib/v3/agent/GoogleCUAClient.ts">

<violation number="1" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:845">
P2: Gemini-3.x click-family actions are only partially implemented; right/middle click and mouse down/up are still unhandled. When the model emits these function calls, the agent logs unsupported and performs no action.</violation>
</file>

Architecture diagram

sequenceDiagram
    participant App as Agent Loop (run)
    participant Client as GoogleCUAClient (executeStep)
    participant Mapper as convertFunctionCallToAction
    participant Exec as Action Executor
    participant SS as Screenshot Capture
    participant API as Google Gemini API

    Note over App,API: Gemini-3.5-flash CUA turn (happy path)

    App->>Client: executeStep(context, logger)
    Client->>API: send request (history + screenshot)
    API-->>Client: response (functionCalls, usageMetadata)

    alt functionCall has predefined function
        Client->>Mapper: for each part.functionCall
        Note over Mapper: NAME_ALIASES maps 3.x names → 2.5 canonicals<br/>e.g. "click" → "click_at", "type" → "type_text_at"
        Mapper->>Mapper: normalize args shape<br/>(keys: string|array, scroll: magnitudeInPixels,<br/>drag: start/end, type: optional coords)
        Mapper-->>Client: normalized AgentAction (e.g. type, click, screenshot)
    end

    alt action.type === "screenshot"
        Client->>Client: log "take_screenshot: capturing current page"<br/>no browser interaction
    else action.type === "type" AND coordinates present
        Client->>Exec: click (x,y left)
        Client->>Exec: select all (if clearBeforeTyping)
        Client->>Exec: type text
    else action.type === "type" AND no coordinates
        Client->>Exec: type text directly<br/>(element already focused)
    else other executable actions (click_at, scroll_at, etc.)
        Client->>Exec: execute action via browser
    end

    Note over Client: Always capture fresh screenshot after processing actions<br/>(even if no executable actions, e.g. only take_screenshot)

    Client->>SS: captureScreenshot()
    SS-->>Client: screenshot bytes

    Client->>Client: build functionResponses: [screenshot part]

    Client->>API: turn call with functionResponses
    API-->>Client: next turn result (final or continue)

    Client->>Client: aggregate usage (input_tokens, output_tokens,<br/>reasoning_tokens, cached_input_tokens, inference_time_ms)
    Note over Client: reasoning_tokens = usageMetadata.thoughtsTokenCount<br/>cached_input_tokens = usageMetadata.cachedContentTokenCount

    Client-->>App: StepResult with actions, message, usage

    App->>App: accumulate totals for all steps
    App-->>App: final response with full usage (including reasoning, cached)

Loading

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

P2: Gemini-3.x click-family actions are only partially implemented; right/middle click and mouse down/up are still unhandled. When the model emits these function calls, the agent logs unsupported and performs no action.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/core/lib/v3/agent/GoogleCUAClient.ts, line 845:

<comment>Gemini-3.x click-family actions are only partially implemented; right/middle click and mouse down/up are still unhandled. When the model emits these function calls, the agent logs unsupported and performs no action.</comment>

<file context>
@@ -794,19 +824,62 @@ export class GoogleCUAClient extends AgentClient {
+    // NOTE: click and take_screenshot are confirmed from live gemini-3.5-flash
+    // traffic; the rest are inferred from the same drop-the-qualifier pattern
+    // and are safe aliases (any unmapped name still hits the warning below).
+    const NAME_ALIASES: Record<string, string> = {
+      click: "click_at",
+      left_click: "click_at",
</file context>

[miguelg719]

addressed

[cubic-dev-ai]

2 issues found across 1 file (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/lib/v3/agent/GoogleCUAClient.ts">

<violation number="1" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:845">
P2: Gemini-3.x click-family actions are only partially implemented; right/middle click and mouse down/up are still unhandled. When the model emits these function calls, the agent logs unsupported and performs no action.</violation>

<violation number="2" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:926">
P3: New Gemini click-family behavior lacks focused unit tests for conversion semantics and edge cases. Add tests that assert produced AgentAction types/buttons/coordinates for double/triple/right/middle/move.

(Based on your team's feedback about adding unit tests for new behavior.) [FEEDBACK_USED].</violation>
</file>

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

P3: New Gemini click-family behavior lacks focused unit tests for conversion semantics and edge cases. Add tests that assert produced AgentAction types/buttons/coordinates for double/triple/right/middle/move.

(Based on your team's feedback about adding unit tests for new behavior.) .

View Feedback

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/core/lib/v3/agent/GoogleCUAClient.ts, line 926:

<comment>New Gemini click-family behavior lacks focused unit tests for conversion semantics and edge cases. Add tests that assert produced AgentAction types/buttons/coordinates for double/triple/right/middle/move.

(Based on your team's feedback about adding unit tests for new behavior.) .</comment>

<file context>
@@ -893,6 +897,40 @@ export class GoogleCUAClient extends AgentClient {
+        };
+      }
+
+      case "move": {
+        const { x, y } = this.normalizeCoordinates(
+          args.x as number,
</file context>

[miguelg719]

addressed in c6e675c

[miguelg719]

@cubic-dev-ai

[cubic-dev-ai]

@cubic-dev-ai

@miguelg719 I have started the AI code review. It will take a few minutes to complete.

[cubic-dev-ai]

2 issues found across 7 files

Confidence score: 3/5

• In packages/core/lib/v3/agent/GoogleCUAClient.ts, alias normalization happening before custom-tool routing can misclassify overlapping custom tool names as computer-use actions, which can send the wrong action path at runtime—check isCustomTool against rawName before applying NAME_ALIASES to de-risk routing correctness before merging.
• In packages/core/lib/v3/agent/GoogleCUAClient.ts, defaulting missing functionCall.args to {} without validating required-arg functions can trigger crashes or emit invalid actions on malformed calls—add a required-args guard that rejects arg-required function names when args are undefined before merging.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/lib/v3/agent/GoogleCUAClient.ts">

<violation number="1" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:845">
P2: Gemini-3.x click-family actions are only partially implemented; right/middle click and mouse down/up are still unhandled. When the model emits these function calls, the agent logs unsupported and performs no action.</violation>

<violation number="2" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:926">
P3: New Gemini click-family behavior lacks focused unit tests for conversion semantics and edge cases. Add tests that assert produced AgentAction types/buttons/coordinates for double/triple/right/middle/move.

(Based on your team's feedback about adding unit tests for new behavior.) [FEEDBACK_USED].</violation>
</file>

Architecture diagram

sequenceDiagram
    participant UI as Client Application
    participant Agent as AgentProvider
    participant GoogleCUA as GoogleCUAClient
    participant GeminiAPI as Gemini 3.5 Flash API
    participant Executor as Action Executor
    participant Screenshot as Screenshot Capture

    Note over UI,Screenshot: Gemini 3.5 Flash Computer Use Flow

    UI->>Agent: initialize agent with model "google/gemini-3.5-flash"
    Agent->>GoogleCUA: create GoogleCUAClient instance

    Note over GoogleCUA,GeminiAPI: Step Loop (per turn)

    GoogleCUA->>GeminiAPI: executeStep() - send system prompt + screenshot
    GeminiAPI-->>GoogleCUA: response with function calls + usage metadata

    Note over GoogleCUA: Extract usage metrics<br/>including reasoning_tokens & cached_input_tokens

    alt Gemini 3.x function call name received
        GoogleCUA->>GoogleCUA: convertFunctionCallToAction()
        Note over GoogleCUA: Apply NAME_ALIASES mapping<br/>e.g., "click" → "click_at", "type" → "type_text_at"
    end

    alt Click-family action (double_click, triple_click, right_click, middle_click, move)
        GoogleCUA->>GoogleCUA: validate x/y coordinates exist
        alt Coordinates missing
            GoogleCUA->>GoogleCUA: return null (drop invalid action)
        else Coordinates present
            GoogleCUA->>GoogleCUA: normalizeCoordinates(0-999 grid to viewport)
            GoogleCUA->>GoogleCUA: preserve click semantics (button type, click count)
        end
    end

    alt Type action from Gemini 3.x
        Note over GoogleCUA: action.type === "type"
        alt Coordinates present (2.5 style type_text_at)
            GoogleCUA->>GoogleCUA: prepend click action at coordinates
        else No coordinates (3.x style)
            Note over GoogleCUA: Skip click - type into focused element
        end
    end

    alt Screenshot function call
        GoogleCUA->>GoogleCUA: return { type: "screenshot" } (no-op)
    end

    Note over GoogleCUA: Process all actions (may be zero)

    loop For each action
        alt Action is "screenshot" or "open_web_browser"
            GoogleCUA->>GoogleCUA: skip execution, just log
        else Other action
            GoogleCUA->>Executor: execute action (click, type, scroll, etc.)
            Executor-->>GoogleCUA: action result
        end
    end

    Note over GoogleCUA: After all actions processed

    GoogleCUA->>Screenshot: capture fresh screenshot for function response
    Screenshot-->>GoogleCUA: screenshot data

    GoogleCUA->>GeminiAPI: return function responses (including screenshot)
    GeminiAPI-->>GoogleCUA: next model response

    Note over GoogleCUA: Track reasoning_tokens + cached_input_tokens across turns

    UI->>GoogleCUA: getFinalResult()
    GoogleCUA-->>UI: AgentResult with aggregated usage metrics

Loading

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[miguelg719]

@cubic-dev-ai

[cubic-dev-ai]

@cubic-dev-ai

@miguelg719 I have started the AI code review. It will take a few minutes to complete.

[cubic-dev-ai]

2 issues found across 7 files

Confidence score: 3/5

• In packages/core/tests/unit/google-cua-click-conversion.test.ts, the click alias path (click_at without coordinates) appears untested and can send undefined into coordinate normalization, yielding NaN; merging as-is risks malformed click actions at runtime—add a unit test for no-coordinate click and guard click_at against missing coordinates before merging.
• In packages/core/lib/v3/types/public/agent.ts, continuing a hardcoded CUA model allowlist risks future model rollout breakage or unnecessary rejects when names change—refactor to capability/config-based validation (or a centralized source of truth) before this pattern spreads further.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/lib/v3/agent/GoogleCUAClient.ts">

<violation number="1" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:845">
P2: Gemini-3.x click-family actions are only partially implemented; right/middle click and mouse down/up are still unhandled. When the model emits these function calls, the agent logs unsupported and performs no action.</violation>

<violation number="2" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:926">
P3: New Gemini click-family behavior lacks focused unit tests for conversion semantics and edge cases. Add tests that assert produced AgentAction types/buttons/coordinates for double/triple/right/middle/move.

(Based on your team's feedback about adding unit tests for new behavior.) [FEEDBACK_USED].</violation>
</file>

<file name="packages/core/tests/unit/google-cua-click-conversion.test.ts">

<violation number="1" location="packages/core/tests/unit/google-cua-click-conversion.test.ts:71">
P2: Missing test for `click` (aliased to `click_at`) without coordinates. Unlike the dedicated click-family cases, `click_at` has no coordinate guard and passes `undefined` to `normalizeCoordinates`, producing NaN. The PR describes coordinate-less click semantics but the test suite doesn't cover this path.</violation>
</file>

Architecture diagram

sequenceDiagram
    participant Client as Client Application
    participant Provider as AgentProvider
    participant ClientCUA as GoogleCUAClient
    participant GeminiAPI as Google Gemini API
    participant Browser as Browser / Page

    Client->>Provider: startAgent(model="google/gemini-3.5-flash")
    Provider->>ClientCUA: new GoogleCUAClient()
    Note over ClientCUA: NEW: model-to-provider mapping added

    loop step loop
        ClientCUA->>GeminiAPI: sendTurn(userMessage, screenshots, tools)
        GeminiAPI-->>ClientCUA: response (functionCalls, usageMetadata)
        Note over ClientCUA: NEW: parse usageMetadata.thoughtsTokenCount, cachedContentTokenCount
        
        alt functionCalls present
            ClientCUA->>ClientCUA: convertFunctionCallToAction()
            Note over ClientCUA: CHANGED: name aliasing (click→click_at, type→type_text_at, etc.)<br/>NEW: click family (double_click, triple_click, right_click, middle_click, move)<br/>NEW: screenshot → no-op, always returns fresh screenshot<br/>NEW: arg validation (navigate requires url, type requires text)
            
            alt valid coordinates present for click-family
                ClientCUA->>ClientCUA: normalizeCoordinates(x, y)
            else missing coordinates for click-family
                ClientCUA-->>ClientCUA: return null (skip action)
                Note over ClientCUA: NEW: malformed click dropped, prevents NaN in executor
            end
            
            alt type action with coordinates
                ClientCUA->>Browser: click(x, y) [first, then select all if clearBeforeTyping]
                ClientCUA->>Browser: type(text)
            else type action without coordinates (Gemini 3.x `type`)
                ClientCUA->>Browser: type(text) directly (no prior click)
                Note over ClientCUA: NEW: Gemini 3.x type has no coordinates, model focused element already
            end
            
            alt navigate action with url
                ClientCUA->>Browser: goto(url)
            else drag action
                ClientCUA->>Browser: drag(startX,startY,endX,endY)
                Note over ClientCUA: NEW: drag start/end pairs
            else scroll action
                ClientCUA->>Browser: scroll(x,y,deltaX,deltaY / magnitude)
                Note over ClientCUA: NEW: supports magnitude_in_pixels
            else key_combination action
                ClientCUA->>Browser: keyPress(keys)  (array or single)
                Note over ClientCUA: NEW: accepts keys array or single key
            else screenshot action (no UI)
                Note over ClientCUA: NEW: take_screenshot recognized as action, executor skips
            end

            opt custom tool conflict
                ClientCUA->>ClientCUA: log warning when predefined name overrides custom tool
                Note over ClientCUA: NEW: detection of name collision
            end

            Browser-->>ClientCUA: action result (success/error)

        else no functionCalls
            Note over ClientCUA: CHANGED: always take a screenshot even if no actions
        end

        ClientCUA-->>ClientCUA: captureScreenshot() (always happens)
        ClientCUA->>GeminiAPI: sendTurn(functionResponses=screenshot, ...)

        alt model signals completed
            GeminiAPI-->>ClientCUA: final response with isComplete
            ClientCUA-->>Client: final result with usage (including reasoning_tokens, cached_input_tokens)
        else continue loop
            GeminiAPI-->>ClientCUA: next functionCalls
        end
    end

    alt error during step
        ClientCUA-->>Client: error result with partial usage
        Note over ClientCUA: NEW: reasoning_tokens, cached_input_tokens always reported
    end

Loading

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

1 issue found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/lib/v3/agent/GoogleCUAClient.ts">

<violation number="1" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:845">
P2: Gemini-3.x click-family actions are only partially implemented; right/middle click and mouse down/up are still unhandled. When the model emits these function calls, the agent logs unsupported and performs no action.</violation>

<violation number="2" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:926">
P3: New Gemini click-family behavior lacks focused unit tests for conversion semantics and edge cases. Add tests that assert produced AgentAction types/buttons/coordinates for double/triple/right/middle/move.

(Based on your team's feedback about adding unit tests for new behavior.) [FEEDBACK_USED].</violation>
</file>

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic</sub>

[miguelg719]

@cubic-dev-ai

[cubic-dev-ai]

@cubic-dev-ai

@miguelg719 I have started the AI code review. It will take a few minutes to complete.

[cubic-dev-ai]

2 issues found across 7 files

Confidence score: 3/5

• In packages/core/lib/v3/agent/GoogleCUAClient.ts, alias collisions between predefined and custom names can cause a CUA step to run but drop the matching screenshot response, which can break multi-turn agent flow and leave the model without expected state — resolve alias collision handling (or enforce unique alias mapping) before merging.
• In packages/core/lib/v3/agent/GoogleCUAClient.ts, the hover alias path does not validate coordinates, so malformed inputs can produce NaN move values and fail in the executor at runtime instead of being safely ignored — add coordinate guards and reject/skip invalid hover calls before merge.

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/lib/v3/agent/GoogleCUAClient.ts">

<violation number="1" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:860">
P2: `hover` alias lacks coordinate validation and can generate NaN move coordinates. This turns malformed model calls into executor/runtime errors instead of safely dropping them.</violation>

<violation number="2" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:877">
P1: Aliased predefined/custom name collisions can drop the CUA function response. The step may execute an action but fail to return the matching screenshot response, breaking multi-turn model flow.</violation>
</file>

Architecture diagram

sequenceDiagram
    participant App as Application
    participant AgentProv as AgentProvider
    participant GoogleClient as GoogleCUAClient
    participant GeminiAPI as Gemini 3.5 Flash API
    participant Executor as Action Executor
    participant Browser as Browser/Page

    App->>AgentProv: Request agent for model "google/gemini-3.5-flash"
    AgentProv-->>App: Return GoogleCUAClient instance

    App->>GoogleClient: executeStep(logger)
    
    Note over GoogleClient,GeminiAPI: Step Loop (may iterate multiple turns)
    
    GoogleClient->>GeminiAPI: Send conversation + screenshot
    GeminiAPI-->>GoogleClient: FunctionCall[] + usage metadata
    
    Note over GoogleClient: Parse usage from response
    Note over GoogleClient: Track reasoning_tokens & cached_input_tokens
    Note over GoogleClient: Total aggregators updated each step
    
    GoogleClient->>GoogleClient: convertFunctionCallToAction(fc, logger)
    
    Note over GoogleClient: NAME_ALIASES maps 3.x names → 2.5 handlers<br/>e.g. "click" → "click_at", "type" → "type_text_at"
    
    alt Click family (click, double_click, triple_click, right_click, middle_click, move)
        GoogleClient->>GoogleClient: Validate coordinates are finite numbers
        alt Valid coords
            GoogleClient->>GoogleClient: Normalize coordinates (0-999 → viewport)
            GoogleClient-->>App: AgentAction (click/double_click/etc.)
        else Missing/NaN/Infinity
            GoogleClient-->>App: null (reject call)
        end
    else Navigate
        alt URL provided
            GoogleClient-->>App: goto action with url
        else No URL
            GoogleClient-->>App: null (reject call)
        end
    else Type actions
        alt Text provided
            GoogleClient->>GoogleClient: If coords present, prepend click first
            alt clearBeforeTyping
                GoogleClient->>GoogleClient: Also add selectAll action
            end
            GoogleClient-->>App: type action(s)
        else No text
            GoogleClient-->>App: null (reject call)
        end
    else take_screenshot / screenshot
        GoogleClient-->>App: "screenshot" action (no-op, treated like open_web_browser)
    else Custom tool
        alt Name collides with predefined function
            GoogleClient->>GoogleClient: Log warning (predefined wins)
            GoogleClient-->>App: Predefined handler used
        else No collision
            GoogleClient-->>App: custom_tool action
        end
    end

    alt Actions produced
        GoogleClient->>Executor: Execute all actions
        Executor->>Browser: Perform UI interactions
        
        alt Action is take_screenshot
            Note over Executor,Browser: No browser interaction, skipped
        end
        
        Executor-->>GoogleClient: Action results
    else No actions (but model still expects response)
        Note over GoogleClient: Execute empty loop to produce function response
    end

    GoogleClient->>Browser: Capture screenshot (always, even on empty turn)
    Browser-->>GoogleClient: Screenshot data
    
    GoogleClient-->>App: StepResult with actions, screenshot, usage metrics

Loading

<sub>Reply with feedback, questions, or to request a fix.

Fix all with cubic | Re-trigger cubic</sub>

[cubic-dev-ai]

P1: Aliased predefined/custom name collisions can drop the CUA function response. The step may execute an action but fail to return the matching screenshot response, breaking multi-turn model flow.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/core/lib/v3/agent/GoogleCUAClient.ts, line 877:

<comment>Aliased predefined/custom name collisions can drop the CUA function response. The step may execute an action but fail to return the matching screenshot response, breaking multi-turn model flow.</comment>

<file context>
@@ -793,25 +826,90 @@ export class GoogleCUAClient extends AgentClient {
+    // Predefined computer-use tools take precedence over custom tools. If a
+    // custom tool was registered under a reserved (aliased) name, note that the
+    // predefined tool wins rather than silently dropping the custom one.
+    if (rawName in NAME_ALIASES && isCustomTool(functionCall, this.tools)) {
+      logger?.({
+        category: "agent",
</file context>

[cubic-dev-ai]

P2: hover alias lacks coordinate validation and can generate NaN move coordinates. This turns malformed model calls into executor/runtime errors instead of safely dropping them.

Prompt for AI agents

Check if this issue is valid — if so, understand the root cause and fix it. At packages/core/lib/v3/agent/GoogleCUAClient.ts, line 860:

<comment>`hover` alias lacks coordinate validation and can generate NaN move coordinates. This turns malformed model calls into executor/runtime errors instead of safely dropping them.</comment>

<file context>
@@ -793,25 +826,90 @@ export class GoogleCUAClient extends AgentClient {
+      left_click: "click_at",
+      type: "type_text_at",
+      type_text: "type_text_at",
+      hover: "hover_at",
+      scroll: "scroll_at",
+      drag: "drag_and_drop",
</file context>

[seanmcguire12]

@miguelg719 nit: can you tighten this up a bit? a little too implementation specific

[cubic-dev-ai]

1 issue found across 1 file (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="packages/core/lib/v3/agent/GoogleCUAClient.ts">

<violation number="1" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:845">
P2: Gemini-3.x click-family actions are only partially implemented; right/middle click and mouse down/up are still unhandled. When the model emits these function calls, the agent logs unsupported and performs no action.</violation>

<violation number="2" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:860">
P2: `hover` alias lacks coordinate validation and can generate NaN move coordinates. This turns malformed model calls into executor/runtime errors instead of safely dropping them.</violation>

<violation number="3" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:877">
P1: Aliased predefined/custom name collisions can drop the CUA function response. The step may execute an action but fail to return the matching screenshot response, breaking multi-turn model flow.</violation>

<violation number="4" location="packages/core/lib/v3/agent/GoogleCUAClient.ts:926">
P3: New Gemini click-family behavior lacks focused unit tests for conversion semantics and edge cases. Add tests that assert produced AgentAction types/buttons/coordinates for double/triple/right/middle/move.

(Based on your team's feedback about adding unit tests for new behavior.) [FEEDBACK_USED].</violation>
</file>

<file name=".changeset/gemini-3-5-flash-cua.md">

<violation number="1" location=".changeset/gemini-3-5-flash-cua.md:5">
P3: Fix malformed markdown spacing in changeset summary. Missing space before the inline code token reduces changelog readability.</violation>
</file>

<sub>Tip: Review your code locally with the cubic CLI to iterate faster.

Fix all with cubic | Re-trigger cubic</sub>