Skip to content

automations: keep cloud automation drafts off sandbox routing and show pending target status neutrally - #340652

Merged
Ben Villalobos (benvillalobos) merged 2 commits into
microsoft:mainfrom
benvillalobos:investigate/cloud-automation-create-20261008
Oct 9, 2026
Merged

Ben Villalobos (benvillalobos) merged 2 commits into
microsoft:mainfrom
benvillalobos:investigate/cloud-automation-create-20261008

Conversation

@benvillalobos

Copy link
Copy Markdown
Member

Code stats

Measured against origin/main at 52f6dc2c2918e2c92fb7106ed645358c35d4b5c7.

Category What it does Implementation Tests
Shared draft purpose Pass one explicit automation-draft flag +4/-2 +8/-5
Cloud provider boundary Keep scheduled automation configuration off ordinary sandbox routing +4/-3 +152/-2
Dialog pending status Separate progress from validation errors and preserve save guards +39/-9 +223/-0
Documentation Explain the draft-purpose contract +2/-0 +0/-0
Total, including documentation +49/-14 +383/-7

Context

Creating a Cloud automation with Claude Opus 5.5 fails when the chat.agentHost.cloudSandbox.enabled experiment is on. Switching targets also shows "Saving the current session configuration..." in red, even though it is progress rather than an error.

Changes made

Automation drafts reused ordinary Cloud routing. With the sandbox experiment on, the picker selected a sandbox-qualified model ID whose schema caused capture to return {}. The scheduled-Cloud provider correctly rejected that defined model configuration, but the dialog showed a generic connection error.

Pass explicit automation purpose so these drafts keep the scheduled-Cloud model catalogue. Ordinary Cloud sessions keep sandbox routing. The sandbox-only provider rejects automation drafts, and strict unsupported-setting validation and existing {} reset behavior are unchanged.

The two shared Sessions files have only a one-flag change: sessionsProvider.ts declares optional isAutomationDraft; sessionsManagementService.ts sets it in the two automation creation methods.

Target capture and repository eligibility now have separate pending status in normal foreground, without aria-invalid. Save stays blocked until pending work finishes. Actual failures remain errors, and unchanged messages are not repeatedly written to live regions.

Testing

  • Wrote three regression tests before changing production code. All three failed with the sandbox-qualified Opus ID, {} capture rejection, and pending messages presented as validation errors. The same three passed after the fix.
  • After rebasing onto the base above, npm run typecheck-client passed and npm run build-fast -- --client-only passed.
  • The following five-suite command passed: 632 passing, 6 pre-existing pending.
.\scripts\test.bat --run src\vs\sessions\contrib\providers\copilotChatSessions\test\browser\copilotChatSessionsProvider.test.ts --run src\vs\sessions\contrib\providers\copilotChatSessions\test\browser\cloudAutomationStore.test.ts --run src\vs\sessions\contrib\automations\test\browser\automationDialog.test.ts --run src\vs\sessions\services\sessions\test\browser\sessionsManagementService.test.ts --run src\vs\sessions\contrib\chat\test\browser\sessionModelSelection.test.ts --reporter spec

Coverage includes sandbox on/off and setting changes, ordinary-session isolation, restored configuration, raw model payloads, strict rejection, pending-to-error transitions, cancellation, retry, Save races, and unchanged live-region text.

Live signed-in verification in Insiders is pending until sign-in is refreshed.

References

Keep scheduled Cloud automation drafts separate from ordinary sandbox routing. Show target capture and repository checks as pending status while preserving save guards and strict configuration validation.
Copilot AI balanced review requested due to automatic review settings October 9, 2026 03:32

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The authoritative Automation and Copilot provider specifications currently contradict each other about sandbox routing.

1 open finding
What changed in this PR

Preserves scheduled Cloud routing for Automation drafts and presents pending target checks as neutral progress.

Changes:

  • Adds explicit Automation-draft routing.
  • Separates pending target status from validation errors.
  • Adds comprehensive routing, validation, accessibility, and race tests.
File Description
sessionsProvider.ts Adds the draft-purpose option.
sessionsManagementService.ts Marks Automation drafts explicitly.
sessionsManagementService.test.ts Tests option propagation and isolation.
copilotChatSessionsProvider.ts Bypasses sandbox routing for Automation drafts.
copilotChatSessionsProvider.test.ts Tests model routing and strict validation.
cloudAutomationStore.test.ts Tests template validation and dispatch.
automationDialog.ts Adds neutral pending-target state.
automationDialogService.ts Blocks saves during pending work.
automationDialog.css Styles pending status neutrally.
automationDialog.test.ts Tests status, accessibility, retries, and races.
AUTOMATIONS.md Documents draft-purpose routing.

🧠 Review effort: Balanced


💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/vs/sessions/AUTOMATIONS.md Resolved
Scope sandbox-setting routing to interactive Cloud drafts and link the automation draft contract.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cloud routing depends on external service behavior, and signed-in end-to-end verification remains pending.

0 open findings

1 resolved since last review

🧠 Review effort: Balanced

@benvillalobos
Ben Villalobos (benvillalobos) marked this pull request as ready for review October 9, 2026 04:54
@vs-code-engineering

Copy link
Copy Markdown
Contributor

📬 CODENOTIFY

The following users are being notified based on files changed in this PR:

Sandeep Somavarapu (@sandy081)

Matched files:

  • src/vs/sessions/services/sessions/browser/sessionsManagementService.ts
  • src/vs/sessions/services/sessions/common/sessionsProvider.ts
  • src/vs/sessions/services/sessions/test/browser/sessionsManagementService.test.ts

Ladislau Szomoru (@lszomoru)

Matched files:

  • src/vs/sessions/services/sessions/browser/sessionsManagementService.ts
  • src/vs/sessions/services/sessions/common/sessionsProvider.ts
  • src/vs/sessions/services/sessions/test/browser/sessionsManagementService.test.ts

@benvillalobos

Ben Villalobos (benvillalobos) commented Oct 9, 2026 •

Copy link
Copy Markdown
Member Author

signed in e2e validation passed, this should now allow testing the feature at all on insiders with this flag set

@connor4312 Connor Peet (connor4312) left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed for security and correctness, including automation-draft routing, configuration validation, pending-target save guards, and regression coverage. No substantiated P1 or P2 defects found.

Written by Connor's agent running GPT-6.1 Sol

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants