-
Notifications
You must be signed in to change notification settings - Fork 1.5k
allow more services in Scenario Mode #653
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
allow more services in Scenario Mode #653
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR introduces conditional logic to allow more production services to run when in Scenario Automation mode, which is a testing mode that aims to measure the real user experience while still disabling certain codepaths like telemetry and OAuth flows. The changes modify extension mode checks throughout the codebase to exclude scenario automation from test-mode restrictions.
- Adds
isScenarioAutomationchecks to various extension mode conditions - Enables production services (URL opener, notifications) in scenario automation mode
- Maintains telemetry and experimentation service restrictions in scenario automation
- Adjusts timeout behavior for prompt elements to use production timeouts
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/extension/prompts/node/panel/symbolDefinitions.tsx | Updates timeout logic to use production timeout when in scenario automation |
| src/extension/prompts/node/panel/safeElements.tsx | Prevents throwing errors in scenario automation mode for better production simulation |
| src/extension/prompts/node/panel/referencesAtPosition.tsx | Updates timeout logic to use production timeout when in scenario automation |
| src/extension/prompts/node/panel/definitionAtPosition.tsx | Updates timeout logic to use production timeout when in scenario automation |
| src/extension/onboardDebug/vscode-node/copilotDebugCommandContribution.ts | Adjusts version nonce generation to use production behavior in scenario automation |
| src/extension/log/vscode-node/loggingActions.ts | Disables fetcher telemetry collection in scenario automation mode |
| src/extension/inlineChat/vscode-node/inlineChatCommands.ts | Enables review service context value updates in scenario automation |
| src/extension/extension/vscode/services.ts | Enables production URL opener and notification services in scenario automation |
| src/extension/extension/vscode/extension.ts | Prevents early return from activation in scenario automation mode |
| src/extension/extension/vscode-node/services.ts | Maintains telemetry and experimentation service restrictions in scenario automation |
You can also share your feedback on Copilot code review for a chance to win a $100 gift card. Take the survey.
Co-authored-by: Copilot <[email protected]>
When running in Scenario Automation (distinct from
stestmode and set via an env var), as much production code as possible should be used so the real user experience is measured. However, certain codepaths — like telemetry — should be disabled regardless of the formal VSCodeExtensionMode, and certain codepaths (like full OAuth flow) must not be run.When running local builds,
extensionModeisExtensionMode.Testwhile Marketplace downloaded builds areExtensionMode.Productionso relying solely onextensionModeis not possible.An abandoned iteration of this PR attempted to colocate all this logic within one central location but I found that made the callsites that dependend on that logic particularly hard to follow, hence this PR proposes a less DRY, but more readable approach.