Skip to content

Use dedicated MAI transcription for dictation - #334042

Merged
Megan Rogge (meganrogge) merged 3 commits into
microsoft:mainfrom
accnops:arthurcnops/mai-dictation-transcription
Sep 3, 2026
Merged

Use dedicated MAI transcription for dictation#334042
Megan Rogge (meganrogge) merged 3 commits into
microsoft:mainfrom
accnops:arthurcnops/mai-dictation-transcription

Conversation

@accnops

Copy link
Copy Markdown
Contributor

MAI dictation now uses the dedicated /api/v1/realtime/transcription WebSocket instead of sharing Voice Mode's /voice session.

  • derives the transcription endpoint without changing Voice Mode routing
  • adds a strict, lifecycle-safe transcription transport with ping/pong and terminal error handling
  • treats MAI final transcripts as authoritative, including shorter or empty finals
  • waits for the backend's bounded finalization window while allowing cancel and rapid restart to finish immediately
  • preserves the existing local dictation path

Fixes #332324.

This restores and adapts the dedicated transcription-client direction from #326228 by Mir Imad Ahmed.

Testing:

  • focused browser tests: 70 passed
  • touched-file TypeScript diagnostics: none
  • ESLint on all changed files
  • hygiene pre-commit hook
  • Fallow new-only audit: pass

Copilot AI balanced review requested due to automatic review settings September 2, 2026 16:45

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Voice routing changes unexpectedly, and stale socket errors can terminate a restarted dictation session.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Moves MAI dictation from Voice Mode’s shared socket to a dedicated transcription transport.

Changes:

  • Adds endpoint derivation and a lifecycle-aware transcription client.
  • Integrates authoritative MAI finals and bounded finalization.
  • Adds focused endpoint, transport, and dictation tests.
File summaries
File Description
voiceEndpoint.ts Resolves voice/transcription URLs and authentication.
voiceClientService.ts Uses shared endpoint helpers.
voiceCodeTranscriptionClient.ts Implements dedicated WebSocket transport.
chatSpeechToTextService.ts Integrates MAI transcription and finalization.
chat.shared.contribution.ts Registers the transcription service.
voiceEndpoint.test.ts Tests endpoint handling.
voiceCodeTranscriptionClient.test.ts Tests transport lifecycle and protocol.
chatSpeechToTextService.test.ts Tests MAI integration and final selection.
Review details
  • Files reviewed: 8/8 changed files
  • Comments generated: 4
  • Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/vs/workbench/contrib/chat/browser/speechToText/chatSpeechToTextService.ts Outdated
Comment thread src/vs/workbench/contrib/chat/browser/voiceClient/voiceEndpoint.ts Outdated
Comment thread src/vs/workbench/contrib/chat/test/browser/chatSpeechToTextService.test.ts Outdated
auto-merge was automatically disabled September 2, 2026 18:49

Head branch was pushed to by a user without write access

Arthur Cnops and others added 3 commits September 3, 2026 11:20
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c568e3e0-eabb-48db-84a4-a7976df7c474
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c568e3e0-eabb-48db-84a4-a7976df7c474
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: c568e3e0-eabb-48db-84a4-a7976df7c474
@meganrogge
Megan Rogge (meganrogge) force-pushed the arthurcnops/mai-dictation-transcription branch from e717706 to f4c6b81 Compare September 3, 2026 15:20
auto-merge was automatically disabled September 3, 2026 15:20

Head branch was pushed to by a user without write access

@meganrogge
Megan Rogge (meganrogge) merged commit c5e2568 into microsoft:main Sep 3, 2026
34 of 60 checks passed
@vs-code-engineering vs-code-engineering Bot added this to the 1.137.0 milestone Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Dictation replaces or deletes the correct visible transcript after Stop, even with llmCleanup disabled

5 participants