fix(webrtc): resolve native cursor drift with absolute positioning - #594
Conversation
…rsor is visible Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
📝 WalkthroughNote: range_2119 above should be range_46d8bf45ef75. WalkthroughAdds absolute cursor position reporting to the cursor overlay controller, introduces a new absolute mouse input type (INPUT_MOUSE_ABS) with payload interface and encoder in the input protocol, and updates the WebRTC client to send absolute mouse packets pinning server cursor position when the overlay cursor is visible. ChangesAbsolute mouse cursor support
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant Overlay as GfnCursorOverlayController
participant Client as GfnWebRtcClient
participant Encoder as InputEncoder
participant Server
Overlay->>Client: getAbsolutePosition()
Client->>Client: queueMouseMovement sets pendingMouseAbs
Client->>Encoder: encodeMouseAbsolute(pendingMouseAbs)
Encoder-->>Client: absolute packet
Client->>Server: send INPUT_MOUSE_ABS packet
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit a553ab3. Configure here.
| // Movement was already consumed by the overlay position; drop any | ||
| // stale relative residue so it cannot double-apply after this packet. | ||
| this.pendingMouseDxFloat = 0; | ||
| this.pendingMouseDyFloat = 0; |
There was a problem hiding this comment.
Stale absolute mouse pending
Medium Severity
After the overlay cursor becomes hidden, pendingMouseAbs can still hold the last visible-cursor position. The next flushMouse run prefers that stale absolute packet, sends it, and zeroes accumulated relative deltas, so pointer-lock movement in hidden-cursor mode is dropped and the server may jump to an outdated position.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit a553ab3. Configure here.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a553ab3921
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| this.pendingMouseAbs = abs; | ||
| this.pendingMouseDxFloat = 0; | ||
| this.pendingMouseDyFloat = 0; |
There was a problem hiding this comment.
When the cursor becomes visible while a relative mouse batch is still pending (for example, a game toggles from hidden/raw input back to a menu before mouseFlushIntervalMs fires), this branch replaces the batch with an absolute packet and clears pendingMouseDxFloat/pendingMouseDyFloat, so the last raw movement is never sent. The reverse transition can also lose deltas because flushMouse prioritizes a stale pendingMouseAbs and then zeroes any relative movement queued after the cursor hid. Flush or keep separate pending state when switching between absolute and relative modes instead of clearing the other mode's movement.
Useful? React with 👍 / 👎.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@opennow-stable/src/renderer/src/gfn/webrtcClient.ts`:
- Around line 3676-3684: Clear any queued absolute mouse move before sending the
direct overlay alignment in gfn/webrtcClient.ts. In the overlayAbs branch of the
cursor handling path, make sure any existing pendingMouseAbs state or scheduled
flush is canceled/reset before calling inputEncoder.encodeMouseAbsolute and
sendReliable, so a later flush cannot replay a stale position. Use the
pendingMouseAbs handling logic around the absolute-move queue and the
direct-send branch to keep them in sync.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 781eb9ac-c1b4-42ae-ae67-f89bfa6ac6f5
📒 Files selected for processing (4)
opennow-stable/src/renderer/src/gfn/cursorChannel.tsopennow-stable/src/renderer/src/gfn/inputProtocol.test.tsopennow-stable/src/renderer/src/gfn/inputProtocol.tsopennow-stable/src/renderer/src/gfn/webrtcClient.ts
| if (overlayAbs) { | ||
| // Overlay cursor is visible: pin the server cursor with one | ||
| // absolute packet instead of simulating relative moves. | ||
| const movePayload = this.inputEncoder.encodeMouseAbsolute({ | ||
| ...overlayAbs, | ||
| timestampUs: timestampUs(), | ||
| }); | ||
| this.sendReliable(movePayload); | ||
| markServerCursorAt(overlayAbs); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Clear stale pending absolute movement before direct alignment send.
Line 3679 sends an absolute alignment packet directly, but any older pendingMouseAbs queued by Line 3767 remains scheduled and can later flush, moving the server cursor back to a stale overlay position.
Suggested fix
if (overlayAbs) {
// Overlay cursor is visible: pin the server cursor with one
// absolute packet instead of simulating relative moves.
+ this.pendingMouseAbs = null;
+ this.pendingMouseDxFloat = 0;
+ this.pendingMouseDyFloat = 0;
+ this.pendingMouseTimestampUs = null;
+ this.mouseCoalescedBatchEntries = 0;
const movePayload = this.inputEncoder.encodeMouseAbsolute({
...overlayAbs,
timestampUs: timestampUs(),
});
this.sendReliable(movePayload);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (overlayAbs) { | |
| // Overlay cursor is visible: pin the server cursor with one | |
| // absolute packet instead of simulating relative moves. | |
| const movePayload = this.inputEncoder.encodeMouseAbsolute({ | |
| ...overlayAbs, | |
| timestampUs: timestampUs(), | |
| }); | |
| this.sendReliable(movePayload); | |
| markServerCursorAt(overlayAbs); | |
| if (overlayAbs) { | |
| // Overlay cursor is visible: pin the server cursor with one | |
| // absolute packet instead of simulating relative moves. | |
| this.pendingMouseAbs = null; | |
| this.pendingMouseDxFloat = 0; | |
| this.pendingMouseDyFloat = 0; | |
| this.pendingMouseTimestampUs = null; | |
| this.mouseCoalescedBatchEntries = 0; | |
| const movePayload = this.inputEncoder.encodeMouseAbsolute({ | |
| ...overlayAbs, | |
| timestampUs: timestampUs(), | |
| }); | |
| this.sendReliable(movePayload); | |
| markServerCursorAt(overlayAbs); |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@opennow-stable/src/renderer/src/gfn/webrtcClient.ts` around lines 3676 -
3684, Clear any queued absolute mouse move before sending the direct overlay
alignment in gfn/webrtcClient.ts. In the overlayAbs branch of the cursor
handling path, make sure any existing pendingMouseAbs state or scheduled flush
is canceled/reset before calling inputEncoder.encodeMouseAbsolute and
sendReliable, so a later flush cannot replay a stale position. Use the
pendingMouseAbs handling logic around the absolute-move queue and the
direct-send branch to keep them in sync.
![capy-ai[bot]](https://avatars.githubusercontent.com/in/1915919?s=60&v=4){kind=small}
…rsor is visible (OpenCloudGaming#594) (OpenCloudGaming#595) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…rsor is visible (OpenCloudGaming#594) Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
1 participant
This PR fixes cursor drift between the native WebRTC overlay cursor and the server-side cursor by implementing the official GFN client's absolute-position input path (type 5) while the client-rendered cursor is visible. Previously, we only sent relative deltas (type 7), which accumulated rounding and clamp errors that caused visible drift. The official client sends absolute coordinates normalized to a client-defined extent to pin the server cursor to the overlay position exactly.
Input Protocol:
INPUT_MOUSE_ABSconstant (type 5) toinputProtocol.tsencodeMouseAbsolutewith official 26-byte layout:[type 4B LE][x 2B BE][y 2B BE][reserved 2B BE][width 2B BE][height 2B BE][reserved 4B BE][timestamp 8B BE]clampU16helper to constrain coordinates to 0–65535isPartiallyReliableHidTransferEligibleto includeINPUT_MOUSE_ABSCursor Overlay:
isCursorVisible()onGfnCursorOverlayControllerto query visibility stategetAbsolutePosition()to return the clamped overlay cursor position and viewport extent in CSS pixelsWebRTC Client:
pendingMouseAbsstate to track the latest absolute position awaiting transmissionqueueMouseMovementto send absolute positions when the overlay cursor is visible instead of relative deltasflushMousewith an absolute-position path that bypasses relative scaling and marks the server cursor at the exact overlay positionTests:
INPUT_MOUSE_ABSNote
Medium Risk
Touches the live WebRTC mouse input path and coalescing behavior; incorrect coordinates could misplace the remote cursor, but scope is limited to local-cursor mode and is covered by new encoder tests.
Overview
Fixes cursor drift between the client-rendered overlay and the remote cursor by sending absolute mouse position (input type 5) while the overlay is visible, instead of only relative deltas (type 7) that accumulate rounding error.
The input layer adds
INPUT_MOUSE_ABS, a 26-byteencodeMouseAbsolutelayout aligned with the official GFN client, and treats absolute moves like relative moves for partially reliable HID transfer. The cursor overlay exposesisCursorVisible()andgetAbsolutePosition()(clamped position plus viewport extent in CSS pixels).GfnWebRtcClientqueuespendingMouseAbson pointer movement when the overlay is shown (latest position wins, relative residue cleared), flushes absolute packets ahead of relative scaling, and uses absolute alignment on pointer-lock entry instead of simulated relative jumps.Reviewed by Cursor Bugbot for commit a553ab3. Configure here.
Summary by CodeRabbit
New Features
Bug Fixes