Skip to content

[server] Avoid memory copy when follower fetch log from leader#1186

Merged
wuchong merged 1 commit into
apache:mainfrom
swuferhong:fix-follower-fetchLog
Jun 27, 2025
Merged

[server] Avoid memory copy when follower fetch log from leader#1186
wuchong merged 1 commit into
apache:mainfrom
swuferhong:fix-follower-fetchLog

Conversation

@swuferhong
Copy link
Copy Markdown
Contributor

Purpose

Linked issue: close #1183

As described in #1183, the current Fluss server consumes excessive heap memory and triggers frequent GC. Debugging revealed that this memory footprint stems from an off-heap to heap memory copy during follower-to-leader data sync. This PR aims to resolve this issue by eliminating the unnecessary copy. Post-fix, heap memory usage is significantly reduced, with before-and-after results shown below:

before fix:
image

after fix:
image

image

Brief change log

Tests

API and Format

Documentation

Copy link
Copy Markdown
Contributor

@loserwang1024 loserwang1024 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have left some comment

@swuferhong swuferhong force-pushed the fix-follower-fetchLog branch from 0ac142e to 5cac37e Compare June 26, 2025 11:26
Copy link
Copy Markdown
Member

@wuchong wuchong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wuchong wuchong merged commit 476c3ce into apache:main Jun 27, 2025
4 checks passed
polyzos pushed a commit to polyzos/fluss that referenced this pull request Aug 30, 2025
polyzos pushed a commit to HZY-Wade/fluss that referenced this pull request Aug 31, 2025
Ugbot pushed a commit to Ugbot/fluss that referenced this pull request Apr 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Zero-Copy for follower fetching logs from leader

3 participants