Honor configured AOF segment size in TsavoriteLogSettings#1798
Conversation
Agent-Logs-Url: https://github.com/microsoft/garnet/sessions/cc0abf6b-ee36-45e2-8578-3c89663bdc1b Co-authored-by: badrishc <18355833+badrishc@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR makes AOF file segmentation honor server configuration by introducing an explicit AofSegmentSize option and plumbing it into the TsavoriteLogSettings used for AOF. This aligns AOF truncation/rollover behavior with configured limits by allowing smaller on-disk segment (file) granularity than Tsavorite’s prior implicit default.
Changes:
- Add new
AofSegmentSizeserver option (default1g), including defaults and documentation. - Pass
SegmentSizeBitsinto AOFTsavoriteLogSettingsand add page-size vs segment-size validation. - Add config test coverage to ensure the option flows through and validation triggers.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| website/docs/getting-started/configuration.md | Document new --aof-segment-size configuration setting. |
| test/Garnet.test/GarnetServerConfigTests.cs | Add tests verifying default/override segment size and invalid page>segment validation. |
| libs/server/Servers/GarnetServerOptions.cs | Add AofSegmentSize option, compute bits, plumb into TsavoriteLogSettings, and validate page vs segment. |
| libs/host/defaults.conf | Define default AofSegmentSize: 1g. |
| libs/host/Configuration/Options.cs | Add CLI option --aof-segment-size with memory-size validation and map into GarnetServerOptions. |
Comments suppressed due to low confidence (1)
test/Garnet.test/GarnetServerConfigTests.cs:1363
- Same as above: this second GetAofSettings call disposes only LogDevice; LogCommitManager should also be disposed, and cleanup should be in a finally/using pattern to avoid leaking resources if assertions throw.
serverOptions = options.GetServerOptions();
serverOptions.GetAofSettings(0, out logSettings);
ClassicAssert.AreEqual(1, logSettings.Length);
ClassicAssert.AreEqual(1L << 26, logSettings[0].SegmentSize);
foreach (var s in logSettings)
s.LogDevice?.Dispose();
Agent-Logs-Url: https://github.com/microsoft/garnet/sessions/172f9b38-65ea-4ef5-acee-e348757c9259 Co-authored-by: badrishc <18355833+badrishc@users.noreply.github.com>
Agent-Logs-Url: https://github.com/microsoft/garnet/sessions/ffdda9db-bb9a-4574-a510-c52b3647caf3 Co-authored-by: badrishc <18355833+badrishc@users.noreply.github.com>
|
@badrishc I’m not seeing this change in release/v1 (v1.1.x). Could this be cherry-picked/backported into the next v1.1.x release? |
|
@copilot - please backport this PR to target release/v1 (and update version in that PR so its merge will create a new release). |
* Honor configured AOF segment size in TsavoriteLogSettings (#1798) * Initial plan * Add AofSegmentSize option and apply it to TsavoriteLogSettings Agent-Logs-Url: https://github.com/microsoft/garnet/sessions/cc0abf6b-ee36-45e2-8578-3c89663bdc1b Co-authored-by: badrishc <18355833+badrishc@users.noreply.github.com> * Validate AOF sizes before allocating devices to avoid handle leak Agent-Logs-Url: https://github.com/microsoft/garnet/sessions/172f9b38-65ea-4ef5-acee-e348757c9259 Co-authored-by: badrishc <18355833+badrishc@users.noreply.github.com> * Dispose LogCommitManager + use try/finally in AOF segment-size test Agent-Logs-Url: https://github.com/microsoft/garnet/sessions/ffdda9db-bb9a-4574-a510-c52b3647caf3 Co-authored-by: badrishc <18355833+badrishc@users.noreply.github.com> --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: badrishc <18355833+badrishc@users.noreply.github.com> Co-authored-by: Badrish Chandramouli <badrishc@microsoft.com> (cherry picked from commit 4cc403f) * Backport PR 1798 to release/v1 --------- Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com> Co-authored-by: badrishc <18355833+badrishc@users.noreply.github.com>
Fix: AOF truncation ignores configured segment size (always uses default 1 GB granularity).
GarnetServerOptions.GetAofSettings()was constructingTsavoriteLogSettingswithout settingSegmentSizeBits, so the AOF always used Tsavorite's default 1 GB segment, preventing file-level truncation at smallerAofSizeLimitvalues.AofSegmentSizeoption (default1g, preserving existing behavior).AofSegmentSizeintoTsavoriteLogSettings.SegmentSizeBitsinGetAofSettings(), with page-size <= segment-size validation.defaults.confand website configuration docs.AofSegmentSizeFlowsToTsavoriteLogSettingstest covering default, override, and validation.GetAofDeviceso a validation failure cannot leak file handles. (Also fixes the same pre-existing pattern for the page-vs-memory check.)LogCommitManager(alsoIDisposable) in addition toLogDevice, and usetry/finallyso disposal runs even if an assertion fails.RespAofTestspass.