Fix slowdown during historical sync because of tracking reorgs too early#626
Conversation
WalkthroughThis change refactors event batching and selection logic in the ChainManager module by simplifying the retrieval of the next event and removing unnecessary pattern matching and intermediate functions. The test suite is updated to reflect these changes, and redundant or obsolete code is eliminated for clarity and maintainability. Changes
Sequence Diagram(s)sequenceDiagram
participant Client
participant ChainManager
participant FetchState
Client->>ChainManager: createOrderedBatch()
ChainManager->>ChainManager: getOrderedNextItem(fetchStatesMap)
ChainManager->>FetchState: Check reorg threshold for event
alt Event within threshold
ChainManager-->>Client: Add event to batch
else Event outside threshold
ChainManager-->>Client: Skip event
end
Possibly related PRs
Suggested reviewers
Poem
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (3)
🧰 Additional context used📓 Path-based instructions (2)`**/*.res`: Never use `[| item |]` to create an array. Use `[ item ]` instead. M...
📄 Source: CodeRabbit Inference Engine (.cursor/rules/rescript.mdc) List of files the instruction was applied to:
`**/*.{res,resi}`: ReScript has record types which require a type definition bef...
📄 Source: CodeRabbit Inference Engine (.cursor/rules/rescript.mdc) List of files the instruction was applied to:
🧠 Learnings (1)scenarios/test_codegen/test/ChainManager_test.res (1)⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
🔇 Additional comments (10)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
2 participants
Fixes the bug of entering realtime indexing mode too early for unordered multichain mode. In this mode, we start saving entity history to be able to rollback on reorg, which drastically slows down historical sync if enabled during historical sync.
The issue was caused by invalid logic for checking whether we entered the reorg threshold. For the unordered multichain mode, it checked only the chain we used for batch creation, which became a single chain at a time after https://github.com/enviodev/hyperindex/releases/tag/v2.22.0 - causing the perf regression as soon as one of the chains reaches the head.
In older version the bug also existed, but there was a much slower chance of triggering it - one chain should reach the head and have some events, while all the other chains shouldn't have events to process (fetching).
Summary by CodeRabbit
Refactor
Tests