Prune entity history sequentially per entity + improve metrics - #691
Conversation
WalkthroughReplaces rollback duration histogram with counters and adds per-entity prune metrics in Prometheus.res. Updates GlobalState rollback path to use Hrtime-based timing and increment new RollbackSuccess counters. Removes previous startTimer/endTimer usage and introduces RollbackHistoryPrune with labeled counters. Changes
Sequence Diagram(s)sequenceDiagram
participant GS as GlobalState
participant HR as Hrtime
participant PM as Prometheus.RollbackSuccess
GS->>HR: makeTimer()
activate HR
HR-->>GS: startTime
deactivate HR
GS->>GS: perform rollback/reorg work
GS->>HR: timeSince(startTime)
HR-->>GS: elapsedMillis
GS->>PM: increment(timeMillis=elapsedMillis)
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
Poem
Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. ✨ Finishing Touches🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
| module RollbackDuration = { | ||
| let histogram = PromClient.Histogram.make({ | ||
| "name": "envio_rollback_duration", | ||
| "help": "Rollback on reorg duration in seconds", | ||
| "buckets": [0.5, 1., 5., 10.], | ||
| module RollbackSuccess = { | ||
| let timeCounter = PromClient.Counter.makeCounter({ | ||
| "name": "envio_rollback_time", | ||
| "help": "Rollback on reorg total time in milliseconds", | ||
| }) | ||
|
|
||
| let counter = PromClient.Counter.makeCounter({ | ||
| "name": "envio_rollback_count", | ||
| "help": "Number of successful rollbacks on reorg", | ||
| }) | ||
|
|
||
| let startTimer = () => { | ||
| histogram->PromClient.Histogram.startTimer |
There was a problem hiding this comment.
Found time + count combo much more informative than histogram for these kind of cases. It also takes less space in the metrics response, which is already quite big 😅
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🔭 Outside diff range comments (1)
codegenerator/cli/templates/static/codegen/src/globalState/GlobalState.res (1)
875-914: Pruning is still concurrent; switch to sequential per-entity and emit per-entity metricsThis path still executes pruning concurrently across entities via Promise.all, which contradicts the stated objective “Prune entity history sequentially per entity + improve metrics” and prevents per-entity timing metrics from being emitted. Running sequentially also reduces DB contention.
Refactor to:
- Run prune sequentially per entity.
- Time each entity prune and increment Prometheus.RollbackHistoryPrune with the entity label.
- Optionally, update benchmark summary per-entity.
Apply this diff:
- let timeRef = Hrtime.makeTimer() - let _ = await Promise.all( - Entities.allEntities->Belt.Array.map(entityConfig => { - Db.sql->DbFunctions.EntityHistory.pruneStaleEntityHistory( - ~entityName=entityConfig.name, - ~safeChainIdAndBlockNumberArray, - ~shouldDeepClean, - ) - }), - ) - - if Env.Benchmark.shouldSaveData { - let elapsedTimeMillis = Hrtime.timeSince(timeRef)->Hrtime.toMillis->Hrtime.floatFromMillis - - Benchmark.addSummaryData( - ~group="Other", - ~label="Prune Stale History Time (ms)", - ~value=elapsedTimeMillis, - ) - } + let entities = Entities.allEntities + for idx in 0 to entities->Array.length - 1 { + let entityConfig = entities->Array.getUnsafe(idx) + let tRef = Hrtime.makeTimer() + await Db.sql->DbFunctions.EntityHistory.pruneStaleEntityHistory( + ~entityName=entityConfig.name, + ~safeChainIdAndBlockNumberArray, + ~shouldDeepClean, + ) + let elapsed = Hrtime.timeSince(tRef)->Hrtime.toMillis + Prometheus.RollbackHistoryPrune.increment( + ~timeMillis=elapsed, + ~entityName=entityConfig.name, + ) + + if Env.Benchmark.shouldSaveData { + Benchmark.addSummaryData( + ~group="Other", + ~label=`Prune Stale History (${entityConfig.name}) (ms)`, + ~value=elapsed->Hrtime.floatFromMillis, + ) + } + }
🧹 Nitpick comments (1)
codegenerator/cli/npm/envio/src/Prometheus.res (1)
497-512: RollbackSuccess metrics: approach is soundReplacing histogram with total-time and count counters is consistent with computing averages downstream. No labels keeps cardinality low. Consider adding "labelNames": [] for consistency with other counters, but not required.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (2)
codegenerator/cli/npm/envio/src/Prometheus.res(1 hunks)codegenerator/cli/templates/static/codegen/src/globalState/GlobalState.res(2 hunks)
🧰 Additional context used
📓 Path-based instructions (4)
**/*.{res,resi}
📄 CodeRabbit Inference Engine (.cursor/rules/rescript.mdc)
**/*.{res,resi}: Never use[| item |]to create an array. Use[ item ]instead.
Must always use=for setting value to a field. Use:=only for ref values created usingreffunction.
ReScript has record types which require a type definition before hand. You can access record fields by dot likefoo.myField.
It's also possible to define an inline object, it'll have quoted fields in this case.
Use records when working with structured data, and objects to conveniently pass payload data between functions.
Never use %raw to access object fields if you know the type.
Files:
codegenerator/cli/templates/static/codegen/src/globalState/GlobalState.rescodegenerator/cli/npm/envio/src/Prometheus.res
codegenerator/cli/templates/{dynamic/**/*.hbs,static/**}
📄 CodeRabbit Inference Engine (.cursor/rules/navigation.mdc)
Templates live under codegenerator/cli/templates: dynamic/ for Handlebars (.hbs), static/ for raw Rescript files copied verbatim.
Files:
codegenerator/cli/templates/static/codegen/src/globalState/GlobalState.res
{**/generated/src/**/*.res,codegenerator/cli/templates/static/codegen/src/**/*.res,codegenerator/cli/templates/dynamic/codegen/src/**/*.res}
📄 CodeRabbit Inference Engine (.cursor/rules/navigation.mdc)
Runtime code lives in each project’s generated/src, but template versions (good for editing) are under codegenerator/cli/templates/static/codegen/src or codegenerator/cli/templates/dynamic/codegen/src.
Files:
codegenerator/cli/templates/static/codegen/src/globalState/GlobalState.res
codegenerator/cli/npm/envio/**
📄 CodeRabbit Inference Engine (.cursor/rules/navigation.mdc)
Library-fied runtime shared across indexers lives in codegenerator/cli/npm/envio.
Files:
codegenerator/cli/npm/envio/src/Prometheus.res
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: build_and_test
🔇 Additional comments (2)
codegenerator/cli/templates/static/codegen/src/globalState/GlobalState.res (2)
1042-1042: Good switch to Hrtime-based timingCapturing the rollback start time with Hrtime is appropriate and aligns with the new Prometheus API.
1156-1157: Correct Prometheus usage for rollback successPassing Hrtime.milliseconds directly to Prometheus.RollbackSuccess.increment is type-correct and avoids unnecessary conversions. Looks good.
2 participants
Summary by CodeRabbit
New Features
Refactor
Chores