Skip to content

Dynamic Contract Pre Registration#252

Merged
JonoPrest merged 12 commits into
mainfrom
jp/contract-preregistration
Oct 11, 2024
Merged

Dynamic Contract Pre Registration#252
JonoPrest merged 12 commits into
mainfrom
jp/contract-preregistration

Conversation

@JonoPrest

@JonoPrest JonoPrest commented Oct 4, 2024

Copy link
Copy Markdown
Collaborator

Implements Dynamic Contract Preregistration

  • Adds separate start up process for only fetching events from contractRegister functions configured for pre registration
  • Adds indexer restart resistance
  • Implemented ONLY for HyperSync indexers currently (should not be much work to implement for fuel and RPC)
  • Adds a TUI representation for pre registration

Before this gets merged for a release, we need to validate how the change affects the hosted service UI since events sync state table has been updated

Base automatically changed from release-concurrent-partitions to main October 7, 2024 08:58
@JonoPrest JonoPrest force-pushed the jp/contract-preregistration branch from 3ad7e16 to 952732b Compare October 8, 2024 11:21
@JonoPrest JonoPrest marked this pull request as ready for review October 8, 2024 14:06
@JonoPrest JonoPrest requested a review from JasoonS October 8, 2024 14:06
@JonoPrest

Copy link
Copy Markdown
Collaborator Author

It doesn't break anything on the hosted service. Will need to adapt the UI a bit to represent the pre registration step.

type eventConfig<'eventFilter> = {
wildcard?: bool,
eventFilters?: SingleOrMultiple.t<'eventFilter>,
preRegister?: bool,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
preRegister?: bool,
shouldPreRegisterDynamicContracts?: bool,

(and everywhere else)

I think it is good to have consistent naming on this. I assume you want it to be less verbose in the config and more verbose in the code. Maybe then we just use preregister everywhere instead of the longer variable. I don't think it is unclear to just be preRegister

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMO I prefer the verbose naming since it's quite an esoteric feature.

The shortened version I just used for the user API since I don't think they should have such a verbose flag. In the same way, we use isWildcard for our internal config everywhere but for the user API it's simply wildcard

I would be open to renaming the the user facing config flag but would prefer to keep the verbose name everywhere else.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, maybe middle ground - preRegisterDynamicContracts everywhere?

I don't really agree that the variable will be confusing, especially since the variable comes from the user's handlers and will be in our box, etc., but I think we can make it more descriptive

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JasoonS for the user API we have Contract.Event.contractRegister so maybe even preRegisterContracts I was thinking?

Still prefer verbose naming with "should" or "is" prefix in internal code to imply boolean type.


//Instantiate each time to add new registered contract addresses
let recieptsSelection = getRecieptsSelection(~contractAddressMapping, ~shouldApplyWildcards)
let recieptsSelection = if isPreRegisteringDynamicContracts {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, is there no way to catch this at codegen time? I guess the user will get notified as soon as it starts. So, fine.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nvm - it is temprorary too.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No since the config is set with the js API

@JasoonS JasoonS left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, I think the only real thing is to change the variable to pre-register everywhere, even internally. The code seems really good. I didn't notice any bugs or anything.

It is quite a bit to follow though, so I don't fully trust myself to have caught bugs - but I think we can start testing it in the wild.

//On the first time we enter the reorg threshold, copy all entities to entity history
//And set the isInReorgThreshold isInReorgThreshold state to true
dispatchAction(SetIsInReorgThreshold(true))
//TODO: persist the isInReorgThreshold state to the db in a transaction with copy

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just removed?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's implemented 👍🏼

Comment thread codegenerator/cli/templates/static/codegen/src/Config.res
buffered={latestFetchedBlockNumber - firstEventBlockNumber}
outOf={toBlock - firstEventBlockNumber}
loadingColor=Secondary
loadingColor={isPreRegisteringDynamicContracts ? Primary : Secondary}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 👍

chainWorker: module(ChainWorker.S),
}

let shouldPreRegisterDynamicContracts = (chainConfig: chainConfig) => {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this function name is fine, but then change the variable to just preRegister (as the user sees it)

@JonoPrest JonoPrest force-pushed the jp/contract-preregistration branch from b4add71 to c8418e6 Compare October 10, 2024 14:31
@JonoPrest JonoPrest merged commit 2ef29c7 into main Oct 11, 2024
@JonoPrest JonoPrest deleted the jp/contract-preregistration branch October 11, 2024 11:44
DenhamPreen pushed a commit that referenced this pull request Oct 22, 2024
* Wip initial impl of dynamic contract pre-fetching

* Fix contractPreRegistration LogSelection

* Add latest processed blocks during pre registration

* Add restart resistance to pre registration

* Handle multiple partitions with contract preregistration

* Add runtime error for pre registration on fuel and rpc

* Implement user api for pre registration

* Remove logs

* Update TUI

* Fix tests

* Cleaner shouldPreRegisterDynamicContracts function

Co-authored-by: Jason Smythe <jason@envio.dev>

* Update naming of preRegisterDynamicContracts api

---------

Co-authored-by: Jason Smythe <jason@envio.dev>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants