refactor(consumoor): align route batches with schema v2#803
Open
refactor(consumoor): align route batches with schema v2#803
Conversation
Regenerate consumoor columnar route batches for the Schema V2 ClickHouse migration sourced from Savid/chmigration: https://github.com/Savid/chmigration/tree/5b9a8ad00e715f1d8ce5124fa35a4fc63a1f1b95 Update handwritten adapters to match the new ClickHouse column contracts. Key runtime fixes: - populate updated_date_time for replacing-merge routes - write nullable execution payload block hash, block number, and fee recipient fields for beacon v2 blocks - write UInt64 duration_ms values for execution engine instrumentation routes - write nullable remote_port values for libp2p connected and disconnected events - write non-null direction values for libp2p handle metadata and handle status routes - write non-null node_id values for node_record_consensus - populate kzg_commitments for beacon data column sidecar events Generator output also removes legacy version-column handling where Schema V2 dropped those fields and updates route tests to the regenerated row shape.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR aligns
pkg/consumoorwith the Schema V2 ClickHouse migration sourced fromSavid/chmigrationat commit5b9a8ad.The bulk of the diff is regenerated route batch code to reflect the new table definitions. On top of that, the handwritten route adapters were updated where Schema V2 changed column nullability, removed legacy dedup/version columns, or introduced fields that need explicit population at runtime.
What changed
updated_date_timepopulation for replacing-merge routes that now depend on it for dedup/versioning.execution_payload_block_hash,execution_payload_block_number, andexecution_payload_fee_recipientvalues.UInt64duration_msvalues.remote_portvalues.directionvalues.node_idvalues.kzg_commitmentsfor beacon data column sidecar events.versioncolumn handling where Schema V2 dropped those fields from the relevant canonical/libp2p tables.Migration alignment
This PR is intended to match the Schema V2 migration work in
Savid/chmigration, specifically the state captured at5b9a8ad, including:updated_date_timeValidation
Ran uncached route package tests after the regeneration and handwritten fixes:
go test -count=1 ./pkg/consumoor/route/beacon ./pkg/consumoor/route/execution ./pkg/consumoor/route/libp2p ./pkg/consumoor/route/node ./pkg/consumoor/route/canonicalAlso verified the broader consumoor packages locally:
go test ./pkg/consumoor/...Notes