Skip to content

refactor(consumoor): align route batches with schema v2#803

Open
Savid wants to merge 2 commits intomasterfrom
release/ch2-chmigration-ref
Open

refactor(consumoor): align route batches with schema v2#803
Savid wants to merge 2 commits intomasterfrom
release/ch2-chmigration-ref

Conversation

@Savid
Copy link
Copy Markdown
Member

@Savid Savid commented Mar 31, 2026

Summary

This PR aligns pkg/consumoor with the Schema V2 ClickHouse migration sourced from Savid/chmigration at commit 5b9a8ad.

The bulk of the diff is regenerated route batch code to reflect the new table definitions. On top of that, the handwritten route adapters were updated where Schema V2 changed column nullability, removed legacy dedup/version columns, or introduced fields that need explicit population at runtime.

What changed

  • Regenerated consumoor columnar route batches for the updated ClickHouse schemas.
  • Added updated_date_time population for replacing-merge routes that now depend on it for dedup/versioning.
  • Updated beacon v2 block execution payload handling to write nullable execution_payload_block_hash, execution_payload_block_number, and execution_payload_fee_recipient values.
  • Updated execution engine instrumentation routes to write UInt64 duration_ms values.
  • Updated libp2p connected/disconnected routes to write nullable remote_port values.
  • Updated libp2p handle metadata/status routes to write non-null direction values.
  • Updated node record consensus routing to write non-null node_id values.
  • Populated kzg_commitments for beacon data column sidecar events.
  • Removed legacy route-side version column handling where Schema V2 dropped those fields from the relevant canonical/libp2p tables.
  • Refreshed affected tests/snapshots to match regenerated row shapes.

Migration alignment

This PR is intended to match the Schema V2 migration work in Savid/chmigration, specifically the state captured at 5b9a8ad, including:

  • moving replacing-merge tables onto updated_date_time
  • applying nullability/type changes used by the new SQL
  • removing route handling for columns removed from the target schemas

Validation

Ran uncached route package tests after the regeneration and handwritten fixes:

go test -count=1 ./pkg/consumoor/route/beacon ./pkg/consumoor/route/execution ./pkg/consumoor/route/libp2p ./pkg/consumoor/route/node ./pkg/consumoor/route/canonical

Also verified the broader consumoor packages locally:

go test ./pkg/consumoor/...

Notes

  • Most files in the PR are generated output changes caused by the schema migration.
  • The risk is primarily integration with live/staging ClickHouse rather than compile/test correctness; route tests are green locally.

Savid added 2 commits March 31, 2026 11:37
Regenerate consumoor columnar route batches for the Schema V2 ClickHouse migration sourced from Savid/chmigration:

https://github.com/Savid/chmigration/tree/5b9a8ad00e715f1d8ce5124fa35a4fc63a1f1b95

Update handwritten adapters to match the new ClickHouse column contracts.

Key runtime fixes:

- populate updated_date_time for replacing-merge routes

- write nullable execution payload block hash, block number, and fee recipient fields for beacon v2 blocks

- write UInt64 duration_ms values for execution engine instrumentation routes

- write nullable remote_port values for libp2p connected and disconnected events

- write non-null direction values for libp2p handle metadata and handle status routes

- write non-null node_id values for node_record_consensus

- populate kzg_commitments for beacon data column sidecar events

Generator output also removes legacy version-column handling where Schema V2 dropped those fields and updates route tests to the regenerated row shape.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant