Skip to content

feat: on-disk DB format-version guard (txpool + blockchain)#4

Open
orrinfrazier wants to merge 1 commit into
mainfrom
feat/s04-1-db-versioning
Open

feat: on-disk DB format-version guard (txpool + blockchain)#4
orrinfrazier wants to merge 1 commit into
mainfrom
feat/s04-1-db-versioning

Conversation

@orrinfrazier
Copy link
Copy Markdown
Owner

@orrinfrazier orrinfrazier commented May 24, 2026

Closes #14

Summary

Adds an on-disk schema/format-version marker to both fjall databases (txpool + blockchain) and a guard that refuses to start on mismatch — closing the S04-1 beta blocker. Without a persisted version, any future tapes/keyspace schema change on a shipped v0.1.0 database could not be detected, and a newer binary would silently misread or panic instead of refusing/migrating.

What changed

A format_version value (u64 little-endian) is persisted under b"format_version" in each DB's metadata keyspace at creation, and verified on every open:

  • absent → fresh/pre-versioning DB: stamp the current version and proceed
  • present & equal → proceed
  • present & mismatched, or malformed (len ≠ 8) → refuse with DbFormatVersionMismatch { expected, found } (returned, never panics), leaving room for a future migration step

Per piece:

  • txpool (storage/txpool/src/txpool.rs, error.rs): activate the previously-unused metadata keyspace (dropped #[expect(dead_code)] // TODO: version?), add DATABASE_FORMAT_VERSION, add the error variant, and change open_with_databaseResult<Self, TxPoolError> (sole caller init_with_pool already returns TxPoolError, so no caller change).
  • blockchain (storage/blockchain/src/database.rs, error.rs): reuse the existing constants::DATABASE_VERSION; check/stamp it before Tapes::open so an incompatible DB is refused without creating any tape files. Added the error variant + tempfile dev-dep.

Open question (resolved)

Fjall 3.0.4's internal FormatVersion (V1/V2/V3) guards only fjall's own LSM disk format per major release — not the application schema (keyspace layout, value encodings, tapes layout). So an app-level marker is still required.

How to test

cargo test -p cuprate-txpool -p cuprate-blockchain
cargo clippy -p cuprate-txpool -p cuprate-blockchain --all-targets

10 new tests cover all branches in both crates: fresh-stamp, reopen-with-match, mismatch → error, and malformed-value → error (no panic), plus a db_version() sanity check.

Notes

  • Pre-existing repo-wide issues NOT touched by this PR: cuprate-types dead-code warnings (PoolTxInfo, default_zero) and cargo audit advisories in transitive deps (rsa, rustls-webpki, lz4_flex, bincode, paste).
  • Reviewed via 3-way cross-family consensus (Claude + Gemini + Codex); the one finding (fail-fast placement on the blockchain check) was resolved before this PR.

Persist a `format_version` marker (u64 little-endian under `b"format_version"`)
in the `metadata` keyspace of both the txpool and blockchain fjall databases,
and verify it on open: a fresh DB is stamped, a matching version proceeds, and
a mismatched or malformed version is refused with `DbFormatVersionMismatch`
(never panics), leaving room for a future migration step.

- txpool: activate the previously-unused `metadata` keyspace (drop the
  `#[expect(dead_code)] // TODO: version?`), add `DATABASE_FORMAT_VERSION`,
  and change `open_with_database` to return `Result<Self, TxPoolError>`.
- blockchain: check/stamp the existing `DATABASE_VERSION` before `Tapes::open`
  so an incompatible database is refused without creating any tape files.

Resolves the S04-1 beta blocker: without this marker, a future tapes/keyspace
schema change on a shipped v0.1.0 database could not be detected and would
silently misread or panic instead of refusing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

storage: add on-disk DB schema versioning and migration guard

1 participant