Skip to content

feat: Improve consensus test ergonomics with snapshot testing#3

Closed
Jiloc wants to merge 87 commits intojferrant:feat/aac-testing-boiler-platefrom
Jiloc:feat/aac-tests-with-insta
Closed

feat: Improve consensus test ergonomics with snapshot testing#3
Jiloc wants to merge 87 commits intojferrant:feat/aac-testing-boiler-platefrom
Jiloc:feat/aac-tests-with-insta

Conversation

@Jiloc
Copy link
Copy Markdown

@Jiloc Jiloc commented Sep 30, 2025

Description

This PR is about simplifying the process of writing and updating our consensus tests.

the current drill: write a test, run it, see it fail, extract the data and then manually recreate the ExpectedResult struct from the terminal back into your code. It's easy to mess up.

This change introduces snapshot testing using a crate called insta.

Long story short: with this, we don't have to write the ExpectedResult by hand!

So, how does it work

Instead of stuffing the ExpectedResult into our test files, insta saves it automatically into a separate .snap file (or inline, I have an example). These new files live in a snapshots/ directory and we check them into Git. They become the "source of truth" for what the output should be.

Pros:

  • Writing the tests should be easier. You just write the test setup and let insta generate the correct output for you.
  • Our test files look cleaner. They're all about the logic, not giant blocks of data.
  • Easy Diffs for Reviews: When a change breaks consensus, the git diff on the .snap file will show exactly what changed in the output. Reviewing consensus-breaking changes should be easier.
  • Updating the fixtures on consensus-breaking releases can be automated

How do I use this?

Writing a new test:

  1. Set up your ConsensusTestVector like before, but just... don't add an expected_result.
  2. Run the test logic: let result = ConsensusTest::new(...).run();
  3. Add this magic line at the end: insta::assert_ron_snapshot!(result);
  4. Run cargo insta review to create the snapshot for you:
    cargo insta review
  5. Run the test
  6. Just commit the new .snap file along with your code.

Updating a test after you broke something (on purpose!):

  1. Run cargo test. It'll fail, and insta will print a beautiful diff showing you exactly what's different.
  2. If the new output is correct and expected, just run cargo insta review again.
  3. insta will automatically update the snapshot file for you.
  4. Commit your code and the updated snapshot. Your PR will now have a clear "before and after" of the consensus change.

Applicable issues

  • fixes #

Additional info (benefits, drawbacks, caveats)

Checklist

  • Test coverage for new or modified code paths
  • Changelog is updated
  • Required documentation changes (e.g., docs/rpc/openapi.yaml and rpc-endpoints.md for v2 endpoints, event-dispatcher.md for new events)
  • New clarity functions have corresponding PR in clarity-benchmarking repo
  • New integration test(s) added to bitcoin-tests.yml

fdefelici and others added 30 commits September 17, 2025 09:32
Jiloc and others added 23 commits September 24, 2025 12:59
…ate_ephemeral_marf

Feat/block simulate ephemeral marf
…-events

chore: remove `stacks-events` binary
…b-cli-to-stacks-inspect

chore: move `stackslib::cli` module to `stacks-inspect`
- renamed stacks-inspect functions
  - replay_block -> validate_block
  - replay_block_nakamoto -> validate_block_nakamoto
…_replay

Renaming block-replay script to block-validation
…only-for-miner

bugfix: enforce bitcoin rpc only for miner
…fallible-to-compile-phase-6467

refactor: aac convert infallible error to compile time
@Jiloc Jiloc changed the title feat: use insta for easier tests setup feat: Improve consensus test ergonomics with snapshot testing Sep 30, 2025
let msg = format!("Invalid Stacks block 518dfea674b5c4874e025a31e01a522c8269005c0685d12658f0359757de6692: {e:?}");
let test_vector = ConsensusTestVector {
initial_balances,
initial_balances: vec![],
Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh interesting. I am surprised this didn't break the test. Thought it would complain about insufficient stx.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah I found out that the boot_nakamoto function used internally already gives funds to the private key passed

peer_config.initial_balances = vec![(addr.to_account_principal(), 1_000_000_000_000_000_000)];

@Jiloc Jiloc closed this Oct 1, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants