Add more test writing instructions to AGENTS.md#780
Open
alex-clickhouse wants to merge 6 commits into
Open
Conversation
Signed-off-by: Alex Soffronow Pagonidis <alex.soffronow@clickhouse.com>
alex-clickhouse
requested review from
joe-clickhouse and
peter-leonov-ch
as code owners
Signed-off-by: Alex Soffronow Pagonidis <alex.soffronow@clickhouse.com>
mshustov
reviewed
Jun 5, 2026
| - Inspect existing tests covering similar areas to make sure you are writing tests in the right style and with the right coverage. | ||
| - Make sure to test both the happy path as well as sad paths. Make sure to cover all relevant edge cases. | ||
| - When testing types, cover the full type matrix: ClickHouse type hints compose and formatting is recursive, so a change to how a value is formatted must be tested across all the shapes it can take: scalar, Array(T), Tuple(...), Array(Tuple(...)), Nullable(T), and spelling and case variants of the type name. | ||
| - Aim to be complete, but also terse. Don't use two tests to cover what could be done in one. |
Member
There was a problem hiding this comment.
Do we have any noteworthy patterns for parametrized tests that we should point out?
Contributor
There was a problem hiding this comment.
Yes, I'll leave in a separate comment.
joe-clickhouse
approved these changes
Jun 5, 2026
joe-clickhouse
left a comment
joe-clickhouse
left a comment
Contributor
There was a problem hiding this comment.
Looks good! Regarding the test matrix stuff, I'd add one calling out how to paramaterize a test.
- Express a matrix of cases with
@pytest.mark.parametrizerather than near-duplicate test bodies. This satisfies the terseness goal below and is the right place for the type-shape and case-variant coverage above.
And then another specifically bringing up the fact that sync and async client can use a paramaterized fixture to test both one a single test.
- Client behavior must hold for both the sync and async clients. For integration tests use the
param_clientandcallfixtures fromtests/integration_tests/conftest.py, which run one test body against both transports via theclient_modeparameter. Seetests/integration_tests/test_temporal.pyfor the pattern. Do not write a sync-only test for a change that touches shared client code.
Signed-off-by: Alex Soffronow Pagonidis <alex.soffronow@clickhouse.com>
Contributor
Author
|
Don't merge this yet pls, I'm running some experiments and they'd be contaminated if we changed AGENTS.md mid-way through. |
Contributor
|
@alex-clickhouse just let me know when you're ready to merge. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ran into some issues when testing agentic bug fixes, hopefully this will lead the agent to behave a bit better :)