[MAINTENANCE] Add SQL result_format regression coverage for expect_column_values_to_be_unique#11964
Open
joshua-stauffer wants to merge 7 commits into
Open
[MAINTENANCE] Add SQL result_format regression coverage for expect_column_values_to_be_unique#11964joshua-stauffer wants to merge 7 commits into
joshua-stauffer wants to merge 7 commits into
Conversation
…_be_unique Pin down behaviors of the column_values.unique metric on SQL backends that are easy to break when its implementation changes: - COMPLETE + unexpected_index_column_names composed with a row_condition - include_unexpected_rows composed with a row_condition - unexpected_index_query must execute against the source database and return exactly the unexpected rows - exclude_unexpected_values must return the columnar index-list shape shared by all SQL map expectations - importing great_expectations must not emit metric re-registration warnings on stderr
✅ Deploy Preview for niobium-lead-7998 canceled.
|
- Narrow batch.datasource to SQLDatasource before calling get_engine() (mypy attr-defined) - Allow 30s for the subprocess import test; the default unit timeout of 2s is too tight for a full library import
The ignore on the NotImported fallback assignment is only needed when pyarrow is installed (mypy then types the name as Module). In environments without pyarrow the module resolves as Any, and --warn-unused-ignores fails the build. Adding unused-ignore makes the ignore valid in both environments.
…thub.com/fivetran/great_expectations into m/unique-result-format-regression-guards
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds regression coverage for
ExpectColumnValuesToBeUniqueresult_format behaviors on SQL backends (PostgreSQL, SQLite). These behaviors depend on internals of thecolumn_values.uniquemetric that are easy to break when its SQL implementation changes, and were previously untested:COMPLETE+unexpected_index_column_namescomposed with arow_condition— the test data is constructed so a value is duplicated only when the filtered-out row is included, proving the condition is applied and the index list hydrates correctly.include_unexpected_rowscomposed with arow_condition.unexpected_index_querymust be executable against the source database and return exactly the unexpected rows (the query string is surfaced to users in validation results and Data Docs).exclude_unexpected_values=Truemust return the columnar{index_column: [values, ...]}index-list shape shared by all SQL map expectations.great_expectationsmust not emit metric re-registration warnings on stderr.Changes
tests/integration/data_sources_and_expectations/expectations/test_expect_column_values_to_be_unique.py: five new tests plus a dedicated dataset and a small helper that fails loudly if any metric raised instead of producing a value.Testing
develop.