[BUGFIX] Fix crash rendering results with unexpected_index_column_names and no domain column#11948
Closed
Bharath-970 wants to merge 1 commit into
Closed
Conversation
…es and no domain column
_convert_unexpected_indices_to_df computes domain_column_name_list as
whatever keys of an unexpected_index_list entry aren't in
unexpected_index_column_names. When every key IS an ID/PK column (no
separate domain/value column present -- e.g. result_format=COMPLETE with
unexpected_index_column_names covering all tracked keys), that list is
empty and pandas.DataFrame.groupby([]) raises:
ValueError: No group keys passed!
aborting Data Docs rendering for the affected result.
With no domain column to group by, each unexpected row already stands on
its own -- there's nothing to merge across rows. Skip the groupby/agg in
that case and wrap each row's values in a single-item list directly,
matching the same output shape .groupby().agg() would otherwise produce.
Added a regression test exercising the previously-crashing input shape;
confirmed as a true guard (fails without the fix, passes with it). Full
tests/render/ suite: 168 passed.
Closes fivetran#11933.
👷 Deploy request for niobium-lead-7998 pending review.Visit the deploys page to approve it
|
Author
|
Duplicate — reopening #11943 instead, which was mistakenly closed based on a wrong assumption that this repo was a fork rather than the current (renamed/transferred) upstream. Sorry for the noise. |
Open
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #11933.
Problem
When rendering a validation result for Data Docs with
result_formatCOMPLETEandunexpected_index_column_namesset to the entire set of keys present in eachunexpected_index_listentry (i.e. no separate domain/value column is tracked alongside the ID/PK columns), rendering crashes:Root cause
In
great_expectations/render/util.py,_convert_unexpected_indices_to_dfcomputes:When every key in
first_unexpected_indexis itself one ofunexpected_index_column_names, this set difference is empty. The function then calls:.groupby([])raisesValueError: No group keys passed!since pandas requires at least one group key.Fix
When there is no domain column to group by, there is nothing to aggregate across rows -- each row already stands on its own. Skip the groupby/agg and instead wrap each cell's own value in a single-item list, matching the same per-cell shape
.groupby().agg(_agg_func)would otherwise produce (a list of values per cell), just with each list containing exactly one element instead of being merged across rows.Test
Added
test_convert_unexpected_indices_to_df_no_domain_columnintests/render/test_util.py, covering the exact scenario from the issue (dict keys ==unexpected_index_column_names).Verified as a true regression guard: fails with the original
ValueErrorwhen the fix is reverted, passes with it restored. Also verified the existing grouped (domain-column-present) test cases are unaffected.