Skip to content

Support DuckDB percentile medians#185

Open
YahiaQuacks wants to merge 1 commit into
fivetran:mainfrom
YahiaQuacks:bugfix/duckdb-percentile-medians
Open

Support DuckDB percentile medians#185
YahiaQuacks wants to merge 1 commit into
fivetran:mainfrom
YahiaQuacks:bugfix/duckdb-percentile-medians

Conversation

@YahiaQuacks
Copy link
Copy Markdown

Please provide your name and company

Yahia Mostafa, MotherDuck.

Link the issue/feature request which this PR is meant to address

No public issue yet. This addresses a DuckDB compatibility issue in the Jira package’s median metric models.

Detail what changes this PR introduces and how this addresses the issue/feature request linked above.

This PR updates the Jira median metric models so DuckDB uses the grouped aggregate percentile path already used by Postgres. Depends on fivetran/dbt_fivetran_utils#155, which adds the DuckDB dispatch for fivetran_utils.percentile.

DuckDB supports percentile_cont(...) within group (order by ...) as an aggregate, but does not support the Redshift-style percentile_cont(...) within group (...) over (...) window form emitted through the default fivetran_utils.percentile dispatch.

This PR:

  • Adds DuckDB to the grouped median CTE condition in user metrics.
  • Adds DuckDB to the grouped median CTE condition in project metrics.
  • Updates the fivetran_utils dependency floor to a version that includes duckdb__percentile.
  • Updates changelog, package versions, and dependency documentation.

How did you validate the changes introduced within this PR?

Validated against DuckDB:

  • Aggregate form succeeds: percentile_cont(0.5) within group (order by x).
  • Windowed ordered-set form fails with Catalog Error: Aggregate Function with name percentile_cont does not exist!, matching the reported failure.
  • A grouped Jira-style median query returns expected per-user medians.

Also ran git diff --check.

Which warehouse did you use to develop these changes?

DuckDB/MotherDuck.

Did you update the CHANGELOG?

  • Yes

Did you update the dbt_project.yml files with the version upgrade (please leverage standard semantic versioning)? (In both your main project and integration_tests)

  • Yes

Typically there are additional maintenance changes required before this will be ready for an upcoming release. Are you comfortable with the Fivetran team making a few commits directly to your branch?

  • Yes
  • No

If you had to summarize this PR in an emoji, which would it be?

🦆

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant