Skip to content

Comments

fix(clickhouse sink): add support for default columns#24692

Open
benjamin-awd wants to merge 9 commits intovectordotdev:masterfrom
benjamin-awd:clickhouse-default-columns
Open

fix(clickhouse sink): add support for default columns#24692
benjamin-awd wants to merge 9 commits intovectordotdev:masterfrom
benjamin-awd:clickhouse-default-columns

Conversation

@benjamin-awd
Copy link
Contributor

@benjamin-awd benjamin-awd commented Feb 19, 2026

Summary

This PR adds support for handling default columns like MATERIALIZED, ALIAS, EPHEMERAL, DEFAULT for the Clickhouse ArrowStream format.

Currently, users must use batch_encoding.allow_nullable_fields: true which is not desirable when strict schema enforcement is required.

How did you test this PR?

Unit + integration tests

Change Type

  • Bug fix
  • New feature
  • Non-functional (chore, refactoring, docs)
  • Performance

Is this a breaking change?

  • Yes
  • No

Does this PR include user facing changes?

  • Yes. Please add a changelog fragment based on our guidelines.
  • No. A maintainer will apply the no-changelog label to this PR.

References

Closes #24667

Notes

  • Please read our Vector contributor resources.
  • Do not hesitate to use @vectordotdev/vector to reach out to us regarding this PR.
  • Some CI checks run only after we manually approve them.
    • We recommend adding a pre-push hook, please see this template.
    • Alternatively, we recommend running the following locally before pushing to the remote branch:
      • make fmt
      • make check-clippy (if there are failures it's possible some of them can be fixed with make clippy-fix)
      • make test
  • After a review is requested, please avoid force pushes to help us review incrementally.
    • Feel free to push as many commits as you want. They will be squashed into one before merging.
    • For example, you can run git merge origin master and git push.
  • If this PR introduces changes Vector dependencies (modifies Cargo.lock), please
    run make build-licenses to regenerate the license inventory and commit the changes (if any). More details here.

@benjamin-awd benjamin-awd requested a review from a team as a code owner February 19, 2026 13:30
@github-actions github-actions bot added the domain: sinks Anything related to the Vector's sinks label Feb 19, 2026
@pront pront enabled auto-merge February 23, 2026 17:46
@pront pront added this pull request to the merge queue Feb 23, 2026
@github-merge-queue github-merge-queue bot removed this pull request from the merge queue due to failed status checks Feb 23, 2026
Copy link
Member

@pront pront left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: integration tests failed (link)

…n test

The Arrow serializer expects a typed Timestamp value, not a plain string,
for DateTime columns. Using a string caused a chrono parse error that
prevented the batch from being sent.
@pront pront enabled auto-merge February 24, 2026 19:57
auto-merge was automatically disabled February 25, 2026 07:12

Head branch was pushed to by a user without write access

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

domain: sinks Anything related to the Vector's sinks

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Clickhouse sink: ArrowStream format fails schema validation on MATERIALIZED/DEFAULT/ALIAS columns

2 participants