Skip to content

Add client-level HTTP headers#776

Merged
joe-clickhouse merged 7 commits into
ClickHouse:mainfrom
river-walras:main
Jun 8, 2026
Merged

Add client-level HTTP headers#776
joe-clickhouse merged 7 commits into
ClickHouse:mainfrom
river-walras:main

Conversation

@river-walras

Copy link
Copy Markdown
Contributor

Summary

Add a connection-level headers keyword to both sync and async client creation so callers can attach HTTP headers to every request made by clickhouse-connect, including the initialization queries sent during client construction.

Use Case

This supports ClickHouse deployments exposed through HTTP gateways that require request headers before traffic reaches ClickHouse. A concrete example is Cloudflare Tunnel with Cloudflare Access / Zero Trust service tokens:

client = clickhouse_connect.get_client(
    host="clickhouse.example.com",
    secure=True,
    username="default",
    password="...",
    headers={
        "CF-Access-Client-Id": "...",
        "CF-Access-Client-Secret": "...",
    },
)

Per-request transport_settings can already send HTTP headers for normal queries, but it cannot help with client initialization because _init_common_settings() runs before the caller can pass per-request options. Connection-level headers close that gap.

Parameter Design

I used an explicit headers parameter instead of promoting transport_settings to the constructor because the new values are plain HTTP headers, and transport_settings already has overloaded request-level meaning in the driver. Keeping the names separate makes the public API easier to discover and avoids spreading that ambiguity to client creation.

Header merge order is intentionally:

  1. Driver defaults, including auth and User-Agent
  2. Client-level headers
  3. Per-request transport_settings
  4. Forced Host from server_host_name

This lets users intentionally override driver-generated headers when they need a gateway-specific escape hatch, while preserving per-request overrides for one-off calls.

Tests

  • Added unit coverage for sync client headers being present during initialization.
  • Added sync and async unit coverage for request-level override behavior.
  • Added shared client factory coverage with Cloudflare-style headers.

Local validation run:

  • uv run ruff check clickhouse_connect/driver/__init__.py clickhouse_connect/driver/httpclient.py clickhouse_connect/driver/asyncclient.py tests/integration_tests/test_client.py tests/unit_tests/test_driver/test_httpclient.py
  • uv run pytest tests/unit_tests/test_driver/test_httpclient.py::TestHttpClientHeaders tests/unit_tests/test_driver/test_httpclient.py::TestAsyncClientHeaders -q

The focused integration test could not be completed locally because no ClickHouse server was listening on localhost:8123.

Need: clients behind Cloudflare Access or similar HTTP gateways must send authentication headers on every request, including the initialization queries that run during client creation.

Design: expose an explicit headers keyword on sync and async factories instead of reusing transport_settings, because these values are actual HTTP headers and transport_settings already has overloaded request-level meaning. Merge after driver defaults so users can intentionally override headers like Authorization or User-Agent, while per-request transport_settings still wins for one-off requests.

Tests: cover sync and async header merge behavior plus client_factory construction with Cloudflare-style headers.
@CLAassistant

CLAassistant commented Jun 4, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new connection-level headers option to ClickHouse Connect HTTP clients so callers can attach HTTP headers to all requests (including the initialization queries executed during client construction).

Changes:

  • Added headers: dict[str, str] | None to sync HttpClient and async AsyncClient constructors and applied them after driver-generated defaults.
  • Plumbed a headers keyword through create_client/create_async_client (and thus get_client/get_async_client), including docstring updates.
  • Added unit and integration tests to validate initialization-time header presence and request-level override behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
clickhouse_connect/driver/httpclient.py Adds a headers constructor arg and merges client-level headers into the default header set.
clickhouse_connect/driver/asyncclient.py Adds a headers constructor arg and merges client-level headers into the default header set for async requests.
clickhouse_connect/driver/__init__.py Exposes headers in sync/async client factory APIs and documents intended precedence.
tests/unit_tests/test_driver/test_httpclient.py Adds unit tests for initialization-time headers and request-vs-client header precedence (sync/async).
tests/integration_tests/test_client.py Adds an integration test verifying client-level headers are stored and normal commands still work.

Comment thread clickhouse_connect/driver/__init__.py
Comment thread clickhouse_connect/driver/__init__.py

@joe-clickhouse joe-clickhouse left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @river-walras thanks for the contribution and thanks for addressing the comments so far. I have two things on my end:

  1. Sync ping() currently bypasses the header merge in _raw_request() because it calls self.http.request(...) directly in httpclient.py. Please build the ping headers from the client headers, e.g. headers=dict_copy(self.headers) so that ping has them as well. Note the async ping() already sends them through the session defaults so we're good there. This will restore parity. A unit test for sync ping with client headers would be good to add.
  2. I left a note in the code about Accept-Encoding in the docstrings.

Otherwise we're about there!

Comment thread clickhouse_connect/driver/__init__.py Outdated
Comment thread clickhouse_connect/driver/__init__.py Outdated

@joe-clickhouse joe-clickhouse left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks @river-walras!

@joe-clickhouse joe-clickhouse merged commit 6e54147 into ClickHouse:main Jun 8, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants