Skip to content

auto-promote large query parameters to form body#789

Merged
joe-clickhouse merged 3 commits into
mainfrom
joe/740-query-parameters-sent-in-url-by-default-causes-http-414-with-large-in-lists-vector-embeddings
Jun 10, 2026
Merged

auto-promote large query parameters to form body#789
joe-clickhouse merged 3 commits into
mainfrom
joe/740-query-parameters-sent-in-url-by-default-causes-http-414-with-large-in-lists-vector-embeddings

Conversation

@joe-clickhouse

@joe-clickhouse joe-clickhouse commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Large query parameter payloads are now sent as form data in the request body instead of the URL query string. This avoids HTTP 414 errors from proxies such as nginx, AWS ALB, and CloudFront when a query carries a large IN list or a high-dimensional vector embedding.
  • The client switches to the body automatically once the encoded parameter length passes a threshold. Small parameters still go in the URL as before, so existing behavior is unchanged for typical queries.
  • Setting form_encode_query_params=True still forces form encoding for all queries.
  • Queries that use binary parameter binds stay on the existing path and are never auto-promoted unless form encoding is forced.
  • This does not change the server per-value size limit, which is governed by http_max_field_value_size and applies to both URL and form transports.
  • Applies to both the sync and async clients.

Closes #740

Checklist

Delete items not relevant to your PR:

  • Unit and integration tests covering the common scenarios were added
  • A human-readable description of the changes was provided to include in CHANGELOG

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the HTTP transport behavior so that very large server-side bind parameters are automatically sent as multipart form fields in the POST body (instead of being URL-encoded into the request URI), reducing the likelihood of HTTP 414 failures from common proxies and load balancers.

Changes:

  • Added a shared use_form_encoding decision helper (with a URL-length budget) to determine when to auto-promote bind params to form-data.
  • Updated both sync (HttpClient) and async clients to use the helper so large bind payloads go through the body while keeping existing behavior for typical queries.
  • Added unit and integration tests for auto-promotion behavior and updated public docstrings / changelog messaging.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
clickhouse_connect/driver/binding.py Introduces the URL-length threshold and use_form_encoding helper used to decide when to route params via form-data.
clickhouse_connect/driver/httpclient.py Switches query/raw_query request construction to use auto form-encoding when bind params are large.
clickhouse_connect/driver/asyncclient.py Mirrors the sync client logic for auto form-encoding in async query and raw_query paths.
clickhouse_connect/driver/__init__.py Updates create_client / create_async_client docstrings to describe the new auto-promotion behavior.
tests/unit_tests/test_driver/test_httpclient.py Adds unit coverage asserting that large bind params are routed into multipart fields even when the flag is off.
tests/unit_tests/test_driver/test_binding.py Adds direct unit coverage for the new use_form_encoding helper (including binary-query behavior).
tests/integration_tests/test_form_encode_query.py Adds integration coverage ensuring large-parameter queries succeed on a default client (sync/async via shared fixtures).
CHANGELOG.md Documents the user-visible behavior change and links it to #740.

Comment thread clickhouse_connect/driver/__init__.py Outdated
Comment thread clickhouse_connect/driver/__init__.py Outdated
Comment thread CHANGELOG.md Outdated
Comment thread clickhouse_connect/driver/binding.py Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 1 comment.

Comment thread clickhouse_connect/driver/binding.py Outdated
Comment thread clickhouse_connect/driver/binding.py
@joe-clickhouse joe-clickhouse merged commit 4976348 into main Jun 10, 2026
28 checks passed
@joe-clickhouse joe-clickhouse deleted the joe/740-query-parameters-sent-in-url-by-default-causes-http-414-with-large-in-lists-vector-embeddings branch June 10, 2026 05:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Query parameters sent in URL by default causes HTTP 414 with large IN-lists / vector embeddings

3 participants