Skip to content

[BugFix] Allow request body cap to be set via MAX_REQUEST_BUFFER_BYTES#10

Merged
bhuvan-teja-glean merged 2 commits into
gleanwork:mainfrom
ashish-kumar-glean:ashish/configurable-max-request-buffer
Jun 11, 2026
Merged

[BugFix] Allow request body cap to be set via MAX_REQUEST_BUFFER_BYTES#10
bhuvan-teja-glean merged 2 commits into
gleanwork:mainfrom
ashish-kumar-glean:ashish/configurable-max-request-buffer

Conversation

@ashish-kumar-glean

@ashish-kumar-glean ashish-kumar-glean commented May 29, 2026

Copy link
Copy Markdown
Collaborator

Description: DynamicHttpFiltersSourceAdapter.getMaximumRequestBufferSizeInBytes() is hard-coded to 5 MB, which becomes the maxContentLength LittleProxy passes to Netty's HttpObjectAggregator. Any request body above 5 MB is rejected with HTTP 413 inside the aggregator before any filter runs.

This change reads the cap from a new MAX_REQUEST_BUFFER_BYTES env var (positive integer, bytes). When unset/invalid/non-positive, it falls back to the existing 5 MB default — so existing deployments are unaffected. Customers who need a larger cap (e.g. for /api/index/v1/indexdocuments) set the env var explicitly.

Context: Ericsson on-prem proxy traffic to /api/index/v1/indexdocuments was returning 413 for ~28 MB document payloads, while the same request worked when bypassing the proxy and hitting DSE directly. Root cause was the hard-coded 5 MB cap; DSE itself does not enforce an HTTP body-size limit at the framework level.

Test plan: End-to-end repro and fix verified in sa-sandbox proxy VM (10.50.0.20) from aws-salessavvy-test test VM via TGW.

With the patched JAR deployed and MAX_REQUEST_BUFFER_BYTES=67108864 (64 MB):

Body size Before (5 MB hard-coded) After (env-set to 64 MB)
4.5 MB 400 (under cap) 400 (under cap)
5.15 MB 413, upload=0 400, upload=5403512 — body fully read
28.3 MB (Ericsson size) 413, upload=0 400, upload=29719301 — body fully read
64.4 MB n/a 413, upload=0 — new cap enforced

(The 400 in the post-fix column is the proxy's own URI-routing rejection downstream of aggregation, because the repro path bypasses the inbound NLB and doesn't carry full auth context — irrelevant to the size-cap bug; what matters is that upload now matches the file size.)

Also confirmed the patched class is what's actually running by extracting DynamicHttpFiltersSourceAdapter.class from the JAR inside the container and verifying the new symbols (MAX_REQUEST_BUFFER_BYTES, resolveMaxRequestBufferBytes, DEFAULT_MAX_REQUEST_BUFFER_BYTES) are present in the bytecode.

With the env var unset, the cap stays at 5 MB exactly as before.


Change Type

  • Flag-gated development/Internal fix
  • Bug Fix/Enhancement
  • Security or Permissions related change
  • Feature launch

Platform (Choose one if applicable)

  • AWS only change
  • GCP only change

@ashish-kumar-glean ashish-kumar-glean changed the title [BugFix] Make request body cap configurable; raise default to 64 MB [BugFix] Allow request body cap to be set via MAX_REQUEST_BUFFER_BYTES May 29, 2026
@ashish-kumar-glean ashish-kumar-glean force-pushed the ashish/configurable-max-request-buffer branch from 23f1d44 to 39a8862 Compare May 29, 2026 08:19
ashish-kumar-glean and others added 2 commits June 9, 2026 15:38
The hard-coded 5 MB cap in getMaximumRequestBufferSizeInBytes() makes
Netty's HttpObjectAggregator reject any request body over 5 MB with HTTP
413 before any filter runs. This breaks customer flows such as
/api/index/v1/indexdocuments where payloads commonly exceed 5 MB
(e.g. Ericsson sending ~28 MB documents).

Read the cap from MAX_REQUEST_BUFFER_BYTES (positive integer, bytes) and
default to 64 MB. Invalid or non-positive values log a warning and fall
back to the default.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Per review feedback, leave the default cap unchanged so existing
deployments behave identically. Customers needing larger payloads
(e.g. Ericsson on /api/index/v1/indexdocuments) set
MAX_REQUEST_BUFFER_BYTES explicitly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@ashish-kumar-glean ashish-kumar-glean force-pushed the ashish/configurable-max-request-buffer branch from 39a8862 to 31bedf8 Compare June 9, 2026 10:08
@bhuvan-teja-glean bhuvan-teja-glean merged commit e02a28b into gleanwork:main Jun 11, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants