Skip to content

decoder: raise default max depth to 1500 and add DecodeUnlimitedDepth#32

Merged
urvisavla merged 1 commit into
masterfrom
decoder-unlimited-depth
May 29, 2026
Merged

decoder: raise default max depth to 1500 and add DecodeUnlimitedDepth#32
urvisavla merged 1 commit into
masterfrom
decoder-unlimited-depth

Conversation

@urvisavla

Copy link
Copy Markdown

What

  • Raise DecodeDefaultMaxDepth from 250 → 1500. This is the bound applied to untrusted, user-supplied XDR (anyone using NewDecoder/Unmarshal or MaxDepth: 0).
  • Add a new sentinel DecodeUnlimitedDepth = uint(math.MaxUint). Callers decoding trusted XDR (e.g. emitted by stellar-core) can set DecodeOptions{MaxDepth: xdr3.DecodeUnlimitedDepth} to disable the depth limit.

Why

The previous default of 250 was too shallow for some legitimately deep XDR. User-supplied input still needs a guard against unbounded recursion / stack growth, so the default is raised to 1500 rather than removed. Trusted core output, which is acyclic and finite but can nest deeply, gets an explicit opt-out.

Usage

// User-supplied / untrusted XDR — capped at 1500
xdr3.Unmarshal(r, &v)

// Trusted XDR emitted by stellar-core — no depth limit
xdr3.UnmarshalWithOptions(r, &v, xdr3.DecodeOptions{
    MaxDepth: xdr3.DecodeUnlimitedDepth,
})

Implementation notes

  • 0 already means "use the default", so a separate sentinel was needed for "unlimited". math.MaxUint reuses the existing countdown logic untouched — no hot-path change — and is effectively unbounded for any real nesting.
  • Trade-off: DecodeUnlimitedDepth is not truly unbounded; a genuinely cyclic/adversarial input would no longer be caught by the depth check. That's intended, since this knob is only for trusted input (documented on the constant).

Testing

  • Added TestDecodeUnlimitedDepth: builds a chain nested 100 levels past the default, confirms the default limit rejects it (ErrMaxDecodingDepth) and DecodeUnlimitedDepth decodes it cleanly.
  • Full suite green across xdr, xdr2, xdr3.

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings May 29, 2026 17:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Raises the default XDR decoding depth limit in xdr3 from 250 to 1500 and introduces a DecodeUnlimitedDepth sentinel (math.MaxUint) for callers decoding trusted XDR (e.g., stellar-core output) where deep nesting is expected.

Changes:

  • Raise DecodeDefaultMaxDepth from 250 to 1500 and expand its doc comment.
  • Add new DecodeUnlimitedDepth constant and document it in DecodeOptions.MaxDepth.
  • Add TestDecodeUnlimitedDepth covering both the default-limit rejection and the unlimited-depth success path via a self-referential linkedNode type.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
xdr3/decode.go Bumps default max depth to 1500 and adds DecodeUnlimitedDepth sentinel with docs on MaxDepth.
xdr3/decode_test.go Adds linkedNode helper and TestDecodeUnlimitedDepth exercising default-limit failure and unlimited-depth success.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

The previous default decoding depth of 250 was too shallow for some
legitimately deep XDR. Raise the default to 1500 for untrusted,
user-supplied input, and add a DecodeUnlimitedDepth sentinel
(math.MaxUint) so callers decoding trusted XDR emitted by stellar-core
can disable the limit entirely.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@urvisavla urvisavla force-pushed the decoder-unlimited-depth branch from baf18dc to a00ebd7 Compare May 29, 2026 17:57
@urvisavla urvisavla merged commit 0bf8f49 into master May 29, 2026
8 checks passed
@urvisavla urvisavla deleted the decoder-unlimited-depth branch May 29, 2026 21:08
sisuresh added a commit to sisuresh/go that referenced this pull request Jun 3, 2026
Pulls in stellar/go-xdr#32 which lifts the default XDR decode depth from
250 → 1500 and adds the DecodeUnlimitedDepth sentinel. Necessary so the
CAP-71 SorobanDelegateSignature delegate chain (which the protocol allows
to nest arbitrarily, with only Soroban's runtime trap policing depth) can
be decoded by stellar-core LCM consumers without stalling.
sisuresh added a commit to sisuresh/stellar-horizon that referenced this pull request Jun 3, 2026
go-stellar-sdk@130456cc9b69 is regenerated from stellar-xdr@68fa1ac (post-
stellar/stellar-xdr#303 ungate of CAP_0071) with XDR_FEATURES cleared, so
the bind no longer carries the CAP-0083 STELLAR_VALUE_EMPTY_TX_SET path —
matching the p27 release scope (CAP-0071 only; CAP-0083 deferred).

The SDK bump also transitively picks up stellar/go-xdr#32, which raises
DecodeDefaultMaxDepth 250 → 1500, so the CAP-71 240-deep delegate fixture
(test-lcms/InvokeHostFunctionTests/a7a45d93c64cf3d9.xdr) now ingests
without ErrMaxDecodingDepth.

The HerderTests fixture (network_externalizes_empty-tx-set_on_missing_value)
no longer decodes under CAP-71-only XDR (StellarValueType 2 is gone) and is
removed.
sisuresh added a commit to sisuresh/stellar-horizon that referenced this pull request Jun 4, 2026
- Add load-test-ledgers-v27.xdr.zstd / load-test-fixtures-v27.xdr.zstd,
  generated via TestGenerateLedgers using the same stellar-core build CI
  pins (27.0.0-3288.7696c069d, buildtests). Fixes
  TestLoadTestLedgerBackendWithoutMerge, which looks up the fixture for
  MaxSupportedProtocolVersion (now 27).
- Core 27's apply-load command force-overrides the network passphrase to
  "Apply Load", so the load tests now select the fixture passphrase by
  protocol version.
- Restore the 240-deep CAP-71 delegate-tree LCM fixture
  (InvokeHostFunctionTests/a7a45d93c64cf3d9.xdr, still referenced by
  index.json): it was dropped while go-xdr's max decoding depth was 250,
  and ingests cleanly again with stellar/go-xdr#32 (depth 1500).
Shaptic pushed a commit to stellar/go-stellar-sdk that referenced this pull request Jun 5, 2026
* xdr: regenerate for Protocol 27 CAP-0071 + CAP-0083

Bump XDR to stellar-xdr@5187e69 (CAP-0071 address-bound/delegated Soroban
credentials + CAP-0083 STELLAR_VALUE_EMPTY_TX_SET). goxdr and Ruby xdrgen
cannot parse #ifdef, so CAP_0071/CAP_0083 gates are resolved with
'stellar-xdr xfile preprocess' (rs-stellar-xdr #503) before codegen, driven
by XDR_FEATURES. Regenerated xdr/, gxdr/, and xdr/xdr_views_generated.go.

* xdr: drop CAP-0083 from regen; only CAP-0071 ships in p27

- Bump XDR_COMMIT to stellar-xdr@68fa1ac (post-#303 which ungated CAP-0071).
- Clear XDR_FEATURES default so CAP-0083 #ifdef blocks are stripped during
  preprocess. CAP-0071 is ungated and needs no feature flag.
- Regenerate gxdr/, xdr/xdr_generated.go, xdr/xdr_views_generated.go.
- randxdr: add IsDeepNestedDelegates preset to cap recursion through CAP-71
  SorobanDelegateSignature.nestedDelegates, mirroring the existing
  IsDeepAuthorizedInvocationTree handling for subInvocations. Without it
  TestView_RandXDR_RawRoundTrip stack-overflows on unbounded random trees.

* go.mod: bump go-xdr to 0bf8f49 (raise DecodeDefaultMaxDepth to 1500)

Pulls in stellar/go-xdr#32 which lifts the default XDR decode depth from
250 → 1500 and adds the DecodeUnlimitedDepth sentinel. Necessary so the
CAP-71 SorobanDelegateSignature delegate chain (which the protocol allows
to nest arbitrarily, with only Soroban's runtime trap policing depth) can
be decoded by stellar-core LCM consumers without stalling.

* go.sum: drop stale go-xdr@a87d4d0 entry (go mod tidy)

The previous commit bumped go-xdr to 0bf8f49 in go.mod, but go.sum still
carried the old a87d4d0789c3 lines. CI's gomod.sh runs `go mod tidy` and
fails the build on the residual diff.

* ci: bump go-test timeout to 20m

The default per-test-binary timeout is 10m, which is borderline for
this repo's race-cover suite (xdr/ alone takes ~230s with -race
locally, longer on slower CI runners). CAP-71's added generated
XDR pushed the run past 10m, killing the test job with SIGTERM
before xdr/ and txnbuild/ finished.

20m gives generous headroom and matches the conservative timeout
other Stellar Go repos use for race+cover runs.

* txnbuild: cap CAP-71 nestedDelegates recursion in TestOperationCoverage

TestOperationCoverage generates 10,000 random gxdr.Operation values
which now include InvokeHostFunctionOp.auth[].credentials carrying
the new SorobanAddressCredentialsWithDelegates with the recursive
nestedDelegates field. Without bounding, the random generator
recurses unboundedly through SorobanDelegateSignature.nestedDelegates
and the test hangs past go test's default 10-minute per-binary
timeout, killing the CI test matrix.

Add IsDeepNestedDelegates preset (already defined in
randxdr/presets.go for LedgerCloseMetaPresets) — caps the tree height
at 2. With the preset, the test runs in ~8s under -race instead of
timing out.

Pairs with the -timeout=20m bump on the test workflow (which is now
just headroom — the real fix is this preset).
Shaptic pushed a commit to stellar/stellar-horizon that referenced this pull request Jun 8, 2026
* Protocol 27 (CAP-0071 + CAP-0083) ingestion support

- Pin go-stellar-sdk to the CAP-71/CAP-83 XDR build.
- Bump MaxSupportedProtocolVersion to 27.
- Implement GetLedgerRaw on fakeLedgerBackend (added to the SDK
  LedgerBackend interface).

* update core test lcm

* add GetLedgerRaw

* update

* Bump go-stellar-sdk to CAP-71-only XDR; drop CAP-83 fixture

go-stellar-sdk@130456cc9b69 is regenerated from stellar-xdr@68fa1ac (post-
stellar/stellar-xdr#303 ungate of CAP_0071) with XDR_FEATURES cleared, so
the bind no longer carries the CAP-0083 STELLAR_VALUE_EMPTY_TX_SET path —
matching the p27 release scope (CAP-0071 only; CAP-0083 deferred).

The SDK bump also transitively picks up stellar/go-xdr#32, which raises
DecodeDefaultMaxDepth 250 → 1500, so the CAP-71 240-deep delegate fixture
(test-lcms/InvokeHostFunctionTests/a7a45d93c64cf3d9.xdr) now ingests
without ErrMaxDecodingDepth.

The HerderTests fixture (network_externalizes_empty-tx-set_on_missing_value)
no longer decodes under CAP-71-only XDR (StellarValueType 2 is gone) and is
removed.

* Bump go-stellar-sdk to latest CAP-71-only build (a8d5b306)

* Bump protocol 26 core version to 27.0.0-3288.7696c069d

* Support core 27 apply-load config in TestGenerateLedgers

Core 27 reworked the apply-load configuration: the sampled load
parameters (APPLY_LOAD_INSTRUCTIONS, APPLY_LOAD_TX_SIZE_BYTES,
APPLY_LOAD_NUM_RW_ENTRIES, etc.) were removed in favor of an
APPLY_LOAD_MODE selector, and the apply-load command now force
overrides NETWORK_PASSPHRASE to "Apply Load" and runs at the core's
current ledger protocol version.

- Add testdata/apply-load-v27.cfg based on core 27's
  docs/apply-load-for-meta.cfg
- Select the default config based on the core binary's major version
- Assert ledger protocol version against the core binary's reported
  protocol version instead of the max supported protocol env var

* Add v27 load-test fixtures; restore deep-delegate LCM fixture

- Add load-test-ledgers-v27.xdr.zstd / load-test-fixtures-v27.xdr.zstd,
  generated via TestGenerateLedgers using the same stellar-core build CI
  pins (27.0.0-3288.7696c069d, buildtests). Fixes
  TestLoadTestLedgerBackendWithoutMerge, which looks up the fixture for
  MaxSupportedProtocolVersion (now 27).
- Core 27's apply-load command force-overrides the network passphrase to
  "Apply Load", so the load tests now select the fixture passphrase by
  protocol version.
- Restore the 240-deep CAP-71 delegate-tree LCM fixture
  (InvokeHostFunctionTests/a7a45d93c64cf3d9.xdr, still referenced by
  index.json): it was dropped while go-xdr's max decoding depth was 250,
  and ingests cleanly again with stellar/go-xdr#32 (depth 1500).
urvisavla added a commit to stellar/stellar-horizon that referenced this pull request Jun 10, 2026
* Protocol 27 ingestion (#186)

* Protocol 27 (CAP-0071 + CAP-0083) ingestion support

- Pin go-stellar-sdk to the CAP-71/CAP-83 XDR build.
- Bump MaxSupportedProtocolVersion to 27.
- Implement GetLedgerRaw on fakeLedgerBackend (added to the SDK
  LedgerBackend interface).

* update core test lcm

* add GetLedgerRaw

* update

* Bump go-stellar-sdk to CAP-71-only XDR; drop CAP-83 fixture

go-stellar-sdk@130456cc9b69 is regenerated from stellar-xdr@68fa1ac (post-
stellar/stellar-xdr#303 ungate of CAP_0071) with XDR_FEATURES cleared, so
the bind no longer carries the CAP-0083 STELLAR_VALUE_EMPTY_TX_SET path —
matching the p27 release scope (CAP-0071 only; CAP-0083 deferred).

The SDK bump also transitively picks up stellar/go-xdr#32, which raises
DecodeDefaultMaxDepth 250 → 1500, so the CAP-71 240-deep delegate fixture
(test-lcms/InvokeHostFunctionTests/a7a45d93c64cf3d9.xdr) now ingests
without ErrMaxDecodingDepth.

The HerderTests fixture (network_externalizes_empty-tx-set_on_missing_value)
no longer decodes under CAP-71-only XDR (StellarValueType 2 is gone) and is
removed.

* Bump go-stellar-sdk to latest CAP-71-only build (a8d5b306)

* Bump protocol 26 core version to 27.0.0-3288.7696c069d

* Support core 27 apply-load config in TestGenerateLedgers

Core 27 reworked the apply-load configuration: the sampled load
parameters (APPLY_LOAD_INSTRUCTIONS, APPLY_LOAD_TX_SIZE_BYTES,
APPLY_LOAD_NUM_RW_ENTRIES, etc.) were removed in favor of an
APPLY_LOAD_MODE selector, and the apply-load command now force
overrides NETWORK_PASSPHRASE to "Apply Load" and runs at the core's
current ledger protocol version.

- Add testdata/apply-load-v27.cfg based on core 27's
  docs/apply-load-for-meta.cfg
- Select the default config based on the core binary's major version
- Assert ledger protocol version against the core binary's reported
  protocol version instead of the max supported protocol env var

* Add v27 load-test fixtures; restore deep-delegate LCM fixture

- Add load-test-ledgers-v27.xdr.zstd / load-test-fixtures-v27.xdr.zstd,
  generated via TestGenerateLedgers using the same stellar-core build CI
  pins (27.0.0-3288.7696c069d, buildtests). Fixes
  TestLoadTestLedgerBackendWithoutMerge, which looks up the fixture for
  MaxSupportedProtocolVersion (now 27).
- Core 27's apply-load command force-overrides the network passphrase to
  "Apply Load", so the load tests now select the fixture passphrase by
  protocol version.
- Restore the 240-deep CAP-71 delegate-tree LCM fixture
  (InvokeHostFunctionTests/a7a45d93c64cf3d9.xdr, still referenced by
  index.json): it was dropped while go-xdr's max decoding depth was 250,
  and ingests cleanly again with stellar/go-xdr#32 (depth 1500).

* Bump integration tests to use Protocol 27 (#189)

* Bump go-stellar-sdk to v0.6.0 (#191)

* Bump verify-range stellar-core to Protocol 27 noble build (#192)

---------

Co-authored-by: Siddharth Suresh <siddharth@stellar.org>
Co-authored-by: urvisavla <urvi.savla@stellar.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants