Skip to content

Draft - New xdr schema#1416

Closed
Ryang-21 wants to merge 13 commits into
modernizationfrom
new-xdr-schema
Closed

Draft - New xdr schema#1416
Ryang-21 wants to merge 13 commits into
modernizationfrom
new-xdr-schema

Conversation

@Ryang-21

Copy link
Copy Markdown
Contributor

What

This PR replaces the SDK's XDR layer. Previously, XDR types were emitted as JavaScript classes by
@stellar/js-xdr, with instance methods like .switch(), .toXDR(), and getter/setter pairs over arm
payloads. They are now generated by two emitters that share a single IR (xdr/xdr.json, compiled from the
canonical .x schemas under xdr/curr/ and xdr/next/):

  • scripts/generate-new-xdr-schema.mjs emits src/base/generated/schema/*.ts — one file per
    Tarjan-SCC of definitions to keep imports acyclic. These are the raw wire-shape descriptors built on a new
    hand-written codec at src/base/new-xdr/ (a pure-Uint8Array reader/writer over DataView, with
    depth-limited recursion and standard XDR padding/length-prefix rules).
  • scripts/generate-new-xdr-dx.mjs emits src/base/generated/dx/*.ts — one file per type, wrapping
    the schema with a friendlier developer surface and a runtime descriptor that the hand-written engine in
    dx/_support.ts uses to bridge DX shape ↔ wire shape.

Note: the XDR library source (src/base/new-xdr/, the generation scripts, and the hand-written
dx/_support.ts engine) is vendored here temporarily for ease of testing. It will be extracted to its own
repository and consumed as a dependency before this lands.

The public namespace is composed in src/base/generated/index.ts:

export * from "./dx/index.js";              // xdr.AccountEntry, xdr.ScVal, ...
export * as xdrSchema from "./schema/index.js"; // raw wire shape, opt-in

Every DX module exposes the same six functions — toXDRObject/fromXDRObject, toJSON/fromJSON,
toXDR/fromXDR — and the engine takes care of the non-mechanical bridges:

  • 128- and 256-bit wide-int parts are surfaced as bigint. The engine splits/joins to the underlying
    {hi, lo} (or {hi_hi, hi_lo, lo_hi, lo_lo}) parts via shift/mask.
  • AssetCode4/AssetCode12 are surfaced as string and bridged to opaque(4)/opaque(12) via ASCII.
  • JSON byte encoding follows per-typedef overrides (ascii for asset codes, base64 for signatures, hex
    otherwise), and aliases inherit transitively.

As a result, the rest of the SDK was reworked to consume the new shape. The recurring patterns:

// values are plain objects, not class instances
new xdr.ScMapEntry({ key, val })         { key, val }
envelope.toXDR("base64")                 xdr.TransactionEnvelope.toXDR(envelope, "base64")
op.body().switch().name === "createAccount"   op.body.type === "createAccount"
body.alphaNum4().assetCode()             body.alphaNum4.assetCode
new xdr.UInt128Parts({ hi, lo })         (hi << 64n) | lo   // bare bigint

The src/base/numbers/ abstraction layer (ScInt, XdrLargeInt, Int128/Uint128/Int256/Uint256)
is gone — wide-ints are bigint at the boundary, validated by the engine — and the associated tests are
deleted alongside. SorobanDataBuilder is now immutable; setters return new builders via spread instead
of mutating in place. Public Address.toBuffer() / StrKey checksum APIs still hand back Buffer (a
Uint8Array subclass) so downstream consumers calling Buffer methods don't break.

A new comprehensive test test/unit/base/new_xdr_dx.test.ts exercises the DX layer end-to-end:
discriminated-union round-trips, canonical JSON shape (bigint as decimal strings, bytes per metadata
encoding), out-of-range and unknown-arm errors with path-labelled messages, wide-int bridging, ASCII
string bridging. Existing tests across test/unit/, test/integration/, and test/e2e/ were migrated to
the new namespace function calls and property accessors; net test diff is a deletion despite ~1k+ touched
lines, because the abstraction removal outweighs the new explicit calls.

Why

The legacy class-based API forced consumers to thread invariants through method-call chains
(scVal.vec()?.map()?.at(0)?.contractData()) where every step both narrowed the discriminator and
dereferenced a payload, and where wrong-arm access threw at a call site many frames from the actual
mistake. The new DX layer makes the discriminator a literal string (.type) and the arm payload a typed
sibling, so TypeScript narrows correctly under a normal if/switch and consumers can use narrowing
helpers once instead of writing per-site guards.

Generating from a single IR also lets us own the surface: we can override individual types (wide-ints to
bigint, asset codes to string) without forking an external codec, and the underlying engine is a
small, auditable Uint8Array reader/writer rather than a Buffer-coupled class hierarchy — which matters
for browser bundle size and for runtimes (workerd, Deno) where Buffer is a polyfill rather than a
primitive. The JSON shape is now canonical and round-trip-safe; the bigint surface drops a class hierarchy
and a runtime cast layer; the wire codec drops a third-party dependency.

@github-project-automation github-project-automation Bot moved this to Backlog (Not Ready) in DevX May 14, 2026
@Ryang-21 Ryang-21 changed the title -Draft- New xdr schema Draft - New xdr schema May 14, 2026
@Ryang-21 Ryang-21 requested a review from quietbits May 14, 2026 07:07
@quietbits quietbits removed their request for review May 26, 2026 12:28
@Ryang-21

Copy link
Copy Markdown
Contributor Author

Closing this in favor of #1422

@Ryang-21 Ryang-21 closed this May 27, 2026
@github-project-automation github-project-automation Bot moved this from Backlog (Not Ready) to Done in DevX May 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant