Skip to content

feat(v9.1.0)!: streaming SSOT + fluent contract-first API + typed control variants + FLATFILES SDK parity#514

Closed
userFRM wants to merge 1 commit into
mainfrom
fix/h1-streaming-ssot
Closed

feat(v9.1.0)!: streaming SSOT + fluent contract-first API + typed control variants + FLATFILES SDK parity#514
userFRM wants to merge 1 commit into
mainfrom
fix/h1-streaming-ssot

Conversation

@userFRM
Copy link
Copy Markdown
Owner

@userFRM userFRM commented May 7, 2026

Summary

v9.1.0 single-queue SSOT for the FPSS streaming pipeline, polymorphic contract-first subscription API, typed FpssControl::* variants on every binding, hand-written FLATFILES surface across Python / TypeScript / C++, dynamic-schema Arrow conversion, and resilience hardening across the FPSS lifecycle. The Go SDK is dropped; Python, TypeScript, C, and C++ are first-class.

Streaming pipeline

  • One queue. The user callback runs as the LMAX Disruptor consumer's handle_events_with closure under std::panic::catch_unwind. The legacy dispatcher shim and crossbeam-channel runtime dep are gone.
  • Pull-iter deliverystart_streaming_iter() (Rust / Python / C++), streaming_iter() Python context manager, for await async iterator (TypeScript), TdxFpssEventIterator (C / C++).
  • Drain barrierawait_drain(timeout) / tdx_*_await_drain quiescence; tdx_*_free polls it (5 s) before destroying the handle.
  • Stop / restart — generation token closes stop_streaming() resurrection race; multi-generation drain via prev_drained: Mutex<Vec<Arc<AtomicBool>>>.
  • Reconnecttdx_unified_reconnect and tdx_fpss_reconnect block on the previous-generation drain flag (5 s budget) before binding the new session to the same C callback / ctx.
  • Self-join detachFpssClient::Drop detects the consumer-thread self-join case via OnceLock<ThreadId> and detaches io_handle.join() onto a helper thread.
  • Config knobsFpssConfig exposes ring_size, flush_mode, tcp_nodelay, tcp_keepalive, connect_timeout, read_timeout on the client builder.

API fluency

  • One polymorphic subscribe(spec) / unsubscribe(spec) method on the unified ThetaDataDxClient. Build a typed spec via quote() / trade() / open_interest() on Contract, or full_trades() / full_open_interest() on SecType. subscribe_many / unsubscribe_many for bulk.
  • Contract::stock("AAPL"), Contract::index("SPX"), Contract::option(symbol, expiration, strike, right). All strings positional in C++ / Rust; keyword-only in Python.
  • Event-carried Contract is fluent — event.contract exposes right, strike_dollars, sec_type (symbolic name), and per-binding-idiomatic shapes for the option side. C++ inline helpers tdx::strike_dollars, tdx::right, tdx::sec_type_name, tdx::reason_name mirror the Python / TS fluent fields.

Typed control variants

Every FpssControl::* Rust variant has a dedicated typed surface:

  • Python: one pyclass per variant (LoginSuccess, ContractAssigned, Disconnected, Reconnecting, ...). Branch on event.kind (snake_case), read the variant's typed payload directly. Disconnected.reason_name / Reconnecting.reason_name surface the RemoveReason enum name.
  • TypeScript: one #[napi(object)] struct per variant; event.disconnected.reasonName mirrors the Python surface.
  • C / C++: one typedef struct { ... } TdxFpss<Variant>; per variant. Dispatch on event->kind (TdxFpssEventKind enum), read the matching event-><variant> payload.

FpssControl::ContractAssigned keeps its diagnostic role; the resolved typed Arc<Contract> is carried directly on every data event, so user code never has to thread a contract-id side table.

FLATFILES SDK parity

Hand-written FLATFILES bindings on Python, TypeScript, and C++ — every endpoint the Rust crate exposes is reachable. Dynamic per-(sec_type, req_type) schema converted to Arrow under the arrow feature; Python returns pyarrow.Table, TypeScript returns Arrow IPC bytes, C++ returns TdxFlatFileRowList with to_arrow_ipc(). MCP server ships tdx_flatfile_* tool surfaces matching the SDK shape.

Cross-language utility parity

condition_name, exchange_name, sequence_signed_to_unsigned, sequence_unsigned_to_signed exposed across Python, TypeScript, and C++ — same names, same return types. TypeScript helpers reject out-of-range BigInt inputs at the i32 / u32 wire-range boundary instead of silently coercing.

Encoding crate (tdbe)

  • mdds::decode::v3 renamed to mdds::decode::dual_type_columns. Schema bumped to v5.
  • Error enum folded Decode / Decompress / Config / Grpc payloads into typed kinds; ~120 callsites migrated.
  • RemoveReason::from_code(i16) -> Self and RemoveReason::as_str(&self) -> &'static str for the symbolic-name accessors used by Python / TS bindings.
  • Canonical Gregorian calendar validator across MDDS, FPSS, and flat-files.

Repository / CI

  • Root tree trimmed; ADRs inlined into Rust doc comments; generated SDK files moved under _generated/ subdirectories.
  • TypeScript npm test runs all 6 test files (19 tests pass) on every advertised platform.
  • Stale RawData / Empty event variants hidden from the public surface; contract_id integer removed from all data events.
  • Observability path surfaces SystemTime and JSON-serialization failures instead of swallowing them.

Breaking changes since v9.0.x

Surface Before After
Client type name ThetaDataDx ThetaDataDxClient
Subscribe API client.subscribe_quotes(symbol) / subscribe_full_trades(sec_type) and the per-kind family client.subscribe(Contract::stock("AAPL").quote()) / client.subscribe(SecType::Option.full_trades())
Option builder Contract::option(symbol, occ_string) (2-arg) Contract::option(symbol, expiration, strike, right) (4-arg, all strings)
C ABI Contract option side bool has_is_call; bool is_call; bool has_right; char right; (ASCII 'C' / 'P')
Python / TS event-Contract sec_type: int, is_call: Optional[bool] sec_type: str, right: Optional[str], plus strike_dollars
Control envelope flat TdxFpssControl { kind, id, detail } per-variant typed structs (TdxFpssLoginSuccess, TdxFpssDisconnected, ...)
Data event contract id contract_id: i32 plus side-table lookup typed Arc<Contract> directly on the event
Hidden internals FpssData::RawData / FpssData::Empty public hidden from user surface
Go SDK shipped removed end-to-end

cargo semver-checks --baseline-rev v9.0.0 reports the breaking set; the release lands as a 9.x minor on the v9 line.

Closes

Test plan

  • cargo fmt --all -- --check
  • cargo clippy --workspace --locked -- -D warnings
  • cargo test --workspace --locked
  • cargo deny check all
  • cargo build --release -p thetadatadx-ffi --locked
  • C++ CMake build (cpp library, examples, validate, fpss_smoke, flatfiles)
  • TypeScript npm run build + npm test (19/19 pass)
  • Python cargo check + ABI3 smoke on macOS / Linux / Windows × py3.9, py3.14
  • python3 scripts/check_docs_consistency.py exits 0
  • cargo run -p thetadatadx --bin generate_sdk_surfaces -- --check reports no drift
  • cargo semver-checks --baseline-rev v9.0.0 — fails with the documented intentional-break set, not blocking
  • Live FPSS smoke against production (run by maintainer with creds before tag)

@userFRM userFRM changed the title fix(fpss): single-queue SSOT for streaming pipeline (closes #513) fix(fpss): single-queue SSOT for streaming pipeline + lifecycle barrier (closes #513) May 7, 2026
@userFRM userFRM changed the title fix(fpss): single-queue SSOT for streaming pipeline + lifecycle barrier (closes #513) feat(v9.1.0)!: streaming SSOT + flatfiles SDK parity + typed control variants + resilience hardening May 7, 2026
@userFRM userFRM requested a review from Copilot May 8, 2026 07:15
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of lines (20,000). Try reducing the number of changed lines and requesting a review from Copilot again.

userFRM added a commit that referenced this pull request May 8, 2026
External audit closed-out:

HIGH-001 — multi-generation drain barrier. The single-slot
`prev_drained: Mutex<Option<Arc<AtomicBool>>>` overwrote on every
retire, so a stacked stop/start/stop where the earlier session was
still draining when the later one retired silently lost the earlier
flag. `await_drain()` then returned `true` based on the latest
generation while the earlier callback could still fire on a freed
FFI `ctx` — a use-after-free hazard under reconnect-storm scenarios.
Replaced with `Mutex<Vec<Arc<AtomicBool>>>` on both `ThetaDataDx`
and `TdxFpssHandle`; every retired session's flag is pushed onto
the Vec, and `await_drain()` / `tdx_*_free` walk the full set with
lazy GC. Soak coverage at
`streaming_soak_tests::multi_gen_drain_waits_for_all_retired_sessions`
drives three flags through the production poll cadence with a
staggered drain order so the pre-fix code path is a hard regression
gate.

MED-001 — WS payload now carries `unresolved_contract_id` for
pre-`ContractAssigned` ticks. The decoder builds an unresolved-
contract sentinel whose `symbol` is `__pending:<id>` (the canonical
`sec_type == SecType::Unknown` check still gates consumer code
paths); the WS formatter detects the prefix, emits
`contract: {"status": "pending"}`, and surfaces the parsed wire id
as a top-level integer. Public SDK callback signature unchanged.

LOW-001 — WS `/subscribe` option path now runs the canonical
Gregorian validator alongside the bounds check. Impossible dates
like `20260230` (Feb 30) or `20260431` (Apr 31) no longer leak
through.

REPO-MED-001 — documented the explicit-handoff contract on Python
and TypeScript `stop_streaming`, `shutdown`, and `reconnect`.
Sourced from the codegen surface (`sdk_surface.toml` +
`build_support/sdk_surface/{python,typescript}.rs`) so the
generated `streaming_methods.rs` files stay in sync.

CHANGELOG, `.github/release-notes/v9.1.0.md`, and
`docs-site/docs/changelog.md` updated. Codegen `--check` clean,
banned-vocab grep zero matches, full workspace tests green.
@userFRM userFRM force-pushed the fix/h1-streaming-ssot branch from 316b16c to 1b51a36 Compare May 8, 2026 13:37
@userFRM userFRM changed the title feat(v9.1.0)!: streaming SSOT + flatfiles SDK parity + typed control variants + resilience hardening feat(v9.1.0)!: streaming SSOT + fluent contract-first API + typed control variants + FLATFILES SDK parity May 8, 2026
userFRM added a commit that referenced this pull request May 8, 2026
…, TS rename completeness, doc drift

Four findings from the PR-wide audit on PR #514, all closed:

1. **FFI reconnect race (High)** — both `tdx_unified_reconnect` and
   `tdx_fpss_reconnect` previously stopped the old session and
   immediately started a new session bound to the same C callback /
   `ctx`, without waiting for the prior consumer thread to finish
   firing the old callback. The C ABI documents single-threaded
   callback invocation; without an intervening drain the old consumer
   could still be inside the user's `ctx` when the new consumer
   started firing on a different thread, breaking the "no internal
   locks needed" guarantee. Both reconnect paths now block on the
   previous generation's drain flag (5 s budget, matching the FFI
   free-path) before opening the new session, and surface a clear
   error on timeout.

2. **TypeScript rename incomplete (High)** — the prior commit caught
   the obvious sites but missed
   `sdks/typescript/scripts/emit_validator_manifest.mjs` (line 28,
   134) and four `__tests__/*.test.mjs` files that hardcoded
   `mod.ThetaDataClient`. The validator manifest emitter now succeeds
   (`emitted 8 records`); `npm test` runs 8 tests / 8 pass. Repo URLs
   in `package.json` and the platform npm packages were also stale
   (`userFRM/ThetaDataClient` instead of `userFRM/ThetaDataDx`) —
   fixed.

3. **TypeScript `Contract` alias (Medium)** — napi-rs exposes the
   fluent contract type as `ContractRef` because `Contract` already
   names the FPSS event-payload data class. The CommonJS entry
   (`streaming-session.js`) and its `.d.ts` companion now ship the
   `Contract = ContractRef` alias the quickstart documents, so
   `import { Contract } from 'thetadatadx'` works end-to-end.

4. **Doc drift (Medium)** — purged stale public-surface references:
   - `docs-site/docs/streaming/index.md` and `events.md`: dropped
     `RawData` (hidden from public surface; replaced by
     `FpssControl::UnknownFrame` typed control variant). The
     "ring-reader thread" callback-thread description on Python is
     now "LMAX Disruptor consumer thread" (the actual runtime).
   - `docs-site/docs/api-reference.md`: removed
     `subscribe_option_*` / `unsubscribe_option_*` row, removed
     stale `contract_map` / `contract_lookup` rows; replaced with
     the polymorphic `subscribe(spec)` / `unsubscribe(spec)` row.
     Stripped the residual `Wave H2` history vocabulary from the
     code-comment example.
   - `scripts/check_docs_consistency.py` gained guards that fail the
     build if `FpssEvent::RawData`, `RawData (undecoded fallback)`,
     `ring-reader thread`, or any `subscribe_*`-family token regress
     into any `docs-site/docs/streaming/*.md`.

Pre-push pipeline: cargo fmt --all -- --check, clippy --workspace
--locked -- -D warnings, test --workspace --locked, deny check all,
docs consistency, generate_sdk_surfaces --check, npm run build, npm
test (8/8 pass), C++ CMake all green. cargo semver-checks fails on
the documented intentional breaks only.
userFRM added a commit that referenced this pull request May 8, 2026
…ft, soak scripts, dead-code escapes

Four findings from the r11 PR-wide audit on PR #514, all closed.

1. **TS Util sequence helpers (High)** — `sequenceSignedToUnsigned`
   took raw `i64`, which napi-rs rejects when the JS caller passes a
   `BigInt`. The own example in the rustdoc passed a BigInt, so the
   helper failed at first use. Both directions now use `BigInt`
   end-to-end (i64 wire range exceeds `Number.MAX_SAFE_INTEGER`):
   - `sequenceSignedToUnsigned(BigInt)` → `BigInt`
   - `sequenceUnsignedToSigned(BigInt)` → `BigInt` (was returning
     plain `Number`).
   Plus the `npm test` script was running only 3 of 6 test files;
   `util_helpers.test.mjs`, `iter_mode.test.mjs`, and
   `fpss_control_variants.test.mjs` are now wired in. 18 tests pass.

2. **Doc drift (Medium)** — purged the residual stale public-surface
   references the prior commit missed:
   - `ffi/README.md`: removed `tdx_unified_start_streaming`,
     `*_contract_lookup` rows; added the actual current surface
     (`tdx_unified_set_callback`, `tdx_unified_await_drain`,
     `tdx_unified_reconnect`, `tdx_fpss_set_callback`,
     `tdx_fpss_await_drain`, `tdx_fpss_dropped_events`,
     `tdx_fpss_reconnect`).
   - `docs-site/docs/getting-started/streaming.md`: dropped the Go
     SDK row (Wave H removed Go); replaced "ring-reader thread"
     with "LMAX Disruptor consumer thread".
   - `docs/architecture.md`: dropped `client.contract_map()` from
     the Python surface description; replaced `tdx_fpss_subscribe_*`
     /  `tdx_fpss_contract_map` enumeration with the real C-FFI
     surface; replaced the `Quote / Trade / ... / Simple / RawData`
     pyclass list with the typed-pyclass-per-FpssControl-variant
     enumeration.
   - `docs/api-reference.md`: replaced the `quote/trade/.../control/raw_data`
     event-variant phrasing with the per-variant typed structs (one
     per `FpssControl::*` Rust variant including `unknown_frame`).

3. **FPSS smoke + soak scripts (Medium)** — both
   `scripts/fpss_smoke.py` and `scripts/fpss_soak.py` were still
   wired to removed APIs (`start_streaming()` no-arg,
   `subscribe_quotes` / `subscribe_trades` / `subscribe_option_quotes`,
   `client.contract_lookup()`, `client.contract_map()`,
   `client.next_event(timeout_ms=...)`, `ThetaDataDx`,
   `client.shutdown`). Both rewritten against the modern API:
   `ThetaDataDxClient` + push-callback delivery into a thread-safe
   queue + polymorphic `subscribe(Contract.stock(...).quote())` /
   `Contract.option(symbol, expiration=, strike=, right=)` builder
   + `await_drain` on shutdown. The reconnect-and-restore-subscriptions
   semantics carries over with the same assertion shape.

4. **#[allow(dead_code)] escape hatches (Low)** — both reintroduced
   sites cleaned up:
   - `sdks/typescript/src/flatfile_methods.rs`: dropped the unused
     `Either` import and the `_UnusedEither` type alias that was
     keeping it alive. The flatfile surface does not need `Either`,
     and the parity-with-other-generated-methods comment is moot
     once the dead alias is gone.
   - `crates/thetadatadx/tests/callback_watchdog.rs`: replaced the
     dead-named `signature_witness` helper with an in-test closure.
     The compile-time signature check still fires (renaming
     `set_slow_callback_threshold` or `slow_callback_count` fails
     this test's build), but no `dead_code` allow remains.

Pre-push pipeline: cargo fmt --all -- --check, clippy --workspace
--locked -- -D warnings, test --workspace --locked, deny check all,
docs consistency, generate_sdk_surfaces --check, npm run build, npm
test (18/18 pass), C++ CMake all green.
userFRM added a commit that referenced this pull request May 8, 2026
…ion, more doc drift, ROADMAP wave vocabulary

Three findings from the r12 PR-wide audit on PR #514, all closed.

1. **TS sequence helpers reject out-of-range BigInt (Medium)** — the
   prior pass relied on `BigInt::get_u64()` which conflates negative
   inputs with truly out-of-range bigints, so
   `sequenceUnsignedToSigned(-1n)` silently returned `1n` instead of
   throwing, and `sequenceSignedToUnsigned(2^63)` saturated to
   `i64::MAX` instead of throwing. Replaced with a hand-written
   `bigint_to_i32` helper that walks the napi `(sign_bit, words)`
   representation directly. The helpers now:
   - clamp the signed input to the i32 wire range
     (`-2_147_483_648..=2_147_483_647`) — this is the actual
     terminal-protocol wire range; the SDK widens to i64 internally
     but the round-trip is only meaningful in i32 (per
     `crates/tdbe/src/sequences.rs:8-14`);
   - clamp the unsigned input to the unsigned wire range (`0..=2^32 - 1`);
   - throw on negative BigInt to `sequenceUnsignedToSigned`;
   - accept the asymmetric `i32::MIN` boundary correctly.
   Test suite gained range / rejection coverage; `npm test` runs
   19 / 19.

2. **Doc drift remainder (Medium)** — purged the stale public-surface
   references the prior pass missed:
   - `docs/api-reference.md`: TypeScript section described streaming
     as `tdx.startStreaming()` / `tdx.nextEvent()`. Both removed; the
     section now points to push-callback (`tdx.startStreaming(callback)`)
     and pull-iter (`for await (const event of await
     tdx.startStreamingIter())`). The TypeScript code example was
     fully rewritten against the modern API
     (`Contract.stock(...).quote()`, `await client.startStreamingIter()`,
     `awaitDrain` on shutdown). The contract-fields table dropped
     the Go-only column.
   - `docs-site/docs/streaming/connection.md`: replaced the
     `nextEvent()` row in the async/sync table with the actual
     `EventIterator.next(timeoutMs)` shape.
   - `docs-site/docs/getting-started/streaming.md`: the architecture
     paragraph dropped Go from the "polling queue" enumeration
     (Wave H removed Go end-to-end).

3. **ROADMAP.md vocabulary scrub (Low)** — `docs/ROADMAP.md` is a
   user-facing doc page and still leaked Wave / issue-tracker
   vocabulary (`Wave O`, `#431` / `#432` / `#433` / `#436` / `#438`,
   `#424`). The "Tracking" column on the binding-status table and
   the per-row `(Wave O)` annotations on the coverage matrix are
   gone; the open-work paragraph now says "Shipped" without the
   issue-number reference.

Pre-push pipeline: cargo fmt --all -- --check, clippy --workspace
--locked -- -D warnings, test --workspace --locked, deny check all,
docs consistency, generate_sdk_surfaces --check, npm run build, npm
test (19/19 pass), C++ CMake all green.
…trol variants + FLATFILES SDK parity

Single-queue SSOT for the FPSS streaming pipeline, polymorphic
contract-first subscription API, typed `FpssControl::*` variants on
every binding, hand-written FLATFILES surface across Python /
TypeScript / C++, dynamic-schema Arrow conversion, and resilience
hardening across the FPSS lifecycle. The Go SDK is dropped; Python,
TypeScript, C, and C++ are first-class.

This is a v9.1.0 minor release with breaking API changes; the v9.0
line had no external consumers, so the release rolls forward as a
9.x minor instead of bumping to 10.

# Streaming pipeline

- One queue. The user callback runs as the LMAX Disruptor consumer's
  `handle_events_with` closure under `std::panic::catch_unwind`. The
  legacy dispatcher shim and `crossbeam-channel` runtime dep are gone.
- **Pull-iter delivery** — `start_streaming_iter()` (Rust / Python /
  C++), `streaming_iter()` Python context manager, `for await` async
  iterator (TypeScript), `TdxFpssEventIterator` (C / C++). Mutually
  exclusive with the push-callback path on the same client; the
  iterator drains the SPSC queue from the user thread.
- **Drain barrier** — `await_drain(timeout)` /
  `tdx_*_await_drain` quiescence; `tdx_*_free` polls it (5 s) before
  destroying the handle, closing the `ctx` use-after-free window.
- **Stop / restart** — `stop_streaming()` resurrection race closed
  with a generation token; multi-generation drain via
  `prev_drained: Mutex<Vec<Arc<AtomicBool>>>` walks every prior
  generation's flag.
- **Reconnect** — `tdx_unified_reconnect` and `tdx_fpss_reconnect`
  block on the previous-generation drain flag (5 s budget) before
  binding the new session to the same C callback / `ctx`. The
  documented "single-thread callback" contract is now actually
  enforced by the FFI.
- **Self-join detach** — `FpssClient::Drop` detects the
  consumer-thread self-join case via `OnceLock<ThreadId>` and
  detaches `io_handle.join()` onto a helper thread.
- **Config knobs** — `FpssConfig` exposes `ring_size`, `flush_mode`,
  `tcp_nodelay`, `tcp_keepalive`, `connect_timeout`, `read_timeout`
  on the client builder. TypeScript shape-manifest agreement gap
  closed.

# API fluency on every binding

- **Subscriptions** — one polymorphic `subscribe(spec)` /
  `unsubscribe(spec)` method on the unified `ThetaDataDxClient`. Build
  a typed spec via `quote()` / `trade()` / `open_interest()` on
  `Contract`, or `full_trades()` / `full_open_interest()` on
  `SecType`. `subscribe_many` / `unsubscribe_many` for bulk. The
  legacy per-kind family is gone.
- **Contract builder** — `Contract::stock("AAPL")`,
  `Contract::index("SPX")`, `Contract::option(symbol, expiration,
  strike, right)` (all strings positional in C++ / Rust; keyword-only
  in Python). All four arguments required.
- **Event-carried Contract is fluent** across languages — `event.contract`
  exposes:
  - **Rust**: `Contract { symbol, sec_type: SecType, expiration:
    Option<i32>, is_call: Option<bool>, strike: Option<i32> }` plus
    `right() -> Option<Right>` (typed `Call` / `Put`) and
    `strike_dollars() -> Option<f64>`.
  - **Python**: `symbol: str`, `sec_type: str` (`"STOCK"` /
    `"OPTION"` / `"INDEX"` / `"RATE"`), `expiration: Optional[int]`,
    `right: Optional[str]` (`"C"` / `"P"`), `strike_dollars:
    Optional[float]`, `strike: Optional[int]` (wire form).
  - **TypeScript**: same field set, camelCase (`secType`,
    `strikeDollars`).
  - **C / C++**: `TdxContract { symbol; sec_type; has_expiration;
    expiration; has_right; right; has_strike; strike; }`. ASCII
    `'C'` / `'P'` for `right`; integer wire form for `strike`. C++
    inline helpers `tdx::strike_dollars`, `tdx::right`,
    `tdx::sec_type_name`, `tdx::reason_name` mirror the Python / TS
    fluent fields.
- **Hard rename** — `ThetaDataDx` → `ThetaDataDxClient`. No alias.

# Typed control variants

Every `FpssControl::*` Rust variant has a dedicated typed surface:

- **Python**: one pyclass per variant (`LoginSuccess`,
  `ContractAssigned`, `Disconnected`, `Reconnecting`, `Reconnected`,
  `MarketOpen`, `MarketClose`, `ServerError`, `Error`,
  `UnknownFrame`, `UnknownControl`, `Connected`, `Ping`,
  `ReconnectedServer`, `Restart`, `ReqResponse`). Branch on
  `event.kind` (snake_case), read the variant's typed payload
  directly. `Disconnected.reason_name` / `Reconnecting.reason_name`
  surface the `RemoveReason` enum name.
- **TypeScript**: one `#[napi(object)]` struct per variant;
  `event.disconnected.reasonName` mirrors the Python surface.
- **C / C++**: one `typedef struct { ... } TdxFpss<Variant>;` per
  variant. Dispatch on `event->kind` (`TdxFpssEventKind` enum), read
  the matching `event-><variant>` payload.

`FpssControl::ContractAssigned` keeps its diagnostic role; the
resolved typed `Arc<Contract>` is carried directly on every data
event, so user code never has to thread a contract-id side table.

# FLATFILES SDK parity

Hand-written FLATFILES bindings on Python, TypeScript, and C++ —
every endpoint the Rust crate exposes is reachable. Dynamic
per-(sec_type, req_type) schema converted to Arrow under the
`arrow` feature; Python returns `pyarrow.Table`, TypeScript returns
Arrow IPC bytes, C++ returns `TdxFlatFileRowList` with
`to_arrow_ipc()`. MCP server ships `tdx_flatfile_*` tool surfaces
matching the SDK shape.

# Cross-language utility parity

`condition_name(code) -> str`, `exchange_name(code) -> str`,
`sequence_signed_to_unsigned` / `sequence_unsigned_to_signed`
exposed across Python, TypeScript, and C++ — same names, same
return types. TypeScript helpers reject out-of-range BigInt inputs
at the i32 / u32 wire-range boundary instead of silently coercing.

# Encoding crate (`tdbe`)

- `mdds::decode::v3` renamed to `mdds::decode::dual_type_columns` to
  reflect what the module decodes, not which schema version it was
  written for. Schema bumped to v5.
- `Error` enum folded `Decode` / `Decompress` / `Config` / `Grpc`
  payloads into typed kinds; ~120 callsites migrated.
- `RemoveReason::from_code(i16) -> Self` and
  `RemoveReason::as_str(&self) -> &'static str` for the symbolic
  name accessors used by the Python / TS bindings.
- Canonical Gregorian calendar validator across MDDS, FPSS, and
  flat-files.

# Repository / CI

- Root tree trimmed; ADRs inlined into Rust doc comments; generated
  SDK files moved under `_generated/` subdirectories.
- TypeScript `npm test` runs all 6 test files (19 tests pass) on
  every advertised platform (macOS, Linux, Windows × py3.9 + py3.14).
- Stale `RawData` / `Empty` event variants hidden from the public
  surface; `contract_id` integer removed from all data events in
  favour of the typed `Arc<Contract>`.
- Observability path surfaces `SystemTime` and JSON-serialization
  failures instead of swallowing them.

# Breaking changes since v9.0.x

| Surface | Before | After |
|---|---|---|
| Client type name | `ThetaDataDx` | `ThetaDataDxClient` |
| Subscribe API | `client.subscribe_quotes(symbol)` / `subscribe_full_trades(sec_type)` and the per-kind family | `client.subscribe(Contract::stock("AAPL").quote())` / `client.subscribe(SecType::Option.full_trades())` |
| Option builder | `Contract::option(symbol, occ_string)` (2-arg) | `Contract::option(symbol, expiration, strike, right)` (4-arg, all strings) |
| C ABI Contract option side | `bool has_is_call; bool is_call;` | `bool has_right; char right;` (ASCII `'C'` / `'P'`) |
| Python / TS event-Contract | `sec_type: int`, `is_call: Optional[bool]` | `sec_type: str`, `right: Optional[str]`, plus `strike_dollars` |
| Control envelope | flat `TdxFpssControl { kind, id, detail }` | per-variant typed structs (`TdxFpssLoginSuccess`, `TdxFpssDisconnected`, …) |
| Data event contract id | `contract_id: i32` plus side-table lookup | typed `Arc<Contract>` directly on the event |
| Hidden internals | `FpssData::RawData` / `FpssData::Empty` public | hidden from user surface |
| Go SDK | shipped | removed end-to-end |

`cargo semver-checks --baseline-rev v9.0.0` reports the breaking
set; the release lands as a 9.x minor on the v9 line.

# Closes

- #513 streaming SSOT
- #424 cross-language utility parity
- #431, #432, #433, #441, #442, #446 FLATFILES ecosystem completion
- #471 tdbe condition flag (held for v9.2.0)
- #512 v3.rs rename
@userFRM userFRM force-pushed the fix/h1-streaming-ssot branch from 85f7d4b to 50b5213 Compare May 9, 2026 06:13
@userFRM
Copy link
Copy Markdown
Owner Author

userFRM commented May 9, 2026

Superseded by clean-history PR (single squash commit, no orphan references). Same branch fix/h1-streaming-ssot reopened in new PR.

@userFRM userFRM closed this May 9, 2026
@userFRM userFRM deleted the fix/h1-streaming-ssot branch May 9, 2026 07:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants