From ed86aedf11c0be62f30d4c297deedfb8f6852fd0 Mon Sep 17 00:00:00 2001 From: Santiago Date: Fri, 12 Jun 2026 19:31:45 -0300 Subject: [PATCH] chore: consolidate complex-type spec, fleet docs, and cshell submodule MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Single consolidated PR for all in-flight umbrella-repo changes so the working tree lands clean. Supersedes the spec-only PR #11 (its commit's content is included here verbatim). Spec (identical to #11): - sdk-spec/api-surface/args.md: canonical complex param-type model. - sdk-spec/test-vectors/complex-types/complex.tii: shared schema-only fixture. - sdk-spec/test-vectors/README.md: fixture note. - parity-matrix.md: complex param-type interpretation ✅ across all four SDKs; type-directed value validation/encoding tracked as out of scope. Tooling / docs: - tooling/cshell: add as a submodule (git@github.com:txpipe/cshell.git). - skills/release-toolchain: new release skill. - plans/sdk-complex-type-followups.md: follow-up notes. - AGENTS.md / tooling/AGENTS.md: guidance updates. Co-Authored-By: Claude Opus 4.8 (1M context) --- .gitmodules | 4 + AGENTS.md | 7 +- plans/sdk-complex-type-followups.md | 127 ++++++++++++ sdks/parity-matrix.md | 4 +- sdks/sdk-spec/api-surface/args.md | 52 ++++- sdks/sdk-spec/test-vectors/README.md | 8 + .../test-vectors/complex-types/complex.tii | 119 +++++++++++ skills/release-toolchain/SKILL.md | 184 ++++++++++++++++++ tooling/AGENTS.md | 1 + tooling/cshell | 1 + 10 files changed, 502 insertions(+), 5 deletions(-) create mode 100644 plans/sdk-complex-type-followups.md create mode 100644 sdks/sdk-spec/test-vectors/complex-types/complex.tii create mode 100644 skills/release-toolchain/SKILL.md create mode 160000 tooling/cshell diff --git a/.gitmodules b/.gitmodules index d859754..a4d1b6d 100644 --- a/.gitmodules +++ b/.gitmodules @@ -105,3 +105,7 @@ [submodule "protocols/githoney-protocols"] path = protocols/githoney url = git@github.com:open-tx3/githoney-protocols.git +[submodule "tooling/cshell"] + path = tooling/cshell + url = git@github.com:txpipe/cshell.git + branch = main diff --git a/AGENTS.md b/AGENTS.md index 29d58a5..2e21d03 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -20,10 +20,10 @@ working inside that group. - `core/` → wire-format specs: `tii`, `tir`, `trp`. See [`core/AGENTS.md`](./core/AGENTS.md). - `lang/` → the Tx3 language: `tx3`. See [`lang/AGENTS.md`](./lang/AGENTS.md). -- `tooling/` → toolchain binaries and developer tools: `trix`, `tx3up`, `tx3-lsp`, `tx3-mcp`, `tx3-lift`. See [`tooling/AGENTS.md`](./tooling/AGENTS.md). +- `tooling/` → toolchain binaries and developer tools: `trix`, `tx3up`, `tx3-lsp`, `tx3-mcp`, `tx3-lift`, `cshell`. See [`tooling/AGENTS.md`](./tooling/AGENTS.md). - `plugins/` → editor, CI, and agent integrations: `vscode-tx3`, `tx3-skills`, `actions`. See [`plugins/AGENTS.md`](./plugins/AGENTS.md). -- `backends/` → transaction execution backends and gateways: `tx3-hydra`, `protocol-gateway`. See [`backends/AGENTS.md`](./backends/AGENTS.md). -- `protocols/` → third-party Tx3 protocol definitions from the `open-tx3` org: `indigo`, `strike`, `bodega`, `fluid`, `vyfi`, `snek-fun`, `acme`, `githoney`. See [`protocols/AGENTS.md`](./protocols/AGENTS.md). +- `backends/` → transaction execution backends and gateways: `tx3-hydra`, `protocol-gateway`, `dolos`. See [`backends/AGENTS.md`](./backends/AGENTS.md). +- `protocols/` → third-party Tx3 protocol definitions from the `open-tx3` org: `indigo`, `strike`, `bodega`, `fluid`, `vyfi`, `snek-fun`, `acme`, `githoney`, `txpipe`. See [`protocols/AGENTS.md`](./protocols/AGENTS.md). Two submodules and one subtree sit outside the groupings: @@ -69,6 +69,7 @@ Available skills: - `skills/publish-docs-site/` — publish the latest Tx3 docs to the company-wide docs site (`docs.txpipe.io`) by triggering the `txpipe/docs` `update-submodules` workflow. - `skills/commit-umbrella/` — commit the umbrella repo after submodule pointers move, pre-checking that submodules are pushed, track latest `main`, and that grouping `AGENTS.md` routing is up to date. - `skills/add-language-feature/` — roll out a new Tx3 language feature (operator, expression form, builtin) across every toolchain layer: spec, grammar/AST, analysis/lowering, TIR/reduction, downstream consumers, docs, and agent skills. +- `skills/release-toolchain/` — interactively orchestrate a cross-cutting toolchain release: sequence the crates.io publish waves (tir → tx3-lang & siblings → lsp/mcp/registry), bump dependency pins in lockstep, raise the `trix` version floor, and hand off to `commit-umbrella` for the pointer bump. Pauses at each publish gate for the developer. - `sdks/skills/` — SDK-fleet skills (`add-sdk-feature`, `audit-parity`, `propagate-change`, `release-synced`, `release-sdk-patch`, `run-e2e-tests`, `scaffold-new-sdk`). ## Scope of this repo diff --git a/plans/sdk-complex-type-followups.md b/plans/sdk-complex-type-followups.md new file mode 100644 index 0000000..9a16e46 --- /dev/null +++ b/plans/sdk-complex-type-followups.md @@ -0,0 +1,127 @@ +# Plan: SDK complex-type model — deferred follow-ups + +Status: **open / not started** — depends on the complex-type model PRs landing first +Scope: cross-cutting — the four SDK submodules +(`sdks/{rust,go,python,web}-sdk`), `lang/tx3` codegen +(`bin/tx3c/src/codegen.rs` + each SDK's `.trix/client-lib/` templates), and the +shared spec/fixtures (`sdks/sdk-spec/`). +Origin: explicitly-scoped-out items from the complex param-type model rework +(2026-06-12). Shipped PRs: toolchain#11 (spec + parity + fixture), +rust-sdk#41, go-sdk#14, python-sdk#18, web-sdk#34. +Related: [`sdk-codegen-v1beta0-migration.md`](./sdk-codegen-v1beta0-migration.md) +and [`codegen-client-lifecycle-facade.md`](./codegen-client-lifecycle-facade.md) +— same `.trix/client-lib/` templates and `schemaTypeFor` helper; sequence the +codegen workstream (§3) with those. + +## Context + +The merged work brought every SDK's **runtime `ParamType` interpretation** to +parity with the canonical TII schema model (`sdks/sdk-spec/api-surface/args.md`): +all four now read list/tuple/map/record/variant/unit/utxo/anyAsset, match core +`$ref`s by trailing name across both URL forms, resolve +`#/components/schemas/`, and never throw (unrecognized → `unknown`). + +That work deliberately stopped at **interpretation**. Three capabilities were +left out, plus one verification-hardening item. They are tracked in +`sdks/parity-matrix.md` (the "Type-directed value validation / encoding" row is +❌×4 with a note) and collected here so they aren't lost. + +None of this is required for correct resolution today: argument **values** are a +generic-recursive JSON pass-through (byte arrays → `0x`-hex, big integers → wire +encoding, applied recursively into nested lists/maps/records), and the TRP +resolver performs authoritative type checking. These follow-ups improve +client-side ergonomics, type-safety, and feedback — they do not fix a +correctness bug. + +## Workstreams + +### 1. Type-directed value validation / encoding + +Today the resolved `ParamType` is built and exposed (`Invocation::params`, +`inv.Params()`, etc.) but **not** used to validate or encode the args a caller +supplies. A user can pass a structurally-wrong value and only learn of it from a +TRP error. + +The goal: drive arg coercion/validation from the resolved `ParamType` so the SDK +can (a) reject a value whose shape can't match the declared param before sending, +and (b) apply kind-specific encoding (e.g. coerce a tuple's positional elements +by their declared types rather than relying on the runtime value's JS/Go/Python +type). + +- **web-sdk** is the clearest gap: `core/args.ts` has an `ArgValue` tagged union + and a `fromJson(value, ParamTypeTag)` dispatcher whose `ParamTypeTag` enum has + **no** list/tuple/map/record/variant tags — complex values are currently passed + raw. Extend the tag set and the `fromJson`/`toJson` dispatch to recurse over the + full `ParamType`. +- **rust/go/python**: thread the resolved `ParamType` into the arg setters + (`with_arg`/`SetArg`/`arg`) or a dedicated `validate()` pass; coerce per-kind. +- Keep the generic-recursive path as the fallback for `unknown` params. + +Spec: promote the "type-directed validation/encoding" note in `args.md` from +"not yet required" to a defined MUST/SHOULD once the shape is agreed. Update the +parity-matrix row. + +### 2. Variant argument *construction* encoder + +Interpretation models a `variant` (its cases and field types), but no SDK can yet +help a user **construct** a tagged-union value to pass as an arg. `tx3c` codegen +emits a placeholder for this (`TODO: tagged-union codegen pending the variant arg +encoder`). + +The goal: an idiomatic constructor per SDK for externally-tagged variant values +(e.g. `Side.sell({ price })` → `{ "Sell": { "price": … } }`), validated against +the resolved `variant` `ParamType`. This is the value-side counterpart to §1 and +should land with it. It also unblocks the codegen TODO in §3. + +### 3. Codegen typed bindings for tuple / map / variant + +`lang/tx3` `bin/tx3c/src/codegen.rs` `schemaTypeFor` (and the per-SDK +`.trix/client-lib/*.hbs` templates that call it) currently map only: +- `array` + `items` → `List` / `list[T]` / `[]T` +- `object` + `additionalProperties` → `Record` / `dict[str,V]` / `map[string]V` + +It does **not** handle `array` + `prefixItems` (tuple → emits a loose list/`Any`) +or `oneOf` (variant → emits `unknown`/`Any` + the TODO). So generated typed +clients lose tuple positional types and can't express variants. + +The goal: extend `schemaTypeFor` to emit: +- tuples as the language's tuple/fixed-arity type (TS `[A, B]`, Python + `tuple[A, B]`, Go a generated positional struct or `[]any` with a doc note, + Rust a tuple), +- variants as the generated tagged-union type (pairs with §2's constructor), +- records as the already-generated component types (verify nested refs resolve). + +This lives in the **lang/tx3 repo**, not the SDKs (codegen is delegated there), +but the `.hbs` templates that consume the output live in each SDK repo — change +them in lockstep. Sequence with `sdk-codegen-v1beta0-migration.md`. + +### 4. Adopt the shared fixture in each SDK's unit suite + +`sdks/sdk-spec/test-vectors/complex-types/complex.tii` declares one param of every +kind and was cross-checked through the python and go loaders during the rework, +but only as ad-hoc verification. Each SDK should copy/symlink the fixture into its +local fixtures and add a permanent "loads the shared complex-types vector and +resolves the expected `ParamType` kinds" test, so cross-SDK parity on one +canonical input is enforced by CI rather than asserted once. (The per-SDK unit +suites already cover the canonical *table*; this adds the composed end-to-end +load.) + +## Sequencing + +1. Land the five open PRs (toolchain#11 + the four SDK PRs); bump submodule + pointers via `commit-umbrella`; cut the coordinated SDK version bump + (`release-synced`) for the breaking `ParamType` reshape. +2. §4 (fixture adoption) — cheap, independent, do anytime after merge. +3. §1 + §2 together (value validation + variant constructor) — spec the shape in + `args.md` first, then fan out across SDKs via `add-sdk-feature` / `propagate-change`. +4. §3 (codegen) — after §1/§2 define the runtime types the templates render against; + coordinate with the other codegen plans. + +## Verification + +- §1/§2: per-SDK unit tests for accept/reject of well- and ill-shaped values + against each kind; round-trip a variant value through construct → wire → TRP. +- §3: extend `codegen-check.sh` to render a protocol with tuple/map/variant params + and compile the generated client in each language. +- §4: the shared-fixture load test green in all four CIs. +- `parity-matrix.md` updated as each row flips; only mark ✅ where covered by tests. diff --git a/sdks/parity-matrix.md b/sdks/parity-matrix.md index 7118443..766e515 100644 --- a/sdks/parity-matrix.md +++ b/sdks/parity-matrix.md @@ -11,7 +11,7 @@ | ❌ | Not implemented. | | — | Not applicable to this SDK (must be justified in notes). | -**Snapshot date:** 2026-05-24 (post-merge of the unified-builder ports across rust/web/python/go; see [rust-sdk#38](https://github.com/tx3-lang/rust-sdk/pull/38), [web-sdk#29](https://github.com/tx3-lang/web-sdk/pull/29), [python-sdk#13](https://github.com/tx3-lang/python-sdk/pull/13), [go-sdk#10](https://github.com/tx3-lang/go-sdk/pull/10)). +**Snapshot date:** 2026-06-12 (complex-type model parity: all four SDKs reworked their `ParamType` interpretation to the canonical model in `api-surface/args.md` — trailing-name core `$ref` matching across `tii#/$defs/` + legacy `core#` forms, distinct list/tuple/map/record/variant kinds carrying inner types, never-throw `unknown` fallback, component-ref resolution. Shared fixture: `sdk-spec/test-vectors/complex-types/complex.tii`). Prior snapshot 2026-05-24 (post-merge of the unified-builder ports across rust/web/python/go; see [rust-sdk#38](https://github.com/tx3-lang/rust-sdk/pull/38), [web-sdk#29](https://github.com/tx3-lang/web-sdk/pull/29), [python-sdk#13](https://github.com/tx3-lang/python-sdk/pull/13), [go-sdk#10](https://github.com/tx3-lang/go-sdk/pull/10)). --- @@ -42,6 +42,8 @@ Capability references are to `sdk-spec/api-surface/`. | 3.7 | `waitForConfirmed` / `waitForFinalized` + `PollConfig` | ✅ | ✅ [`web-sdk/sdk/src/facade/submitted.ts`](../web-sdk/sdk/src/facade/submitted.ts) | ✅ [`go-sdk/sdk/facade/submitted.go`](../go-sdk/sdk/facade/submitted.go) | ✅ [`python-sdk/sdk/src/tx3_sdk/facade/submitted.py`](../python-sdk/sdk/src/tx3_sdk/facade/submitted.py) | Defaults: 20 attempts, 5 s. Go respects `context.Context` cancellation. | | 3.8 | Discriminated error model | ✅ (`facade::Error` enum) | ✅ Full hierarchy via `instanceof` | ✅ Per-domain marker interfaces + `errors.As()` | ✅ Rooted at `Tx3Error` with category subclasses | Go: `TiiError`, `TrpError`, `SignerError`, `FacadeError` marker interfaces. Note: `MissingTrpEndpoint`/`UnknownProfile`/`UnknownParty` builder-error variants only exist where the builder exists (rust-sdk today). | | 3.9 | Argument marshalling | ✅ (`core::ArgMap`) | ✅ [`web-sdk/sdk/src/core/args.ts`](../web-sdk/sdk/src/core/args.ts) | ✅ [`go-sdk/sdk/core/args.go`](../go-sdk/sdk/core/args.go) | ✅ [`python-sdk/sdk/src/tx3_sdk/core/args.py`](../python-sdk/sdk/src/tx3_sdk/core/args.py) | Go: `ArgValue` tagged union + `CoerceArg()` for native types. | +| 3.9 | Complex param-type interpretation (`list`/`tuple`/`map`/`record`/`variant`) | ✅ [`rust-sdk/sdk/src/tii/mod.rs`](../rust-sdk/sdk/src/tii/mod.rs) | ✅ [`web-sdk/sdk/src/tii/paramType.ts`](../web-sdk/sdk/src/tii/paramType.ts) | ✅ [`go-sdk/sdk/tii/param_type.go`](../go-sdk/sdk/tii/param_type.go) | ✅ [`python-sdk/sdk/src/tx3_sdk/tii/param_type.py`](../python-sdk/sdk/src/tx3_sdk/tii/param_type.py) | All four implement the canonical model in [`api-surface/args.md`](sdk-spec/api-surface/args.md): scalar core `$ref`s matched by **trailing name** across both `tii#/$defs/` and legacy `core#` forms (incl. `Utxo`/`AnyAsset`); distinct `list`(inner)/`tuple`(elements)/`map`(value)/`record`(fields)/`variant`(cases) kinds carrying their element types; `#/components/schemas/` resolved + recursed; **never throws** — unrecognized shapes (bare `string`, unresolved object, unknown `$ref`) become `unknown`/`Unknown`/`UNKNOWN` carrying the raw schema. Each SDK has a unit suite over the full table; Go added a custom `Schema` unmarshaler for the `items:false` / `additionalProperties:false` forms `tx3c` emits, and Rust dropped `schemars` for `serde_json::Value`. **Breaking** `ParamType` reshape → minor/major bumps. Shared fixture: [`sdk-spec/test-vectors/complex-types/complex.tii`](sdk-spec/test-vectors/complex-types/complex.tii). | +| 3.9 | Type-directed value validation / encoding | ❌ | ❌ | ❌ | ❌ | Out of scope for the model rework. Today arg values are generic-recursive JSON pass-through (bytes→`0x`hex, bigint encoding applied to nested elements); the resolved `ParamType` is **not** used to validate/encode each arg, and there is no variant-construction encoder (`tx3c` codegen still emits a `TODO: tagged-union codegen pending the variant arg encoder` placeholder). Codegen typed bindings for tuple/map/variant also pending in `lang/tx3` `schemaTypeFor`. | | §4 | Top-level re-exports | ✅ [`lib.rs`](../rust-sdk/sdk/src/lib.rs) | ✅ [`web-sdk/sdk/src/index.ts`](../web-sdk/sdk/src/index.ts) | ✅ [`go-sdk/sdk/tx3sdk.go`](../go-sdk/sdk/tx3sdk.go) | ✅ [`python-sdk/sdk/src/tx3_sdk/__init__.py`](../python-sdk/sdk/src/tx3_sdk/__init__.py) | All four export `Tx3Client`, `Tx3ClientBuilder`, `Party`, `CardanoSigner`/`Ed25519Signer`, `PollConfig`, plus `Profile` and `MissingTrpEndpointError`. | --- diff --git a/sdks/sdk-spec/api-surface/args.md b/sdks/sdk-spec/api-surface/args.md index 1a18574..7d4c161 100644 --- a/sdks/sdk-spec/api-surface/args.md +++ b/sdks/sdk-spec/api-surface/args.md @@ -4,4 +4,54 @@ An SDK MUST accept native host-language values for transaction args (integers, s The SDK SHOULD expose an `ArgValue`-equivalent type for users who need to construct tagged values explicitly (useful for `UtxoRef`, `UtxoSet`, etc.). -*Rust reference:* `tx3_sdk::core::ArgMap`. *Web reference:* `web-sdk/sdks/src/core/args.ts`. +## The parameter-type model + +Each SDK builds a **parameter-type model** (`ParamType`) by interpreting each property of a transaction's `params` JSON schema (and the protocol `environment` schema). This model drives introspection (`unspecifiedParams`) and — where implemented — value coercion. `tx3c` lowers every Tx3 type into one of a fixed, closed set of schema shapes; an SDK MUST interpret all of them. + +### Scalar (`$ref`) types + +Built-in scalar types are emitted as a JSON `$ref`. The **canonical** form is `https://tx3.land/specs/v1beta0/tii#/$defs/`; the **legacy** form `https://tx3.land/specs/v1beta0/core#` MUST also resolve. An SDK MUST match the type by the **trailing name** — the segment after the last `#` or `/` — so both forms map identically. Matching the full URI is non-conforming (it breaks against current `tx3c` output, which emits the `tii#/$defs/` form). + +| Trailing name | `ParamType` kind | +|---|---| +| `Bytes` | `bytes` | +| `Address` | `address` | +| `UtxoRef` | `utxoRef` | +| `Utxo` | `utxo` | +| `AnyAsset` | `anyAsset` | + +A `$ref` of the form `#/components/schemas/` references a user-defined type; the SDK MUST resolve `` against the TII's `components.schemas` table and interpret the resolved schema recursively. The `components` table MUST therefore be threaded into the parameter-type builder. + +### Primitive types + +| Schema | `ParamType` kind | +|---|---| +| `{ "type": "integer" }` | `integer` | +| `{ "type": "boolean" }` | `boolean` | +| `{ "type": "null" }` | `unit` | + +### Compound types + +| Schema | `ParamType` kind | Carries | +|---|---|---| +| `{ "type": "array", "items": }` | `list` | inner element type | +| `{ "type": "array", "prefixItems": [, , …], "items": false }` | `tuple` | positional element types | +| `{ "type": "object", "additionalProperties": }` | `map` | value type (keys are always strings) | +| `{ "type": "object", "properties": {…}, "required": […] }` | `record` | field name → type | +| `{ "oneOf": [, …] }` (externally tagged) | `variant` | case tag → fields | + +A variant case has the externally-tagged shape `{ "type": "object", "additionalProperties": false, "required": [""], "properties": { "": } }`. The SDK reads the single `required` entry as the case tag and interprets `properties[]` (a record) as its fields. + +An SDK SHOULD model `list` / `tuple` / `map` / `record` / `variant` as **distinct** kinds carrying their element/field types, so downstream consumers can introspect structure. An SDK that does not yet carry the inner types for a kind MUST still accept and marshal values of that kind, and MUST log the gap in `parity-matrix.md`. + +### Never throw; the `unknown` fallback + +Building the parameter-type model MUST NOT fail on an unrecognized schema shape (including a bare `{ "type": "string" }`, an unresolved `{ "type": "object" }` fallback that `tx3c` emits for unresolvable forward references, or an unknown `$ref`). Such shapes MUST map to an `unknown` kind that carries the raw schema. A bare `string` MUST NOT be assumed to be an `address` — `tx3c` always emits `Address` as a `$ref`, so a bare string is genuinely untyped. + +## Value marshalling + +Argument **values** are marshalled to the TRP wire format independently of the parameter-type model (the wire form is a plain JSON value; the TRP resolver performs authoritative type checking). Marshalling MUST be **generic-recursive**: scalar coercions (byte arrays → `0x`-prefixed hex, big integers → their wire encoding) MUST be applied to values nested inside lists, tuples, maps, and records, not only to top-level args. The wire value for a `list`/`tuple` is a JSON array; for a `map`/`record`/`variant` it is a JSON object. + +> Type-*directed* validation/encoding (using the resolved `ParamType` to validate each arg) and the variant-construction encoder are not yet required; track them in `parity-matrix.md` when added. + +*Rust reference:* `tx3_sdk::core::ArgMap`, `tx3_sdk::tii::ParamType`. *Web reference:* `web-sdk/sdk/src/core/args.ts`, `web-sdk/sdk/src/tii/paramType.ts`. diff --git a/sdks/sdk-spec/test-vectors/README.md b/sdks/sdk-spec/test-vectors/README.md index 71e06e6..0757cc1 100644 --- a/sdks/sdk-spec/test-vectors/README.md +++ b/sdks/sdk-spec/test-vectors/README.md @@ -17,3 +17,11 @@ This directory contains canonical, spec-level e2e vectors shared across all Tx3 - `transfer.tx3` - `transfer.tii` - `transfer.preprod.env` +- `complex-types/` + - `complex.tii` — schema-only fixture whose `complex` transaction declares one + param of every `ParamType` kind (integer, boolean, unit, Address, UtxoRef, + AnyAsset, list, tuple, map, plus a component-ref record `AssetClass` and a + component-ref variant `Side`), with scalar `$ref`s in the canonical + `tii#/$defs/` form. Used to verify parameter-type interpretation parity + across SDKs (see `api-surface/args.md`). The TIR envelope is a non-resolvable + placeholder — this vector is for type-model tests, not TRP resolution. diff --git a/sdks/sdk-spec/test-vectors/complex-types/complex.tii b/sdks/sdk-spec/test-vectors/complex-types/complex.tii new file mode 100644 index 0000000..810ff50 --- /dev/null +++ b/sdks/sdk-spec/test-vectors/complex-types/complex.tii @@ -0,0 +1,119 @@ +{ + "tii": { + "version": "v1beta0" + }, + "protocol": { + "name": "complex-types", + "scope": "unknown", + "version": "0.0.1", + "description": "Schema-only fixture exercising every ParamType kind. The TIR envelope is a non-resolvable placeholder; this vector is for parameter-type interpretation tests, not TRP resolution." + }, + "environment": { + "type": "object", + "properties": { + "fee": { + "type": "integer" + } + }, + "required": ["fee"] + }, + "parties": { + "sender": {}, + "receiver": {} + }, + "profiles": { + "local": { + "environment": {}, + "parties": {} + } + }, + "components": { + "schemas": { + "AssetClass": { + "type": "object", + "properties": { + "policy": { "$ref": "https://tx3.land/specs/v1beta0/tii#/$defs/Bytes" }, + "name": { "$ref": "https://tx3.land/specs/v1beta0/tii#/$defs/Bytes" } + }, + "required": ["policy", "name"] + }, + "Side": { + "oneOf": [ + { + "type": "object", + "additionalProperties": false, + "required": ["Buy"], + "properties": { + "Buy": { "type": "object", "properties": {}, "required": [] } + } + }, + { + "type": "object", + "additionalProperties": false, + "required": ["Sell"], + "properties": { + "Sell": { + "type": "object", + "properties": { "price": { "type": "integer" } }, + "required": ["price"] + } + } + } + ] + } + } + }, + "transactions": { + "complex": { + "params": { + "type": "object", + "properties": { + "quantity": { "type": "integer" }, + "flag": { "type": "boolean" }, + "nothing": { "type": "null" }, + "recipient": { "$ref": "https://tx3.land/specs/v1beta0/tii#/$defs/Address" }, + "source": { "$ref": "https://tx3.land/specs/v1beta0/tii#/$defs/UtxoRef" }, + "bag": { "$ref": "https://tx3.land/specs/v1beta0/tii#/$defs/AnyAsset" }, + "amounts": { + "type": "array", + "items": { "type": "integer" } + }, + "pair": { + "type": "array", + "prefixItems": [ + { "type": "integer" }, + { "$ref": "https://tx3.land/specs/v1beta0/tii#/$defs/Bytes" } + ], + "items": false, + "minItems": 2, + "maxItems": 2 + }, + "labels": { + "type": "object", + "additionalProperties": { "type": "integer" } + }, + "asset": { "$ref": "#/components/schemas/AssetClass" }, + "side": { "$ref": "#/components/schemas/Side" } + }, + "required": [ + "quantity", + "flag", + "nothing", + "recipient", + "source", + "bag", + "amounts", + "pair", + "labels", + "asset", + "side" + ] + }, + "tir": { + "content": "00", + "encoding": "hex", + "version": "v1beta0" + } + } + } +} diff --git a/skills/release-toolchain/SKILL.md b/skills/release-toolchain/SKILL.md new file mode 100644 index 0000000..4683116 --- /dev/null +++ b/skills/release-toolchain/SKILL.md @@ -0,0 +1,184 @@ +# Release Toolchain Skill + +## Purpose +Orchestrate a **cross-cutting toolchain release** — a change that originates in an upstream +crate (`core/tir`, `lang/tx3`) and must ripple through every downstream consumer that pins it +via crates.io — interactively, in lockstep with the developer who performs the actual +`crates.io` publishes. This skill is the conductor: it sequences the dependency waves, pauses +at each **publish gate** for the developer to cut the release, then bumps the next wave's +dependency pins, and finishes by moving submodule pointers and the `trix` version floor. It was +battle-tested by the parametric-tuple rollout (tir `0.18.0` → tx3-lang `0.22.0` + siblings → +lsp/mcp/registry → trix floor → umbrella). + +It complements the other skills rather than replacing them: +- **`add-language-feature`** lands the *code* across submodules (one PR each). Run it first. +- **`release-toolchain`** (this) sequences *publishing* those merged crates and bumping the + pins that connect them. Run it after the feature PRs are merged. +- **`commit-umbrella`** is the final wave (submodule-pointer commit) — invoked from here. +- **`channel-version-update`** ships the resulting *binaries* to a release channel — a + separate, later step once the tools cut GitHub releases. + +## Prerequisites +- Run from the umbrella repo root with submodules initialized (`git submodule update --init`). +- The feature/fix code is already **merged** into each affected submodule's `main` (that's + `add-language-feature`'s job). This skill releases what is merged; it does not write feature code. +- `cargo` for building/verifying each Rust workspace; `gh` authenticated for PR/merge checks. +- **The developer publishes to crates.io**, not the agent. The agent prepares dep bumps and + verifies builds; it stops at each publish gate and waits for the developer's "published vX" signal. + +## Context + +### The crates.io dependency graph (what pins what) +Submodules are independent repos depending on each other by **published crates.io version**, not +path deps. A release wave = the set of crates that can publish once their upstream pin is available. + +``` +core/tir (tx3-tir) + │ pinned by ↓ +lang/tx3 (tx3-lang, tx3-cardano, tx3-resolver) ── also: registry, tooling/tx3-lift pin tx3-tir directly + │ pinned by ↓ +tooling/tx3-lsp, tooling/tx3-mcp, registry ── pin tx3-lang (and often tx3-tir transitively) + │ gated by ↓ +tooling/trix ([toolchain] floor + COMPAT_MATRIX) ── version-gates the produced TIR; no crate dep + │ recorded by ↓ +umbrella submodule pointers → manifest-*.json (binaries, later, separate channel release) +``` + +Verify the real pins before sequencing — grep, don't assume: +```bash +grep -rn 'tx3-tir\s*=\|tx3-lang\s*=' --include=Cargo.toml core lang tooling registry +``` + +### Semver reality for 0.x crates — the central hazard +**Every `0.x.y` minor bump is breaking** (cargo treats `^0.17` and `0.18` as incompatible). This +makes a tir bump a *flag-day* for its consumers: +- You **cannot** half-bump. If consumer C directly pins `tx3-tir = 0.18` but still pins a + *published* `tx3-lang` that itself pins `tx3-tir = ^0.17`, cargo must resolve two incompatible + tirs in one tree → it fails. **Hold C until `tx3-lang` is republished against the new tir**, then + bump both pins in C together. +- **Transitive pins via git-rev deps are a trap.** A consumer that pins a sibling by git rev + (e.g. `registry/tracker` → `tx3-lift` at rev `abc123`) drags in *that rev's* tir. Bumping the + consumer's direct tir while the git-rev still carries old tir reproduces the dual-tir conflict. + Advance the git rev to the sibling's merged bump commit as part of the same wave. + +### Publish gates are developer actions +The agent never runs `cargo publish`. Each wave ends at a gate where the developer (or CI-on-tag) +publishes the wave's crates. The skill's job at a gate: state exactly **which crates, which +versions** to publish, then **stop and wait**. The developer's reply ("published tx3-tir 0.18", +"sibling crates published") is the signal to start the next wave's dep bumps. + +### Verifying an un-published wave: patch, build, revert +To confirm a downstream crate *will* build against an upstream that isn't on crates.io yet, use a +temporary `[patch.crates-io]` path override (depth-adjusted per crate), build, then revert **both** +`Cargo.toml` and `Cargo.lock` individually. Never commit a patch or lockfile churn. (Full mechanics +in `add-language-feature`'s "Cross-repo dependency model".) The compile is the safety net for +exhaustive-match breaks — and a `-p ` build can **miss** a sibling binary's broken +match (the tuple rollout's `bin/tx3c` arm surfaced only on a full-workspace build); build the +**whole workspace** of each consumer, not just the changed package. + +## Procedure + +### 0. Map the release and confirm the plan with the developer +1. Identify the originating change and the **lowest** crate it modifies (usually `tx3-tir` or + `tx3-lang`). Grep the pin graph (above) to enumerate every downstream consumer. +2. Decide each crate's **new version** with the developer (0.x minor for a breaking schema add; + patch only for a truly compatible fix — rare across the tir boundary). +3. Lay out the waves as a table and get a go-ahead. Example (tuple rollout): + + | Wave | Crates | Publishes | Then bump | + | --- | --- | --- | --- | + | 1 | `tx3-tir` | → 0.18.0 | tir pin in lang/tx3, registry, tx3-lift | + | 2 | `tx3-lang`,`tx3-cardano`,`tx3-resolver` | → 0.22.0 | tx3-lang pin in lsp/mcp/registry | + | 3 | (consumers don't publish libs) | — | lsp/mcp/registry dep bumps land | + | 4 | `trix` | floor → 0.22.0 | `[toolchain]` + COMPAT_MATRIX | + | 5 | umbrella | pointers | commit-umbrella | + +### 1. Wave 1 — release the root crate (e.g. `tx3-tir`) +1. Confirm the root crate's `main` carries the merged change and its `Cargo.toml` version is the + target (bump it in a small PR if not — that PR is part of `add-language-feature`, not this skill). +2. **Gate:** tell the developer *"publish `tx3-tir 0.18.0` to crates.io, then confirm."* **Stop.** +3. On confirmation, proceed to Wave 2. + +### 2. Wave 2 — bump the pin, land + release the next layer (`tx3-lang` & siblings) +1. In every crate that *directly* pins the just-published crate (`lang/tx3` workspace: tx3-lang, + tx3-cardano, tx3-resolver; plus `registry`, `tooling/tx3-lift`), update the pin to the new + version. Bump the crates' **own** versions where they also re-publish (tx3-lang 0.21→0.22). +2. Verify each builds (full workspace) against the now-published upstream — no patch needed once + it's on crates.io; if you're ahead of the publish, use the patch-build-revert dance. +3. Land these dep-bump commits (PRs, squash-merged). Note in each PR body: *"depends on tx3-tir + 0.18 (published); release after merge."* +4. **Gate:** tell the developer *"publish `tx3-lang`, `tx3-cardano`, `tx3-resolver` 0.22.0."* **Stop.** + +### 3. Wave 3 — bump the leaf consumers (`tx3-lsp`, `tx3-mcp`, `registry`) +1. Now that `tx3-lang` is published against the new tir, bump **both** `tx3-lang` and (if directly + pinned) `tx3-tir` in the leaf consumers **together** — this resolves the dual-tir hazard. +2. For any consumer pinning a sibling by **git rev**, advance the rev to that sibling's merged + bump commit (`registry/tracker` → `tx3-lift` new rev). Confirm the rev is on the sibling's `main`. +3. Build each consumer's **whole workspace**. Environmental test failures (e.g. `DATABASE_URL must + be set` for sqlx integration tests) are not release blockers — distinguish them from compile + breaks and say so. Pre-existing, unrelated breakage (e.g. an upstream protobuf drift) is **out + of scope**: document it in the PR, don't fix it here. +4. Land the consumer dep-bump PRs. + +### 4. Wave 4 — raise the `trix` version floor +The TIR schema addition is **forward-incompatible** (a pre-bump reader hits `unknown variant`). +The gate that protects users is `trix`'s version check, in two places: +1. `tooling/trix/src/spawn/compat.rs` — raise `COMPAT_MATRIX` `tx3c { min }` to the producing + release (e.g. `0.22.0`) and update the rationale comment to name the new feature/variant. +2. Projects pin their own floor in `trix.toml [toolchain] tx3c` (a lower bound only). The matrix + is the global default. Decide with the developer whether this release moves the global floor. +3. `cargo test -p trix` (the `evaluate`/`collect_project_mins` unit tests gate this). Land the PR. + +### 5. Wave 5 — move umbrella pointers +Hand off to **`commit-umbrella`**: it runs the three pre-flight checks (pushed / tracks-latest-main +/ routing), repins each moved submodule to its `origin/main` tip (remember: squash-merge rewrites +SHAs, so `DIVERGED` is confirmed via `gh pr`, not SHA matching), and commits the pointer bump on the +staging branch. Stage only the submodules this release moved; leave unrelated drift (other behind +backends, stray `AGENTS.md` edits) untouched and surface it. + +### 6. (Later, separate) ship binaries to a channel +Releasing the *crates* does not ship the *tools* to users. When the binaries (`tx3c`, `trix`, +`tx3-lsp`, `tx3-mcp`) cut their own GitHub releases, use **`channel-version-update`** to bump +`manifest-*.json`. That's a distinct, developer-initiated step — mention it as the natural +follow-up; don't fold it into this release without being asked. + +## Decision Guidelines +- **Never publish for the developer.** State the crates+versions for each gate and wait. Publishing + is irreversible and outward-facing. +- **Bump consumers of a 0.x crate in lockstep, not piecemeal.** A consumer that pins both + `tx3-lang` and `tx3-tir` must move both in one commit *after* `tx3-lang` republishes — bumping + one early guarantees a dual-version resolve failure. +- **Treat git-rev deps as version pins.** They carry a transitive crate graph; advancing the direct + pin without advancing the rev re-introduces the conflict. +- **Verify merge state via `gh pr`, not SHA** when repinning submodules — squash-merge gives every + merged branch a fresh SHA (see `commit-umbrella`). +- **Patch-build-revert to verify ahead of a publish; never commit the patch or lockfile churn.** +- **Distinguish compile breaks from environmental/pre-existing failures.** Only the former block a + wave. Name the latter to the developer and keep them out of scope. +- **Scope the umbrella commit to this release's submodules.** Don't fast-forward unrelated drift. + +## Safety Checks +- [ ] Wave order respects the pin graph: root crate published *before* any consumer bumps its pin. +- [ ] No consumer was bumped while still pinning a published upstream that carries the old (incompatible) transitive version — dual-version resolves were avoided. +- [ ] Git-rev deps advanced to the sibling's merged bump commit (rev confirmed on `main`). +- [ ] Each consumer's **whole workspace** compiled (not just `-p` the changed crate) — exhaustive-match breaks in sibling binaries caught. +- [ ] Every temporary `[patch.crates-io]` and `Cargo.lock` change reverted before committing. +- [ ] `trix` floor (`COMPAT_MATRIX` + rationale comment) raised to the producing release; `cargo test -p trix` green. +- [ ] Umbrella pointers moved via `commit-umbrella` (its three checks passed); only this release's submodules staged. +- [ ] Each publish was performed by the developer at an explicit gate, not by the agent. + +## Error Handling +- **`failed to select a version … links to two different versions of tx3-tir`** — the dual-tir + conflict: a consumer pins new tir directly while a published `tx3-lang` (or a git-rev sibling) + still pins old tir. Hold the consumer until `tx3-lang` republishes, then bump both pins together; + advance any git-rev sibling to its new-tir commit. +- **`unknown variant ` / wrong-arity decode at runtime** — a pre-bump reader met new TIR. + Expected forward-incompat; the fix is the `trix` floor bump (Wave 4), not a code change. +- **Exhaustive-match compile error surfaces only on full build** — you built `-p ` and missed + a sibling binary's match arm. Rebuild the whole workspace of every consumer. +- **`DATABASE_URL must be set` / other env-dependent test failure** — environmental, not a release + blocker. Confirm the crate *compiles*; note the skipped integration tests to the developer. +- **Pre-existing unrelated breakage in a consumer (e.g. upstream protobuf drift)** — out of scope. + Document in the PR; do not fix it inside the release. +- **Submodule reports `DIVERGED` at the umbrella step** — squash-merge rewrote the SHA. Confirm the + PR merged via `gh pr`, fast-forward the submodule to `origin/main`, pin that tip (see `commit-umbrella`). diff --git a/tooling/AGENTS.md b/tooling/AGENTS.md index 750f555..f4e802f 100644 --- a/tooling/AGENTS.md +++ b/tooling/AGENTS.md @@ -10,6 +10,7 @@ Part of the Tx3 [`toolchain`](../AGENTS.md) umbrella. The `tooling/` grouping ho - `tooling/tx3-lsp/` → [`tx3-lang/tx3-lsp`](https://github.com/tx3-lang/tx3-lsp) — `tx3-lsp` language server. - `tooling/tx3-mcp/` → [`tx3-lang/tx3-mcp`](https://github.com/tx3-lang/tx3-mcp) — MCP server exposing the toolchain to AI agents / editors. - `tooling/tx3-lift/` → [`tx3-lang/tx3-lift`](https://github.com/tx3-lang/tx3-lift) — semantic-enrichment framework: annotates on-chain txs with Tx3 protocol context. +- `tooling/cshell/` → [`txpipe/cshell`](https://github.com/txpipe/cshell) — `cshell` terminal wallet for Cardano. A submodule's own `AGENTS.md` / `CLAUDE.md` / `README.md` overrides this file for work inside that path. diff --git a/tooling/cshell b/tooling/cshell new file mode 160000 index 0000000..5134bbc --- /dev/null +++ b/tooling/cshell @@ -0,0 +1 @@ +Subproject commit 5134bbc05ede93cf7c0639bbac8cddc3bc54656b