Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions .github/workflows/dx-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,13 @@ name: DX E2E
# (install the channel via tx3up, cache it, run one journey) live in the local
# composite action .github/actions/dx-e2e-run.
#
# - stable: a comprehensive pass over *every* journey. Edge journeys whose
# `#@ min-tx3c` exceeds stable's tx3c install + skip (green), and auto-run
# once the feature graduates to stable.
# - beta: cherry-picked *edge-feature* journeys only (e.g. 02-lang-tour's
# tuples) — stable already covers everything broadly.
# - stable: the offline scaffold gate + lang-surface journeys. Edge journeys
# whose `#@ min-tx3c` exceeds stable's tx3c install + skip (green), and
# auto-run once the feature graduates to stable.
# - beta: edge-feature journeys (e.g. 02-lang-tour's tuples), plus the devnet
# round-trip (04). The round-trip currently *fails* on released channels (a
# known, tracked trix bug fixed on main but unreleased) — intentionally red,
# not a tolerated xfail; it goes green once the fix ships to a channel.
#
# Compat lives in one place: the journey's `#@ min-tx3c` header, enforced by the
# runner's skip gate. Native runners only — a DX test must exercise the real
Expand Down Expand Up @@ -56,7 +58,9 @@ jobs:
fail-fast: false
matrix:
os: [ubuntu-latest, ubuntu-24.04-arm, macos-latest]
journey: [02-lang-tour, 03-lang-edge] # cherry-pick: edge-feature journeys only
# edge-feature journeys + the devnet round-trip. The round-trip fails on
# released channels until the trix fix ships (intentionally red, not xfail).
journey: [02-lang-tour, 03-lang-edge, 04-devnet-roundtrip]
steps:
- uses: actions/checkout@v4
- uses: ./.github/actions/dx-e2e-run
Expand Down
74 changes: 29 additions & 45 deletions e2e/README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,10 @@
# DX end-to-end tests

These tests validate the **developer experience of the assembled toolchain**: that a
developer who installs a channel and follows the normal journey
(`init → check → build → test`, including a real local devnet round-trip) actually gets a
working result. They exercise the real `trix`, `tx3c`, `dolos`, and `cshell` binaries
together — interop that no single submodule's own tests cover. This is the umbrella's job,
because only it knows the full channel composition (the `manifest-*.json` files) and is where
developer who installs a channel and follows the normal flow (`init → check → build → test`)
actually gets a working result. They exercise the real `trix`, `tx3c`, `dolos`, and `cshell`
binaries together — interop that no single submodule's own tests cover. This is the umbrella's
job, because only it knows the full channel composition (the `manifest-*.json` files) and is where
releases are cut.

This is deliberately *not* a duplicate of `tooling/trix/tests/e2e/`, which covers `trix init`
Expand All @@ -16,45 +15,33 @@ helper binaries.

```
e2e/
├── run.sh # entrypoint: resolve binaries, run journeys, summarize
├── lib/common.sh # logging + fail-fast assertion helpers
├── run.sh # entrypoint: resolve binaries, run journeys, summarize
├── lib/common.sh # logging + fail-fast assertion helpers
└── journeys/
── 01-basic-init/journey.sh # init → check → build → test (offline devnet round-trip)
── 02-lang-tour/
├── journey.sh # init → swap in feature-dense main.tx3 → check → build → inspect tir
└── main.tx3 # feature-dense fixture (a copy of the lang's lang_tour example)
── <NN>-<name>/
── journey.sh # the journey script (sources lib/common.sh, drives trix)
├── README.md # what this journey covers, its scope, and any caveats
└── <fixtures> # optional per-journey fixtures (e.g. a main.tx3)
```

A **journey** is a self-contained script that sources `lib/common.sh` and drives `trix`
with the assertion helpers. `run.sh` discovers every `journeys/*/journey.sh`, runs each in
an isolated temp working directory, and prints a markdown pass/fail summary, exiting
non-zero if any journey fails.

The journeys cover complementary axes:

| Journey | Covers | Scope |
|---------|--------|-------|
| **01-basic-init** | the default scaffold end to end, including a real devnet round-trip (trix + tx3c + dolos + cshell + resolver) | runtime |
| **02-lang-tour** | the breadth of the language surface — env, records/variants, lists/maps/tuples, policies/assets, spread, locals, and the full Cardano construct set — pushed through `check → build → inspect tir` | compile/lower |
| **03-lang-edge** | the *newest* language additions — user-defined functions, the `*`/`/` operators, parametric tuples (literals + indexing), and `///` doc-comments — pushed through `check → build → inspect tir` (needs tx3c ≥ 0.22) | compile/lower |

`02-lang-tour` is compile/lower only: its feature-dense tx references hard-coded UTxOs, mints,
and plutus scripts, so it can't resolve against a fresh devnet (the round-trip lives in 01).
non-zero if any journey fails. **Each journey documents itself in its own `README.md`** — this
top-level README only describes the harness.

## Channel-aware journeys

A journey's fixture may use language features newer than an older channel's `tx3c` (e.g.
`02-lang-tour` uses tuples, which need tx3c ≥ 0.22 — present on `beta`, not `stable`). A journey
declares its floor with a header comment:
A journey's fixture may use language features newer than an older channel's `tx3c`. A journey
declares the minimum it needs with a header comment in its `journey.sh`:

```sh
#@ min-tx3c: 0.22.0
```

The runner reads the `tx3c` under test and **skips** (does not fail) any journey whose floor isn't
met — `./e2e/run.sh --channel stable` runs `01-basic-init` and skips `02-lang-tour`, exiting 0.
The gate is version-based so it **auto-heals**: when a feature graduates to `stable` (its `tx3c`
bumps past the floor), the journey starts running there with no edit.
met, exiting 0. The gate is version-based so it **auto-heals**: when a feature graduates to an older
channel (its `tx3c` bumps past the floor), the journey starts running there with no edit.

## Which binaries get tested

Expand All @@ -74,9 +61,9 @@ respects `$HOME` on Unix). CI runners are already ephemeral, so they pass `--no-

## Running locally

Prerequisites: `bash`, `git`, and the toolchain reachable in your chosen mode. The basic
journey needs **no secrets and no network** beyond the one-time install — the devnet round-trip
runs entirely on a local Dolos devnet with deterministic cshell wallets.
Prerequisites: `bash`, `git`, and the toolchain reachable in your chosen mode. No journey needs
secrets or a live network beyond the one-time install — runtime journeys run entirely on a local
Dolos devnet with deterministic cshell wallets.

```sh
# Test the toolchain you already have installed:
Expand All @@ -93,7 +80,7 @@ TX3_CSHELL_PATH=/path/to/cshell \
./e2e/run.sh --local

# Run one journey with live output:
./e2e/run.sh --journey 01-basic-init --verbose
./e2e/run.sh --journey <name> --verbose
```

Install `tx3up` (only needed for `--channel`) with the bootstrap script:
Expand All @@ -110,13 +97,9 @@ stable for CI upload.
## CI

`.github/workflows/dx-e2e.yml` runs **one job per channel**, each an `{os × journey}` matrix over
the native runners (`ubuntu-latest`, `ubuntu-24.04-arm`, `macos-latest`):

- **`stable`** — a comprehensive pass over *every* journey. Edge journeys whose `#@ min-tx3c`
exceeds stable's `tx3c` install and **skip** (green), and start running automatically once the
feature graduates to stable (auto-heal).
- **`beta`** — cherry-picked *edge-feature* journeys (the ones exercising features only on beta, like
`02-lang-tour`'s tuples). Stable already covers everything broadly, so beta stays focused.
the native runners (`ubuntu-latest`, `ubuntu-24.04-arm`, `macos-latest`). The `stable` job runs the
broad-coverage journeys; the `beta` job cherry-picks the ones exercising features (or fixes) that
only beta has yet. The per-job journey lists live in the workflow itself.

Compat lives in one place — the journey's `#@ min-tx3c` header, enforced by the runner's skip gate —
so the workflow needs no per-cell compat config. The shared per-cell steps (install via tx3up, cache
Expand All @@ -125,15 +108,16 @@ Native runners only: a DX test must validate the real per-platform install, whic
would misrepresent (and drop macOS). Triggers: pushes to `main` touching `manifest-*.json` /
`e2e/**` / the workflow / the action; a nightly schedule; manual dispatch.

No journey here needs secrets. Future live-network journeys would run in a separate, secrets-gated
job.
Journeys that need secrets (live network) would run in a separate, secrets-gated job.

## Adding a journey

See the **`add-e2e-journey` skill** (`skills/add-e2e-journey/SKILL.md`) — the canonical guide for
the journey contract, the `lib/common.sh` helper API, fixtures, and `xfail`. In brief: add a
`journeys/<NN>-<name>/journey.sh` that sources `${E2E_LIB}` and drives `"${TRIX}"` with the
assertion helpers, and `run.sh` discovers it automatically.
the journey contract, the `lib/common.sh` helper API, fixtures, and capability gating. In brief: add
a `journeys/<NN>-<name>/` directory with a `journey.sh` that sources `${E2E_LIB}` and drives
`"${TRIX}"`, plus a `README.md` describing it, and `run.sh` discovers it automatically. Adding,
removing, or modifying a journey is self-contained to its folder (and the workflow's journey list) —
it should not touch this README.

Planned journeys to grow coverage are tracked in
[`plans/dx-e2e-journey-roadmap.md`](../plans/dx-e2e-journey-roadmap.md).
Expand Down
9 changes: 9 additions & 0 deletions e2e/journeys/01-basic-init/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# 01-basic-init

The fast, fully offline gate: scaffold the default project (`trix init`), validate it through tx3c
(`trix check`), and compile it to the Transaction Invocation Interface (`trix build`).

- **Scope:** compile/lower. No devnet, no secrets, no network beyond the one-time install.
- **Channels:** runs everywhere (no `tx3c` floor).

The real devnet round-trip lives in `04-devnet-roundtrip`.
29 changes: 2 additions & 27 deletions e2e/journeys/01-basic-init/journey.sh
Original file line number Diff line number Diff line change
@@ -1,20 +1,11 @@
#!/usr/bin/env bash
#
# Journey 01 — basic init.
#
# The canonical "zero to a working transaction" journey on the default scaffold,
# fully offline (no secrets, no live network): scaffold a project, validate it,
# build it, and run
# a real devnet round-trip. The `trix test` step is the integration centerpiece
# — it spins a local Dolos devnet, restores deterministic cshell wallets, submits
# the scaffolded transfer transactions, and asserts the resulting balances —
# exercising trix + tx3c + dolos + cshell + the resolver together.
#
# Journey 01 — basic init. See README.md for what this covers.
# Run via e2e/run.sh, which provides $TRIX and an isolated working directory.

source "${E2E_LIB:?E2E_LIB not set — run this journey via e2e/run.sh}"

journey_begin "01-basic-init" "init → check → build → test (offline devnet round-trip)"
journey_begin "01-basic-init" "init → check → build (offline scaffold validation)"

# 1. Scaffold a fresh project and confirm the expected files land.
run_cmd "trix init -y — scaffold a new project" "${TRIX}" init -y
Expand All @@ -32,20 +23,4 @@ assert_output_contains "check passed"
run_cmd "trix build — compile to TII" "${TRIX}" build
assert_found "TII artifact produced under .tx3/tii" ".tx3/tii" "main.tii"

# 4. Devnet round-trip. The scaffolded transfers must resolve and submit, and
# the final balances must match. The submit half works; the balance-assertion
# half is a *known, tracked toolchain bug*: `trix test`'s expect phase calls
# `cshell wallet utxos "@bob"` with the literal placeholder instead of the
# wallet name `bob` (the transaction path strips the `@`, the expect path in
# trix's commands/expect.rs does not), so cshell errors with the signature
# below. Reproduces on both stable (trix 0.25.1) and beta (0.26.0). Until trix
# strips the `@`, this step is an xfail — it auto-promotes to a hard failure the
# moment the bug is fixed (see xfail_cmd). The submit half is still asserted
# strictly, so a regression there is caught regardless.
xfail_cmd "trix test — devnet round-trip" "CShell failed to get wallet utxos" \
"${TRIX}" test tests/basic.toml
assert_output_contains "Dolos daemon started" "devnet came up"
assert_output_contains "bob sends 2 ada to alice" "transfer #1 was driven"
assert_output_contains "alice sends 2 ada to bob" "transfer #2 was driven"

journey_end
13 changes: 13 additions & 0 deletions e2e/journeys/02-lang-tour/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# 02-lang-tour

Coverage journey: swap the default scaffold's `main.tx3` for a deliberately feature-dense protocol
(`main.tx3` here, a copy of the lang's `lang_tour` example) and push it through the full compile +
lower pipeline (`check → build → inspect tir`). Exercises the breadth of the language surface — env
block, records, variants, lists/maps/tuples, policies & assets, record spread, locals, and the whole
Cardano construct set (mint/burn, collateral, reference, withdrawal, certificates, plutus/native
witnesses, treasury donation, metadata, validity).

- **Scope:** compile/lower only — the tx references hard-coded UTxOs, mints, and plutus scripts, so it
can't resolve against a fresh devnet (the round-trip lives in `04-devnet-roundtrip`).
- **Fixture:** `main.tx3`.
- **Capability:** tuples need `tx3c >= 0.22` (`#@ min-tx3c: 0.22.0`), so it's skipped on older channels.
18 changes: 1 addition & 17 deletions e2e/journeys/02-lang-tour/journey.sh
Original file line number Diff line number Diff line change
@@ -1,24 +1,8 @@
#!/usr/bin/env bash
#
# Journey 02 — language tour.
#
# Coverage journey: swap the default scaffold's main.tx3 for a deliberately
# feature-dense protocol (this directory's main.tx3, a copy of the lang's
# lang_tour example) and push it through the full compile + lower pipeline. It
# exercises the breadth of the language surface — env block, records, variants,
# lists/maps/tuples, policies & assets, record spread, locals, and the whole
# Cardano construct set (mint/burn, collateral, reference, withdrawal, the
# certificate forms, plutus/native witnesses, treasury donation, metadata,
# validity) — that the basic-init journey never touches.
#
# Scope is compile/lower only: the tx references hard-coded UTxOs, mints, and
# plutus scripts, so it cannot resolve against a fresh devnet — that's why there
# is no `trix test` step here (the round-trip lives in 01-basic-init).
#
# Journey 02 — language tour. See README.md for what this covers.
# Run via e2e/run.sh, which provides $TRIX and an isolated working directory.
#
# Capability: tuples (Tuple<…> types, literals, and indexing) require tx3c >= 0.22,
# so this journey is skipped on older channels (e.g. stable's 0.21).
#@ min-tx3c: 0.22.0

source "${E2E_LIB:?E2E_LIB not set — run this journey via e2e/run.sh}"
Expand Down
10 changes: 10 additions & 0 deletions e2e/journeys/03-lang-edge/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# 03-lang-edge

Coverage journey focused on the *newest* language additions, complementing 02-lang-tour's broad
surface: user-defined functions (`fn` with `let`), the `*` and `/` operators, parametric tuples
(`Tuple<…>` types, literals, and indexing), and `///` doc-comments. Swaps in this directory's
`main.tx3` and pushes it through `check → build → inspect tir`.

- **Scope:** compile/lower only (no devnet round-trip).
- **Fixture:** `main.tx3`.
- **Capability:** needs `tx3c >= 0.22` (`#@ min-tx3c: 0.22.0`); skipped on older channels (e.g. stable's 0.21).
12 changes: 1 addition & 11 deletions e2e/journeys/03-lang-edge/journey.sh
Original file line number Diff line number Diff line change
@@ -1,16 +1,6 @@
#!/usr/bin/env bash
#
# Journey 03 — language edge.
#
# Coverage journey focused on the *newest* language-level additions, the
# complement to 02-lang-tour's broad surface: user-defined functions (`fn` with
# `let`), the `*` and `/` operators, parametric tuples (`Tuple<…>` types,
# literals, and indexing), and `///` doc-comments. Swaps in this directory's
# feature-dense main.tx3 and pushes it through compile + lower.
#
# Scope is compile/lower only (no devnet round-trip). These features need
# tx3c >= 0.22, so the journey is skipped on older channels (e.g. stable's 0.21).
#
# Journey 03 — language edge. See README.md for what this covers.
# Run via e2e/run.sh, which provides $TRIX and an isolated working directory.
#
#@ min-tx3c: 0.22.0
Expand Down
18 changes: 18 additions & 0 deletions e2e/journeys/04-devnet-roundtrip/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# 04-devnet-roundtrip

The integration centerpiece: scaffold the default project and run a real `trix test`. It spins a
local Dolos devnet, restores deterministic cshell wallets, submits the scaffolded transfers, and
asserts the resulting balances — exercising trix + tx3c + dolos + cshell + the resolver together.

- **Scope:** runtime (needs a working devnet). No secrets, no live network beyond the one-time install.
- **Channels:** runs everywhere (no `tx3c` floor), but the CI workflow currently schedules it on the
**beta** job only.

## Currently failing on released channels — intentionally

The balance-assertion phase hits a known, tracked trix bug (the expect path queried the wrong cshell
store, passed the `@bob` placeholder, and parsed a mismatched utxo shape). Fixed on trix `main`
(tx3-lang/trix#123) but not yet in a released channel, so this journey **fails** on released binaries.
The assertion is kept **strict** (not a tolerated `xfail`) so the broken round-trip is a real,
visible failure. It goes green automatically once the fix ships to a channel; at that point add this
journey to the `stable` job too.
25 changes: 25 additions & 0 deletions e2e/journeys/04-devnet-roundtrip/journey.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
#!/usr/bin/env bash
#
# Journey 04 — devnet round-trip. See README.md for what this covers (and why
# it currently fails on released channels).
# Run via e2e/run.sh, which provides $TRIX and an isolated working directory.

source "${E2E_LIB:?E2E_LIB not set — run this journey via e2e/run.sh}"

journey_begin "04-devnet-roundtrip" "init → test (real devnet round-trip)"

# 1. Scaffold a fresh project. `trix test` compiles the TII itself, so no
# separate build step is needed here (01-basic-init covers check/build).
run_cmd "trix init -y — scaffold a new project" "${TRIX}" init -y
assert_exists "tests/basic.toml" "test scenario scaffolded"

# 2. Devnet round-trip: the scaffolded transfers must resolve, submit, and the
# final balances must match — asserted strictly (must print "Test Passed").
# This currently fails on released channels (known trix bug; see README.md).
run_cmd "trix test — devnet round-trip" "${TRIX}" test tests/basic.toml
assert_output_contains "Dolos daemon started" "devnet came up"
assert_output_contains "bob sends 2 ada to alice" "transfer #1 was driven"
assert_output_contains "alice sends 2 ada to bob" "transfer #2 was driven"
assert_output_contains "Test Passed" "balances asserted — round-trip green"

journey_end
Loading