Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
# Copy this file to .env and fill in your values, then run `docker compose up`.
# .env is gitignored — never commit your real key.

# Required: Demeter utxorpc API key for the mainnet endpoint in deploy/tracker.toml.
# Get a free key at https://demeter.run. The tracker fails to authenticate without it.
DMTR_API_KEY=

# Optional: host port the dashboard is published on (container always listens on 3000).
# PORT=3000

# Optional: tracker log level (error | warn | info | debug | trace).
# RUST_LOG=info
37 changes: 37 additions & 0 deletions .github/workflows/docker-dashboard.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
name: Publish dashboard image to GHCR

permissions:
contents: read
packages: write

on:
workflow_dispatch: {}

jobs:
build:
name: Build and push multi-arch image
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6

- name: Set up QEMU
uses: docker/setup-qemu-action@v4

- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v4

- name: Login to GitHub Container Registry
uses: docker/login-action@v4
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}

- name: Build and push
uses: docker/build-push-action@v7
with:
context: ./frontend
platforms: linux/amd64,linux/arm64
push: true
tags: ghcr.io/tx3-lang/dashboard-frontend:${{ github.sha }}
12 changes: 7 additions & 5 deletions .github/workflows/frontend-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,15 +22,17 @@ jobs:
working-directory: ./frontend/
steps:
- name: Checkout
uses: actions/checkout@v4
uses: actions/checkout@v6

- name: Setup pnpm
uses: pnpm/action-setup@v2
uses: pnpm/action-setup@v6
with:
version: 10
# Version comes from the packageManager field in frontend/package.json.
# (Action inputs don't honor working-directory, so point at the file.)
package_json_file: frontend/package.json

- name: Setup Node.js
uses: actions/setup-node@v4
uses: actions/setup-node@v6
with:
node-version: '24'
cache: 'pnpm'
Expand All @@ -49,4 +51,4 @@ jobs:
run: pnpm run check

- name: Run tests
run: pnpm test --run
run: pnpm test --run
33 changes: 28 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,9 +18,21 @@ Rel(dashboard, utxorpc, "Subscribes to tx stream", "gRPC / TLS")
Rel(operator, registry, "Pulls TII once at vendor time", "GraphQL")
```

## Quick start
## Run with Docker

You need a clone of [`tx3-lang/tx3-lift`](https://github.com/tx3-lang/tx3-lift) as a sibling directory and a Demeter `utxorpc` API key.
The fastest way to get the full stack running. You need Docker (with Compose v2) and a Demeter `utxorpc` API key ([free sign-up at demeter.run](https://demeter.run)).

```bash
cp .env.example .env
# Open .env and set DMTR_API_KEY=dmtr_...
docker compose up
```

Open <http://localhost:3000>. The tracker streams Cardano mainnet and writes `tracker.db` into a Docker volume; the dashboard reads from it. Mainnet matches for the configured protocols typically appear within a few minutes.

## Quick start (from source)

You need a clone of [`tx3-lang/tx3-lift`](https://github.com/tx3-lang/tx3-lift) as a sibling directory, Rust stable, Node 24, and a Demeter `utxorpc` API key.

```bash
# Once: clone tx3-lift sibling
Expand All @@ -44,7 +56,9 @@ Open <http://localhost:3000>. See [`docs/running.md`](docs/running.md) for prere

## What the demo shows

The committed configuration tracks the [`buidler-fest/ticketing-2026`](protocols/buidler-fest/ticketing-2026.tii) protocol on Cardano preview. It defines a single transaction name (`buy_ticket`) with three parties (`buyer`, `treasury`, `issuer`) and roughly 80 real on-chain matches. The matches list at `/` shows every `buy_ticket` the tracker has seen, newest first; clicking through to `/txs/<hash>` shows the parties and their addresses for that transaction.
The Docker Compose stack (`deploy/tracker.toml`) tracks five DeFi protocols on **Cardano mainnet**: Indigo, VyFi, Bodega Market, Fluid Aquarium, and Strike Staking. Matching uses `mode = "best"`, which keeps only the highest-ranked candidate when a single transaction matches multiple TIIs. The matches list at `/` shows every matched transaction the tracker has seen, newest first; clicking through to `/txs/<hash>` shows the lifted parties and their addresses for that transaction.

The `protocols/buidler-fest/` directory contains the earlier preview demo TII for reference, but the active configuration targets mainnet.

## Documentation

Expand All @@ -57,14 +71,23 @@ The committed configuration tracks the [`buidler-fest/ticketing-2026`](protocols
```
dashboard/
├── README.md # this file
├── tracker.toml # config consumed by the external tracker
├── docker-compose.yml # tracker + dashboard via Docker
├── .env.example # DMTR_API_KEY and optional PORT/RUST_LOG
├── tracker.toml # bare-metal tracker config (api_key commented out)
├── deploy/
│ └── tracker.toml # Docker tracker config (mainnet, absolute paths)
├── docs/
│ ├── architecture.md
│ ├── access-patterns.md
│ └── running.md
├── protocols/
│ ├── indigo.tii
│ ├── vyfi.tii
│ ├── bodega_market.tii
│ ├── fluid-aquarium.tii
│ ├── strike-staking.tii
│ └── buidler-fest/
│ └── ticketing-2026.tii # vendored TII for the demo protocol
│ └── ticketing-2026.tii # earlier preview demo (reference only)
└── frontend/ # TanStack Start SSR app
├── package.json
└── src/
Expand Down
51 changes: 51 additions & 0 deletions deploy/tracker.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# Tracker configuration for the Docker Compose stack.
#
# Paths are ABSOLUTE on purpose: the tracker resolves tii_path relative to its
# working directory, and this file is consumed inside the tracker container
# where ./protocols and ./tracker.db do not exist. The compose mounts:
# - ./protocols -> /protocols (read-only bind mount, TII files)
# - ./deploy/tracker.toml -> /etc/tracker/tracker.toml (read-only bind mount)
# - tracker-data -> /data (named volume, shared with the dashboard)
#
# No api_key here: the key is supplied via the DMTR_API_KEY environment variable
# (set in .env). The tracker falls back to DMTR_API_KEY when api_key is absent.

[upstream]
endpoint = "https://cardano-mainnet.utxorpc-m1.demeter.run"
intersect = "tip"

# No upstream filter — every mainnet tx is forwarded; the fingerprint pre-filter
# plus the structural matcher decide which ones count.

[storage]
database_path = "/data/tracker.db"

# Several mainnet protocols are tracked at once, so keep only the highest-ranked
# candidate per transaction to avoid one tx matching multiple unrelated TIIs.
[matching]
mode = "best"

[[sources]]
name = "indigo-mainnet"
tii_path = "/protocols/indigo.tii"
profile = "mainnet"

[[sources]]
name = "vyfi-mainnet"
tii_path = "/protocols/vyfi.tii"
profile = "mainnet"

[[sources]]
name = "bodega-mainnet"
tii_path = "/protocols/bodega_market.tii"
profile = "mainnet"

[[sources]]
name = "fluid-aquarium-mainnet"
tii_path = "/protocols/fluid-aquarium.tii"
profile = "mainnet"

[[sources]]
name = "strike-staking-mainnet"
tii_path = "/protocols/strike-staking.tii"
profile = "mainnet"
72 changes: 72 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
# Docker Compose stack: tracker (writer) + dashboard (reader).
#
# Volume strategy:
# - Bind mounts (./protocols, ./deploy/tracker.toml) give the tracker read-only
# access to host-side config and TII files that operators may want to update
# without rebuilding the image.
# - Named volume (tracker-data) is the shared SQLite database. Using a named
# volume instead of a bind mount avoids UID/GID mismatches between the Rust
# and Node containers and lets Docker manage the lifecycle of the data.
#
# Startup ordering:
# The dashboard opens the DB with fileMustExist: true, so it must not start
# until the tracker has written at least one batch and created tracker.db.
# The healthcheck on the tracker service (test -f /data/tracker.db) gates the
# dashboard via depends_on: condition: service_healthy.

services:

tracker:
# Images are tagged with the git SHA by the "Publish tracker image to GHCR"
# workflow. Pin a published SHA here and bump it deliberately after each publish.
image: ghcr.io/tx3-lang/tx3-lift-tracker:GIT_SHA
restart: unless-stopped
environment:
# API key for the Demeter utxorpc mainnet endpoint — required.
DMTR_API_KEY: ${DMTR_API_KEY}
# Log level; defaults to info so operators don't have to set it explicitly.
RUST_LOG: ${RUST_LOG:-info}
volumes:
# Read-only bind mounts: config files that live in this repo.
- ./protocols:/protocols:ro
- ./deploy/tracker.toml:/etc/tracker/tracker.toml:ro
# Named volume: SQLite database shared with the dashboard.
- tracker-data:/data
# Pass the config path as the sole argument to the tracker binary (ENTRYPOINT).
command: ["/etc/tracker/tracker.toml"]
healthcheck:
# The tracker creates tracker.db on its first cursor advance, which can take
# up to ~30 s on a cold start before the upstream delivers the first block.
# We wait for the file to exist before considering the service healthy.
test: ["CMD-SHELL", "test -f /data/tracker.db"]
interval: 10s
timeout: 5s
retries: 12
start_period: 30s

dashboard:
# Images are tagged with the git SHA by the "Publish dashboard image to GHCR"
# workflow. Pin a published SHA here and bump it deliberately after each publish.
image: ghcr.io/tx3-lang/dashboard-frontend:GIT_SHA
restart: unless-stopped
environment:
# Path inside the container where the dashboard opens the SQLite database.
TRACKER_DB_PATH: /data/tracker.db
# The container always listens on 3000 (set via the image's ENV PORT).
# The host-side port is configurable below via the PORT variable.
volumes:
# Same named volume as the tracker. NOT read-only: SQLite WAL mode requires
# the reader to access -shm/-wal sidecar files, which can fail on a :ro mount.
- tracker-data:/data
ports:
# Publish on the host port set by PORT (default 3000); container port is fixed.
- "${PORT:-3000}:3000"
depends_on:
tracker:
# Do not start the dashboard until the tracker has created tracker.db.
condition: service_healthy

volumes:
# Named volume that holds tracker.db (and its WAL sidecar files).
# Docker manages its lifecycle; use `docker compose down -v` to remove it.
tracker-data:
65 changes: 52 additions & 13 deletions docs/running.md
Original file line number Diff line number Diff line change
@@ -1,28 +1,66 @@
# Running the dashboard

This guide walks you through running the tx3 Dashboard end-to-end against the committed `buidler-fest/ticketing-2026` demo on Cardano preview.
This guide walks you through running the tx3 Dashboard end-to-end against the mainnet DeFi demo (Indigo, VyFi, Bodega, Fluid Aquarium, and Strike on Cardano mainnet).

## Prerequisites

You'll need:

- **Rust stable** (for building and running the tracker from `tx3-lang/tx3-lift`).
- **Node 24** (the version Nitro and TanStack Start are tested against in CI).
- **pnpm 10** — install with `npm install -g pnpm@10` or `corepack enable`.
- **A sibling clone of [`tx3-lang/tx3-lift`](https://github.com/tx3-lang/tx3-lift)** at `../tx3-lift` (relative to this repo). The dashboard does not embed the tracker; you run the tracker binary from that clone as a sidecar.
- **A `utxorpc` endpoint and API key.** The committed `tracker.toml` points at Demeter's Cardano preview endpoint; sign up at [demeter.run](https://demeter.run) for a free `dmtr_…` key.
- **Rust stable** (for building and running the tracker from `tx3-lang/tx3-lift`) — only required for the from-source path.
- **Node 24** (the version Nitro and TanStack Start are tested against in CI) — only required for the from-source path.
- **pnpm 11** — install with `corepack enable` (honours the `packageManager` field) or `npm install -g pnpm@11` — only required for the from-source path.
- **A sibling clone of [`tx3-lang/tx3-lift`](https://github.com/tx3-lang/tx3-lift)** at `../tx3-lift` (relative to this repo) — only required for the from-source path. The dashboard does not embed the tracker; you run the tracker binary from that clone as a sidecar.
- **A Demeter `utxorpc` API key.** Sign up at [demeter.run](https://demeter.run) for a free `dmtr_…` key. The recommended way to supply the key is the `DMTR_API_KEY` environment variable — you do not need to edit `tracker.toml` (the committed file keeps `api_key` commented out for this reason).

## Running with Docker

The simplest way to run the full system is with Docker Compose. You need:

- **Docker with Compose v2** (`docker compose` as a subcommand, not `docker-compose`).
- **A Demeter `utxorpc` API key** (see Prerequisites above).

```bash
# 1. Copy the example env file and fill in your key.
cp .env.example .env
# Open .env and set DMTR_API_KEY=dmtr_...

# 2. Start the stack.
docker compose up
# Add -d to detach: docker compose up -d

# 3. Open the dashboard.
# http://localhost:3000
```

> **Pin an image SHA first.** Images are published to GHCR tagged with the git
> SHA (no `latest`). `docker-compose.yml` ships with a `GIT_SHA` placeholder —
> before the first run, replace it with a published SHA (run the "Publish … to
> GHCR" workflows to produce images), or build the images locally and tag them
> to match.

### How it works

The `tracker` service reads `deploy/tracker.toml` and the TII files from `protocols/` (both bind-mounted read-only into the container). It writes `tracker.db` into a named Docker volume (`tracker-data`). The `dashboard` service mounts the same named volume and reads the database. The dashboard waits for the tracker's healthcheck — which checks that `/data/tracker.db` exists — before starting, so you never see a `SQLITE_CANTOPEN` crash from a race at startup.

### Docker troubleshooting

- **Images not found** — the images are pulled from `ghcr.io/tx3-lang/tx3-lift-tracker` and `ghcr.io/tx3-lang/dashboard-frontend`, tagged with the git SHA. `docker-compose.yml` pins a specific SHA; make sure it points at a published tag. If `docker compose pull` fails, check the SHA is published and the packages are public in the GitHub org.
- **Named volume on a network filesystem** — SQLite WAL mode (`-wal` / `-shm` sidecar files) is not safe on NFS or other network-backed filesystems. The `tracker-data` named volume must reside on a local filesystem. Docker Desktop on macOS and Linux with the default local volume driver both satisfy this requirement.
- **Empty list at `/`** — the tracker needs to scan the tip of mainnet and find a matching transaction before the dashboard has anything to show. Mainnet matches for the configured protocols typically appear within a few minutes. Check `docker compose logs tracker` to confirm blocks are flowing in.
- **Stopping and data lifecycle** — `docker compose down` stops the containers but keeps the `tracker-data` volume (the database is preserved). `docker compose down -v` removes the volume and wipes all stored matches.

## Environment variables

| Variable | Used by | Required | Default | Purpose |
|----------|---------|----------|---------|---------|
| `DMTR_API_KEY` | Tracker | Yes | — | Demeter API key for the configured `utxorpc` endpoint. The tracker fails to start without it. |
| `TRACKER_DB_PATH` | Dashboard | No | `./tracker.db` | Path the dashboard opens read-only. Leave unset to read the file the tracker writes alongside `tracker.toml`. |
| `DMTR_API_KEY` | Tracker | Yes | — | Demeter API key for the configured `utxorpc` endpoint. Supply via `.env` for Docker, or export it in your shell for bare-metal. The tracker reads this variable directly when `api_key` is absent from `tracker.toml`; no wrapper script is needed. |
| `TRACKER_DB_PATH` | Dashboard | No | `./tracker.db` | Path the dashboard opens read-only. Leave unset to read the file the tracker writes alongside `tracker.toml`. In the Docker stack this is set to `/data/tracker.db` inside the container. |
| `RUST_LOG` | Tracker | No | (warn) | Log level for the tracker binary. `info` is comfortable for first-run; `debug` for protocol-level debugging. |
| `PORT` | Dashboard | No | `3000` | Host port the dashboard is published on. In Docker the container always listens on 3000; only the host-side binding uses this variable. |

## First run
## Running from source

You'll need two terminals. From a fresh clone of this repo:
If you prefer to build from source (or are developing the tracker or dashboard itself), you'll need two terminals. From a fresh clone of this repo:

```bash
# Once: clone the tx3-lift sibling
Expand All @@ -45,6 +83,8 @@ pnpm dev

Visit <http://localhost:3000>. The tracker writes `dashboard/tracker.db` (plus its `-wal` / `-shm` companion files); the dashboard reads from the same file. New matches appear after a page reload.

> **API key**: supply `DMTR_API_KEY` as an environment variable (shown above). Do not add it to `tracker.toml` — the committed file keeps `api_key` commented out so the key is never accidentally committed.

## Production build

For an operator-managed deployment without `vite dev`. From the repo root:
Expand All @@ -64,7 +104,7 @@ Set `TRACKER_DB_PATH` if `tracker.db` lives outside the working directory. The N

The dashboard renders "No matches yet — confirm the tracker is running." If you're seeing it indefinitely:

- Confirm the tracker terminal shows `Apply` events flowing in (or, on preview, that the protocol's policy filter actually matches recent on-chain activity).
- Confirm the tracker terminal shows `Apply` events flowing in; on mainnet, confirmed matches for the tracked protocols typically appear within minutes.
- Confirm `tracker.db` exists in the dashboard working directory and is non-empty: `sqlite3 tracker.db 'SELECT COUNT(*) FROM matches;'`.
- Reload the page — the MVP does not auto-refresh.

Expand Down Expand Up @@ -93,12 +133,11 @@ If the upstream chain rolls back past a slot the tracker had recorded, the track
## Tested with

- `tx3-lang/tx3-lift` tracker commit: `04d0b90` (`docs: add integration test report (#8)`)
- Node 24, pnpm 10, Rust stable.
- Node 24, pnpm 11, Rust stable.

## Deferred deployment polish

These are out of scope for M3 and tracked for follow-up iterations:

- **Docker compose** stack that runs both the tracker and the dashboard with a shared volume for `tracker.db`.
- **Service-manager units** (systemd / launchd / pm2 templates) committed alongside the repo for one-shot operator install.
- **Postgres backend** for the tracker (1–3 days upstream PR) plus the dashboard-side dialect swap (~half a day) — see [`architecture.md` § Forward-looking](architecture.md#forward-looking-postgres).
Loading