feat(ci): rust pipeline with native-CLI hosting and binary distribution#25
Open
ob-aion wants to merge 36 commits into
Open
feat(ci): rust pipeline with native-CLI hosting and binary distribution#25ob-aion wants to merge 36 commits into
ob-aion wants to merge 36 commits into
Conversation
- rust/base composite: resolve the toolchain from rust-toolchain.toml, cache cargo + target, run an optional ci/setup.sh native-dep hook, then fmt --check, clippy -D warnings, test — zero-input, mirroring javascript/base - rust-packages.yml reusable workflow: preflight matrix (ubuntu/macos/windows), cargo-deny when deny.toml is present, tag-driven publish (pin Cargo.toml to the tag, generate-changelog, cargo publish to crates.io, github-release, commit-back, rolling vN tag), security — reuses the transverse release and security actions - README: document the pipeline, the rust/base action, and the ci/setup.sh convention
…y-chain job - drop the supply-chain (cargo-deny) job: the npm pipeline has no counterpart (preflight/publish/security only), it partly overlapped security.yml's osv-scanner, and it recompiled cargo-deny on every push — consumers add their own when needed - rust/base: cache only the ~/.cargo deps (drop target, a stale-build risk), terse one-line description matching javascript/base, drop the verbose hook comment - publish: cargo publish --no-verify (the tag is main HEAD, already built and tested by preflight) — removes the duplicated ci/setup.sh hook and the cold build on release - header, name, and permission-comment wording aligned with javascript-npm-packages; README drops the cargo-dist aside and documents the CARGO_REGISTRY_TOKEN secret
I was wrong to drop cargo-deny as "redundant": osv-scanner only scans for known vulnerabilities. cargo-deny owns the controls nothing else here covers — crate sources (crates.io only), licenses, banned/wildcard deps, and unmaintained or yanked advisories. Restored, mapped to the GitLab image-layer hardening model. - supply-chain job: cargo-deny check via the SHA-pinned EmbarkStudios action, on every branch push, reading the repo's deny.toml - rust/base: clippy and test run --locked (committed Cargo.lock, fails on drift) — the lock-pin control; deny.toml + committed Cargo.lock are now consumer requirements - README Security: a "Supply chain — Rust" section mapping each GitLab control (cooldown, firewall, no-scripts, pin) to its Rust analog, the baseline deny.toml, and two accepted residual risks (build.rs runs; crates.io has no publish cooldown)
…+ policy - javascript/base: install Socket Firewall (sfw, the free tokenless build) and run the install through it (sfw pnpm install) — blocks confirmed-malicious packages at the registry fetch, the GitHub-runner equivalent of the GitLab image-baked firewall; fail-closed if sfw cannot install or run - README Security: Firewall and Cooldown sections; the cooldown (minimumReleaseAge, @coroboros/* excluded) belongs in pnpm-workspace.yaml on pnpm 11, not .npmrc - SECURITY.md: vulnerability-reporting policy, matching the GitLab repo
ob-aion
changed the title
Reusable workflows run in the caller's context, so "Move rolling major tag" force-pushed a meaningless vN ref into every consumer package repo. Packages release x.y.z; the @vn ref to coroboros/ci belongs to its own release path. This matches the GitLab model — ADR-0005 keeps rolling tags image-only and gives packages no rolling ref. Commit-back was duplicated inline across the npm and rust publish jobs. Extract it into release/commit-artifacts (files input), mirroring GitLab's commit-release-artifacts template, with [skip ci] so the bot's main push no longer re-triggers the pipeline.
Each scanner (gitleaks, osv-scanner, cargo-deny) becomes a security/* composite, so osv-scanner is defined once and reused by both security.yml and the package supply-chain gates. osv-scanner now runs only when a supported manifest is present, so a dependency-less repo wiring in security.yml skips the scan instead of failing on osv's no-manifest error. self-security.yml references the composites via local ./ refs: a reusable workflow resolves ./ against the caller, so security.yml must pin @v0 while self-CI exercises the in-repo composites.
crates.io Trusted Publishing (OIDC) via rust-lang/crates-io-auth-action is the default; CARGO_REGISTRY_TOKEN is the bootstrap for a new crate's first publish. Drop --no-verify so the verify build compiles the packaged tarball and catches a crate that only builds in-workspace before an immutable release. publish needs the supply-chain job, so cargo-deny gates the release rather than scanning in parallel. The ci/setup.sh hook moves to rust/native-deps, shared by rust/base and the publish verify build.
osv-scanner ran only in the parallel security job, so a known vulnerability could not block a release. A supply-chain job now runs it on every push and publish needs it.
The previous mover ran inside the reusable publish job, in the consumer's context, and force-pushed a meaningless vN ref into every package repo. Moving v0 belongs to coroboros/ci's release path; self-release.yml does it on each stable X.Y.Z tag.
Document the Rust pipeline, the supply-chain gates, the security composites, the manifest-gated osv scan, and the v0 automation. Apply the Coroboros brand voice to the README prose. Bump package.json to 0.2.0, resolving the lag behind the 0.1.14 tag, and prepend the CHANGELOG section.
osv-scanner can't parse the binary bun.lockb (only the text bun.lock), so a bun.lockb-only repo tripped the gate: detect fires, osv resolves nothing, exits 128, the job fails spuriously — the exact case the manifest gate exists to avoid. Add pdm.lock (PDM, osv-supported) so the set matches coroboros/security/ci one-to-one (13 lockfiles).
Add a secrets job (gitleaks composite, full history) to the npm and Rust package workflows; publish now needs [supply-chain, secrets]. A leaked secret blocks the release through the template's needs: graph, not the consumer's branch protection — parity with the GitLab security-gate stage.
Symmetry with the gitleaks row — osv-scanner also runs in self-security.yml.
Opt-in via [package.metadata.dist] in the consumer's Cargo.toml. A tagged binary repo gets prebuilt archives per target, shell/powershell installers, a Homebrew formula in the declared tap, and an npm shim — attached to the one GitHub Release the pipeline already creates, alongside the crates.io publish. The shared pipeline stays the sole release authority: dist only builds (final asset URLs derive from repo + tag), and the release goes live via draft → undraft so installers and the formula resolve against published URLs. Library crates without dist metadata are unchanged — every binary job self-skips. Adds dist-plan/build/host/publish (binary builds need the cargo-deny and gitleaks gates), the release/dist install composite (dist 0.32.0, cargo install --locked), and a draft input on release/github-release. Bumps to 0.3.0.
The cargo set-version tag pin was inlined in four jobs — publish plus the three dist jobs. Extract it to rust/pin-version, called from each: one source for the pin, consistent with the repo's composite-per-shared-step shape. Docs: tighten the README binary-distribution section, cut the duplicated artifact list (the details block is the one home), drop a stale needs: list and a misplaced cargo-dist pinning note, and fix the cfg(target_os) example. Record release/dist + rust/pin-version in CLAUDE.md and the changelog.
House style pins every tooling binary by version — gitleaks, actionlint, cargo-dist. cargo-edit was the last unpinned one. 0.13.11 via env, matching the release/dist install.
The supply-chain gate read the consumer's deny.toml, so a repo could weaken or omit it. Ship a canonical security/deny.toml and pass it via --config (the gitleaks model): the consumer's deny.toml is ignored, and a project-local deny.exceptions.toml — cargo-deny's one remaining license escape hatch — fails the job. The ruleset hard-fails on vulnerability, yanked, unmaintained, and unsound advisories, restricts sources to crates.io (git + alternative registries denied), and denies wildcard version requirements. Validated against cargo-deny 0.19.8.
The supply-chain job blurb still said cargo-deny "reads deny.toml"; the risk table omitted unsound; the Security prose re-listed the controls the table already covers. Point both at the imposed security/deny.toml and drop the duplication.
Remove workflow/composite comments that restate what the code does or duplicate the README: the cargo-deny / gitleaks / osv job-purpose blocks (and a stale "reads the repo's deny.toml" line), the packages_install and npm-auth notes. Trim the verbose ones to their why — the Socket Firewall, verify-build, undraft-ordering, and install_dist comments. Inline scope justifications, the pnpm/corepack workaround, and the URL-determinism rationale stay.
Inline the dist setup (mimic the npm pipeline) — drop the release/dist and rust/pin-version composites; the `cargo install cargo-dist` / `cargo-edit` + `cargo set-version` steps live in the dist jobs and publish, with tool versions hoisted to workflow env (CARGO_DIST_VERSION, CARGO_EDIT_VERSION). Logs use gh-native commands per context: verify-tag detail folded into one `::error::`, status lines to `::notice::`, the check-docs context dump into a `::group::`. Trim the gitleaks and self-security comments to one line.
0.2.0 was never tagged — the latest release is 0.1.14 — so the whole rust-packages PR ships as one version. Revert the second bump and merge the binary-distribution and imposed-cargo-deny notes into the v0.2.0 changelog.
…bumps - rust/install-dist: cargo-dist installed prebuilt + SHA-256 verified per OS, replacing the from-source compile in dist-plan/dist-build/dist-host - rust/pin-version: install only the cargo-set-version binary (--bin), pin in-composite - pin dist-plan/dist-build/dist-host checkouts to the tag commit; verify-tag stays on publish - guard cargo publish with a dirty-tree allowlist before the immutable crates.io release - security/cargo-deny: drop redundant CLI deny flags (single-sourced in deny.toml) - self-actions: PR smoke tests for verify-tag, generate-changelog, commit-artifacts, cargo-deny, install-dist against synthetic fixtures - renovate.json: review-gated auto-bumps for the version-pinned tooling
Reserved GITHUB_* env vars (GITHUB_SHA/GITHUB_REF_NAME) can't be overridden into a composite's run steps — the runner re-injects the canonical value — so the fixture- with-overrides smokes fed the composites the real PR ref and failed. Run them against the real checkout instead (HEAD == GITHUB_SHA on PR/push) and cover only the non-tag- reachable logic; tag-driven paths stay validated at release.
Run Renovate as a scheduled, SHA-pinned Action (PAT in RENOVATE_TOKEN) instead of the Mend app — no third-party app, and postUpgradeTasks can run. A postUpgradeTask script (allowlisted via RENOVATE_ALLOWED_COMMANDS) re-syncs each tool's tarball SHA-256 to the bumped version in the same PR, so the version pin and its checksum never drift.
Three seams plus a release-ordering fix let a native/C++ CLI ride the shared pipeline without it learning zig, clang, or musl: - dist-build exports CARGO_DIST_TARGET to the target-aware ci/setup.sh hook; host preflight sees it empty. - rust/test-deps loads ci/test.env into the job env and runs ci/test-setup.sh before cargo test, so model/ffmpeg-gated tests fail loud instead of skipping. - Swatinem rust-cache caches ~/.cargo + target/ — per-target key on dist-build, default-branch save in rust/base. - publish gates on dist-build via !cancelled() + result != failure, so a failed binary build never publishes to crates.io; library crates still publish. Workflow concurrency serializes same-ref releases; the tap push rebase-retries. Also: explicit rustup toolchain install, --provenance on both npm-shim paths, and rename the secrets gate to secret-scan. self-actions smokes both new hooks.
- rust-packages concurrency keys cancel-in-progress on ref_type: superseded branch CI cancels, an in-flight tag/release never does — was never-cancel, which queued branch runs. - javascript-npm-packages: rename the `secrets` gate job to `secret-scan`, matching rust-packages and disambiguating it from the workflow's `secrets:` block; publish needs: follows. - docs: fix the rust/base step order (native-deps, fmt, clippy, then test-deps and test), reconcile the rust/base toolchain/cache CHANGELOG line with the shipped behavior, document the rustup-on-PATH contract for custom dist-build containers, set the v0.2.0 date.
- rust-packages concurrency: key the group per repo on tags (was per ref), so two release tags of one repo serialize instead of racing release/commit-artifacts' push to main — a rebase-retry there would conflict on the Cargo.toml version line. Branch CI unchanged (per-ref, cancels superseded runs). - self-actions: run the install-dist smoke on ubuntu + macos + windows (was ubuntu-only). cargo-dist ships the Windows zip flat (dist.exe at root) and the tarballs nested, so the per-OS extraction in rust/install-dist now has coverage on every path it claims to support; the windows/macos asset SHA-256 pins were checked against the real releases.
javascript-npm-packages had no `concurrency`, so two release tags of one repo could run their publish jobs concurrently and race release/commit-artifacts' push to main — one fails non-fast-forward, leaving npm published but the version bump uncommitted. Add the same ref-keyed group as rust-packages: tags serialize per repo, branches key per ref and cancel superseded CI. The two top-level blocks are duplicated because Actions can't share a concurrency key across reusable workflows.
Migrating the first binary consumer surfaced that cargo-dist 0.32 reads its
workspace-global keys (cargo-dist-version, ci, publish-jobs, allow-dirty) only from
[workspace.metadata.dist], and needs allow-dirty = ["ci"] so `dist plan` doesn't
require its own generated workflow (this pipeline owns it). Without both, a binary
consumer's dist-plan job fails — a panic ("couldn't find the release") or an
out-of-date-workflow error. State the requirement in the consumer contract.
dist-plan only grepped [package.metadata.dist], but cargo-dist 0.32 keeps its global keys in [workspace.metadata.dist] — a consumer using that layout would be misread as a library crate and skip the binary jobs. Match either table. The tap extraction already scans the whole file, so it resolves from either.
Drop a redundant concurrency restatement in the npm changelog entry, fold the duplicated allow-dirty mention in the binary-distribution contract, and reword the install-dist smoke comment to why-only.
ob-aion
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
rust-packages.yml): preflight matrix (Linux, macOS, Windows), acargo-denysupply-chain gate, a packaged-tarball verify build, and tag-driven crates.io publish via OIDC.CARGO_REGISTRY_TOKENbootstraps a new crate's first publish.ci/setup.sh(target-aware viaCARGO_DIST_TARGET),ci/test.env+ci/test-setup.shtest hooks, andrust-cacheover~/.cargo+target/.distonly builds.publishgates ondist-buildso a failed binary build never reaches crates.io; releases serialize per repo; a commit-back allowlist and a tap rebase-retry close the remaining races.gitleaks/osv-scanner/cargo-denycomposites as the single source behind thesecret-scanandsupply-chaingates, an imposedsecurity/deny.toml, and Socket Firewall onjavascript/base;self-actionssmokes every composite on PRs andself-releasemoves the rollingv0. Version0.2.0.Test plan
actionlint -shellcheck=shellcheckandyamllint -c .yamllint .exit 0self-actionssmokes pass, includinginstall-diston Linux, macOS, and Windowsdist planresolves with[workspace.metadata.dist]+allow-dirty = ["ci"]; the first tag builds its native targets and attaches installersdist-buildskipspublish(no crates.io); a library crate still publishessupply-chainand blocks the tagged publish