Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 46 additions & 0 deletions .changeset/v1.0.0.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
"@attest/cli": major
"@attest/core": major
"@attest/detectors-ts": major
"@attest/diff": major
"@attest/runner": major
"@attest/schema": major
"@attest/symbols": major
---

# attest v1.0.0

First public release. Deterministic, locally-runnable CLI that verifies AI agent
manifests against actual diffs and outcomes — no LLM in the verification path.

**What's in v1.0**

- **Three verifier families** (SPEC §3): declared-change verification,
undeclared-change detection, outcome verification (build/test/lint execution in
worktree isolation).
- **Language-agnostic structural verification** via tree-sitter for TypeScript,
Python, and Go.
- **Closed claim taxonomy** (`file_change`, `symbol_added/removed/modified`,
`test_added/modified`, `outcome`); semantic claims return `unverifiable` with
an LLM-review pointer — never a heuristic.
- **Worktree-isolated runner** for `outcome` claims; commands never touch the
live working tree.
- **Fixture corpus** (SPEC §10): 21 cases across TS, Python, Go covering the
full oracle (honest, lying, partial, undeclared, allowlisted, outcome-fail,
behavioral). CI runs the corpus on every commit.
- **Detectors-ts demoted** to opt-in, best-effort plugin (does not affect exit
code; out of CI gating).
- **Apache-2.0** licensed.

**Packages (7)**

- `@attest/schema` — manifest, verdict, audit JSON Schema + ajv validator
- `@attest/diff` — unified diff parser
- `@attest/symbols` — tree-sitter symbol extraction
- `@attest/core` — verification engine + undeclared detection
- `@attest/runner` — worktree-isolated outcome execution
- `@attest/cli` — `attest verify` and `attest schema` commands
- `@attest/detectors-ts` — opt-in authentication detector (advisory only)

See `docs/SPEC.md` for the full v1.0 contract and `README.md` for a 20-minute
zero-to-first-verdict quickstart.
87 changes: 87 additions & 0 deletions .github/workflows/attest-fixture.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
name: attest

# Live documentation. The fixture workflow materialises a small TypeScript
# repo, materialises one honest and one lying manifest, and asserts that the
# honest case is green and the lying case is red. The same fixtures live in
# corpus/ts/cases/ — this workflow just wires them into CI.

on:
push:
branches: [main]
pull_request:

permissions:
contents: read

jobs:
verify-honest:
name: honest (expect pass)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Materialise fixture repo
shell: bash
run: |
mkdir -p /tmp/attest-demo
cp -a "$GITHUB_WORKSPACE/corpus/ts/base/." /tmp/attest-demo/
cd /tmp/attest-demo
git init -q
git config user.email "ci@example.com"
git config user.name "ci"
git add -A
git commit -qm "base"

- name: Run attest
uses: ./
with:
manifest: corpus/ts/cases/honest/manifest.json
diff: corpus/ts/cases/honest/change.diff
repo-root: /tmp/attest-demo
format: human

verify-lying:
name: lying (expect fail)
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Materialise fixture repo
shell: bash
run: |
mkdir -p /tmp/attest-demo
cp -a "$GITHUB_WORKSPACE/corpus/ts/base/." /tmp/attest-demo/
cd /tmp/attest-demo
git init -q
git config user.email "ci@example.com"
git config user.name "ci"
git add -A
git commit -qm "base"

- name: Run attest (lying manifest — expect non-zero exit)
id: attest
continue-on-error: true
uses: ./
with:
manifest: corpus/ts/cases/lying/manifest.json
diff: corpus/ts/cases/lying/change.diff
repo-root: /tmp/attest-demo
format: human

- name: Assert non-zero exit
if: steps.attest.outputs.exit-code == '0'
run: |
echo "expected non-zero exit on the lying manifest, got 0"
exit 1

- name: Assert result=fail
if: steps.attest.outputs.result != 'fail'
run: |
echo "expected result=fail, got ${{ steps.attest.outputs.result }}"
exit 1

- name: Assert result=fail
if: steps.attest.outputs.result != 'fail'
run: |
echo "expected result=fail, got ${{ steps.attest.outputs.result }}"
exit 1
33 changes: 33 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -32,3 +32,36 @@ jobs:

- name: Test
run: pnpm test

corpus-acceptance:
needs: ci
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: pnpm/action-setup@v4
with:
version: 10

- uses: actions/setup-node@v4
with:
node-version: 20
cache: pnpm

- uses: actions/setup-python@v5
with:
python-version: "3.12"

- uses: actions/setup-go@v5
with:
go-version: "1.22"
cache: true

- name: Install dependencies
run: pnpm install --frozen-lockfile

- name: Build
run: pnpm build

- name: Corpus acceptance (SPEC §6.7)
run: pnpm --filter @attest/cli test -- corpus.test.ts
5 changes: 4 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
node_modules/
dist/
.changeset/*.md
*.tsbuildinfo
coverage/
.env
.package.json.dev
packages/cli/grammars/
packages/cli/*.tgz
attest-SPEC.md
13 changes: 13 additions & 0 deletions .prettierignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
**/dist/**
**/node_modules/**

# Corpus fixtures: base/ and overlay/ trees are byte-stable inputs that the
# generated change.diff files derive from — reformatting them would silently
# invalidate the diffs. The .diff files and shell tools have no prettier parser.
corpus/**/base/**
corpus/**/overlay/**
corpus/**/*.diff
corpus/tools/*.sh

# Vendored prebuilt tree-sitter grammars (binary wasm) — no parser, never format.
packages/symbols/grammars/**
54 changes: 42 additions & 12 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,64 @@
# Contributing to attest

## How to add a detector
## How to add a detector property

`@attest/detectors-ts` is the demoted, opt-in, advisory plugin layer (SPEC §6.5).
It is **not** part of the verdict path: nothing in `@attest/core` or
`@attest/cli` calls it, and `verdict.exit_code` is never computed from its
output. A "detector" here means a best-effort advisory that surfaces a
human-signal in review.

1. Create `packages/detectors-ts/src/<property>/` directory.
2. Implement the detector class implementing the `Detector` interface from `detector.ts`.
3. Add per-framework modules following the pattern in `src/authentication/`.
4. Register your detector in `registerDetectors()` in `src/detector.ts`.
5. Write a fixture suite following the instructions below — minimum 4 fixtures per framework supported.
6. Ensure `pnpm --filter @attest/detectors-ts test` exits 0 with ≥85% line coverage on your new module.
2. Implement a function returning `DetectorOutput` — never a verdict. The
public type lives in `packages/detectors-ts/src/types.ts`:
- `status: "advisory_present" | "advisory_absent" | "advisory_inconclusive"`
- `warnings` is always `DETECTOR_WARNINGS` so the advisory nature is visible
3. Add per-framework modules following the pattern in
`src/authentication/{framework,chain,classify}.ts`.
4. Wire your property into `runDetectors` (`src/run-detectors.ts`). Add a
`findRoutesInFile`-like enumerator for whatever targets the property cares
about, then call your function per target.
5. Export your function from `src/index.ts` and tag every output's `detector`
field with a stable, lowercased identifier (e.g. `"authentication"`).
6. Write a fixture suite following the instructions below — minimum 4 fixtures
per framework supported.
7. Ensure `pnpm --filter @attest/detectors-ts test` exits 0 with ≥85% line
coverage on your new module.

> **Hard rule:** never let your detector's output flow into a
> `ClaimResult.status` (`verified` / `failed` / `unverifiable`). Those three
> values are owned by `@attest/core` and form the closed verdict taxonomy
> (SPEC §4.2). Re-introducing semantic verdicts at the detector layer is what
> killed the v0.1 attempt — do not do it.

## How to add a fixture

Fixtures live in `packages/detectors-ts/fixtures/<property>/`.

1. Create `<fixture-name>.ts` — a minimal TypeScript file that exercises the specific case.
2. Create `<fixture-name>.expected.json` with the expected verdict:
1. Create `<fixture-name>.ts` — a minimal TypeScript file that exercises the
specific case.
2. Create `<fixture-name>.expected.json` with the expected advisory shape:
```json
{
"verdict": "verified",
"reason_code": null,
"evidence_contains": ["authMiddleware", "Layer 1"]
}
```
`evidence_contains` is an array of strings — each must appear in at least one evidence entry's `note`.
3. Run `pnpm --filter @attest/detectors-ts test` and confirm the new fixture passes.
4. Never mark fixtures as "todo" or skip them. Every fixture must pass before merging.
The `verdict` field is the v0.1 vocabulary — it is translated to the
current `DetectorStatus` at test time (`verified`→`advisory_present`,
`unverified`→`advisory_absent`, `partial`→`advisory_inconclusive`).
`evidence_contains` is an array of strings — each must appear in at least
one evidence entry's `note`.
3. Run `pnpm --filter @attest/detectors-ts test` and confirm the new fixture
passes.
4. Never mark fixtures as "todo" or skip them. Every fixture must pass before
merging.

## Commit conventions

Follow conventional commits: `feat(scope): message`, `fix(scope): message`, `test(scope): message`.
Follow conventional commits: `feat(scope): message`, `fix(scope): message`,
`test(scope): message`.

Scope is the package short name: `schema`, `core`, `detectors-ts`, `cli`.

Expand Down
Loading
Loading