Skip to content

feat: orca CLI (verify, audit, checks)#1

Merged
aisona-lab merged 8 commits into
mainfrom
feat/orca-cli
Jun 17, 2026
Merged

feat: orca CLI (verify, audit, checks)#1
aisona-lab merged 8 commits into
mainfrom
feat/orca-cli

Conversation

@aisona-lab

Copy link
Copy Markdown
Owner

Summary

Adds an orca command-line tool that wraps the existing verification engine, so the library can be used without writing Python. Same product, wider reach; no new core dependencies (YAML is an optional [cli] extra).

  • orca verify [OUTPUT] -c CONFIG [--source ...] [--json] — load a JSON/YAML check config, read the output from a file or stdin, run from_config(...).check(...), print a per-check table or --json. Exit 0 pass / 1 verification failed / 2 usage error.
  • orca audit verify|export LEDGER [-o OUT] — verify or export a Provenance hash chain. Fully offline, no API key.
  • orca checks — list registered checks (including plugins), marking judge-requiring ones with *.
  • Judge auto-detected from env (ANTHROPIC_API_KEYOPENAI_API_KEYORCA_JUDGE_BASE_URL) via a single seam, judge_from_env; offline checks run with zero setup.
  • Errors at the config/ledger boundary (unknown check, missing checks key, malformed/locked-out YAML, missing ledger) map to a clean exit 2 with a message, never a traceback.
  • Version bumped 0.1.0 → 0.2.0; README documents the CLI.

Implementation notes

  • Stdlib argparse, one new module src/orcaverify/cli.py (~130 lines). Built TDD across 8 commits.
  • judge_from_env is the only code touching real model SDKs, so every test monkeypatches it — the suite stays fully offline.
  • Reviewed end-to-end; all final-review findings (error-handling exit codes, plugin loading in verify, missing-ledger false-positive) addressed in 590c6e4.

Test Plan

  • pytest -q — 104 passed
  • ruff check . clean, ruff format --check . clean
  • cli.py 95% covered (only the real-SDK judge constructors uncovered, by the no-network rule)
  • pip install -e . then orca checks / orca verify / orca audit work end-to-end
  • Reviewer: confirm exit-code contract (0/1/2) matches README

Final-review fixes: unknown check name, config without 'checks', missing
pyyaml, and malformed YAML now exit 2 with a clear message instead of an
uncaught traceback. verify() now loads plugins so configs can reference
third-party checks. audit rejects a missing ledger instead of reporting a
vacuous 'intact'. Adds type hints on judge_from_env/main.
@aisona-lab aisona-lab merged commit a1fe56c into main Jun 17, 2026
2 checks passed
@aisona-lab aisona-lab deleted the feat/orca-cli branch June 18, 2026 08:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant