Auto-Jobs

Simulate your users.

Auto-Jobs is a CLI-first toolkit for simulated user testing and automated customer development. You describe your product through a JTBD map, generate virtual users grounded in that context, run structured interviews, and score a target site against the jobs and outcomes you care about.

What It Does Today

• Initializes a product-agnostic project scaffold with JTBD, interview, and test config
• Generates virtual personas from your actors, jobs, and desired outcomes
• Runs deterministic simulated interviews and writes markdown reports
• Runs browser-based simulated user tests with Playwright
• Can attach to an existing Chrome session via CDP for Chrome DevTools MCP-style control
• Captures screenshots during test runs and includes them in reports
• Uses codex or claude as the test harness for planning and visual analysis
• Supports single-harness or dual-harness consensus analysis for browser decisions and vision summaries
• Tracks explicit actor / planner / reviewer subagent provenance for each simulated run
• Supports scripted browser setup and journey steps for login, form fill, upload, keyboard, and assertion flows
• Falls back gracefully to deterministic heuristics when a harness is unavailable or times out
• Tracks score history over time
• Generates and serves a local single-file dashboard

Current MVP Scope

The current MVP is strongest at:

• JTBD scaffolding
• Persona generation
• Interview synthesis
• Browser-based site inspection and scoring
• Scripted authenticated flows and richer form interactions
• Harness-assisted screenshot analysis with deterministic fallback
• Codex / Claude harness switching
• Explicit simulated subagent metadata
• Local reports and dashboard views

Not built yet:

• Slack or webhook notifications

Install

Requires Node.js 20+.

npm install

Run the CLI directly:

node cli/index.js --help

Quick Start

Initialize a project:

node cli/index.js init --yes --force \
  --name "My Product" \
  --description "A tool for fast-moving teams" \
  --url "https://example.com" \
  --harness codex \
  --secondary-harness claude \
  --orchestration single \
  --actors "Primary User,Team Lead" \
  --stages "Discover,Set Up,Use"

Generate personas:

node cli/index.js generate-personas --count 4 --archetype all --vary tech_comfort,patience

Run interviews:

node cli/index.js interview

Run a scored test against a live URL:

node cli/index.js test --url https://example.com

Run a Playwright test with a specific harness:

node cli/index.js test \
  --runner playwright-agent \
  --harness codex \
  --url https://example.com

Run with Claude instead:

node cli/index.js test \
  --runner playwright-agent \
  --harness claude \
  --url https://example.com

Run with consensus orchestration:

node cli/index.js test \
  --runner playwright-agent \
  --harness codex \
  --secondary-harness claude \
  --orchestration consensus \
  --url https://example.com

Attach to an existing Chrome session:

node cli/index.js test \
  --runner chrome-devtools-agent \
  --cdp-url http://127.0.0.1:9222 \
  --harness codex \
  --url https://example.com

View reports and score history:

node cli/index.js report
node cli/index.js score --history

Build or serve the dashboard:

node cli/index.js dashboard --build-only
node cli/index.js dashboard --port 4040

Commands

init

Creates:

• config/project.yaml
• config/jtbd.yaml
• config/interviews.yaml
• config/test-scenarios.yaml
• config/personas.yaml
• scores/history.json

generate-personas

Generates personas from the JTBD map and appends them to config/personas.yaml.

interview

Runs deterministic simulated interviews for selected personas and writes a run report under runs/.

test

Opens a real browser session, follows the configured scenario path, captures screenshots, scores the experience, writes a test report, and appends score history.

Useful flags:

• --url <url>
• --scenario <scenarioId>
• --persona <personaId>
• --runner <runner>
• --cdp-url <url>
• --harness <harness>
• --secondary-harness <harness>
• --orchestration <mode>
• --subagents
• --subagent-execution <mode>
• --max-reviewers <count>
• --headed
• --ci
• --threshold <score>
• --exit-code

score

Shows latest scores, history, comparisons, and CSV export.

report

Lists runs, opens the latest report path, and compares run manifests.

dashboard

Builds a single-file dashboard from local config and run history, then optionally serves it.

Output Layout

auto-jobs/
├── cli/
├── config/
├── dashboard/
├── runs/
├── scores/
├── templates/
└── SPEC.md

Typical generated files:

• runs/YYYY-MM-DD-HHmm-interview/report.md
• runs/YYYY-MM-DD-HHmm-test/report.md
• runs/YYYY-MM-DD-HHmm-test/scores.json
• runs/YYYY-MM-DD-HHmm-test/screenshots/...
• scores/history.json
• dashboard/index.html

Scripted Scenarios

Authenticated and form-heavy journeys can be declared directly in config/test-scenarios.yaml:

scenarios:
  - id: signed-in-dashboard
    name: Reach the live dashboard
    entry_url: /login
    target_job: merchant_collect_cash
    setup_steps:
      - action: fill
        label: Email address
        value: demo@example.com
      - action: fill
        label: Password
        value: demodemo
      - action: click
        role: button
        name: Sign in
    journey_steps:
      - action: assert
        url_includes: /dashboard
      - action: click
        text: Invoices
      - action: assert
        text: All Invoices

Supported step actions:

• goto
• click
• fill
• select
• upload
• press
• wait
• assert
• check
• uncheck
• hover

Harness Configuration

Harnesses are configured in config/project.yaml:

ai:
  harness:
    primary: codex
    secondary: claude
    orchestration: single
    timeout_ms: 20000
  subagents:
    enabled: true
    execution: sequential
    max_reviewers: 2

browser:
  runner: playwright-agent
  headless: true
  max_steps: 4
  cdp_url: http://127.0.0.1:9222

vision:
  enabled: true
  max_snapshots: 1

You can also set these at project creation time:

node cli/index.js init --yes \
  --harness claude \
  --secondary-harness codex \
  --orchestration consensus \
  --runner playwright-agent

Supported harnesses:

• codex
• claude

Supported orchestration modes:

• single
• consensus

Supported browser runners:

• playwright-agent
• chrome-devtools-agent
• deterministic-http

Scenario Steps

Scenarios can define setup_steps for authentication or environment prep, and journey_steps for deterministic rich flows before the exploratory agent takes over.

Supported actions:

• goto
• click
• fill
• select
• upload
• press
• assert
• wait
• check
• uncheck
• hover

Scoring

Test runs are scored across these dimensions:

• job_completion
• time_to_value
• findability
• comprehension
• error_recovery
• satisfaction
• accessibility
• performance
• trust
• outcome_alignment

The current scorer blends:

• browser navigation evidence
• page structure and content heuristics
• screenshot capture
• optional harness-based visual analysis
• persona trait adjustments

When a harness is slow or unavailable, the browser agent falls back to deterministic heuristics so runs still complete.

Philosophy

Auto-Jobs is designed to stay:

• CLI-first
• local-file based
• framework-light
• JTBD-grounded
• easy to inspect and hack on

Repo Notes

• Source of truth product/spec context lives in SPEC.md
• Config is YAML
• Reports are Markdown
• Scores and history are JSON
• Dashboard pages are single-file HTML with no frontend build step

Status

This repo is an active MVP. The core local loop works today:

1. define product context
2. generate personas
3. run interviews
4. run browser-based scored tests with Codex or Claude as the harness
5. inspect reports and dashboard