Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 12 additions & 2 deletions .github/workflows/autopilot-create-issue.yml
Original file line number Diff line number Diff line change
@@ -1,11 +1,15 @@
name: Autopilot Issue Intake
name: Autopilot Issue Intake

on:
workflow_run:
# Update this list in each repo to match the workflows you want monitored.
workflows: ["Demo CI"]
types: [completed]

concurrency:
group: autopilot-intake-${{ github.event.workflow_run.head_sha }}
cancel-in-progress: false

permissions:
actions: read
issues: write
Expand All @@ -15,9 +19,10 @@ jobs:
create-issue:
if: ${{ github.event.workflow_run.conclusion == 'failure' }}
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Create or update issue
uses: actions/github-script@v7
uses: actions/github-script@60a0d83039c74a4aee543508d2ffcb1c3799cdea # v7.0.1
with:
script: |
const run = context.payload.workflow_run;
Expand Down Expand Up @@ -57,6 +62,10 @@ jobs:
`Branch: ${run.head_branch}`,
`SHA: ${run.head_sha}`,
`Run: ${run.html_url}`,
`Attempt: ${run.run_attempt}`,
`Event: ${run.event}`,
`Actor: ${run.actor?.login || 'unknown'}`,
`Conclusion: ${run.conclusion}`,
'',
'Failed steps:',
stepSummary,
Expand Down Expand Up @@ -92,6 +101,7 @@ jobs:
owner,
repo,
issue_number: issue.number,
state: 'open',
body,
});
await github.rest.issues.addLabels({
Expand Down
41 changes: 26 additions & 15 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -1,29 +1,40 @@
name: CI
name: CI

on:
push:
branches: [main]
pull_request:
branches: [main]

permissions:
contents: read

jobs:
validate:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2

- name: Install validation dependencies
run: python -m pip install --requirement requirements-dev.txt

- name: Validate workflow YAML
shell: python
run: |
python -c "
import yaml, glob
files = glob.glob('.github/workflows/*.yml')
errors = []
for f in files:
try:
with open(f) as fh:
yaml.safe_load(fh)
print(' OK:', f)
except yaml.YAMLError as e:
print(' WARN:', f, '-', str(e).split('\n')[0])
print('Validated', len(files), 'workflow files')
"
import pathlib
import yaml

files = sorted(pathlib.Path('.github/workflows').glob('*.yml'))
if not files:
raise SystemExit('No workflow files found')

for path in files:
with path.open(encoding='utf-8') as stream:
yaml.safe_load(stream)
print(f'OK: {path}')

print(f'Validated {len(files)} workflow files')

- name: Run workflow contract tests
run: python -m unittest discover -s tests -v
18 changes: 17 additions & 1 deletion .github/workflows/demo-ci.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,30 @@
name: Demo CI
name: Demo CI

on:
workflow_dispatch:
inputs:
simulate_failure:
description: "Fail the demo job to exercise Autopilot intake"
required: true
type: boolean
default: false

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update the documented demo command to request failure

With this input defaulting to false, the gh workflow run demo-ci.yml -R Coding-Autopilot-System/autopilot-demo command under README.md's "Running the demo" section now produces a successful run, so the intake workflow does not create an issue and the documented end-to-end demonstration stalls. Update that command to pass simulate_failure=true or otherwise ensure the primary demo instructions explicitly select the failure input.

Useful? React with 👍 / 👎.

push:
branches: [main]

permissions:
contents: read

jobs:
demo:
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Demo sanity
run: |
echo "Simulated demo step (non-blocking)."

- name: Simulate repairable failure
if: ${{ inputs.simulate_failure }}
run: |
echo "Intentional demo failure requested."
exit 1
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
__pycache__/
*.py[cod]
7 changes: 7 additions & 0 deletions .planning/phases/01-enterprise-hardening/01-SUMMARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
# Enterprise Hardening Summary

The GSD audit classified eight findings as auto-fixable and two GitHub administration items as manual-only. All eight repository findings were fixed, tested, and recorded in `memory/examples/`.

The demo now provides a deliberate failure switch, strict CI validation, immutable Action references, least-privilege workflow boundaries, executable workflow contracts, serialized intake, richer audit evidence, and a repeatable operator runbook.

F-08 failed its first contract test because the intake update path did not reopen closed issues. That attempt was reverted, corrected, and verified on retry.
23 changes: 23 additions & 0 deletions .planning/phases/01-enterprise-hardening/01-UAT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Enterprise Hardening UAT

**Date:** 2026-06-10
**Source:** `gsd-audit-fix --severity all --max 8`
**Scope:** demo reproducibility, security, tests, CI, user journey, failure modes, and docs

## Classification

| ID | Severity | Classification | Finding | Files |
|---|---|---|---|---|
| F-01 | high | auto-fixable | The documented demo cannot produce a failure because Demo CI always succeeds. | `.github/workflows/demo-ci.yml`, `README.md` |
| F-02 | high | auto-fixable | CI prints YAML parse warnings but still exits successfully. | `.github/workflows/ci.yml` |
| F-03 | high | auto-fixable | Third-party Actions use mutable major-version tags instead of immutable commit SHAs. | `.github/workflows/ci.yml`, `.github/workflows/autopilot-create-issue.yml` |
| F-04 | medium | auto-fixable | CI and Demo CI do not declare least-privilege token permissions or job timeouts. | `.github/workflows/ci.yml`, `.github/workflows/demo-ci.yml` |
| F-05 | medium | auto-fixable | No automated contract tests protect workflow behavior and security invariants. | `tests/test_workflows.py`, `.github/workflows/ci.yml` |
| F-06 | medium | auto-fixable | Intake processing has no concurrency guard or timeout, allowing duplicate/racing issue updates. | `.github/workflows/autopilot-create-issue.yml` |
| F-07 | low | auto-fixable | Intake issues omit run attempt, event, actor, and failure conclusion from the audit evidence. | `.github/workflows/autopilot-create-issue.yml` |
| F-08 | low | auto-fixable | The demo runbook lacks prerequisites, reset steps, and failure-mode troubleshooting; repeat demonstrations can leave a matching intake issue closed. | `.github/workflows/autopilot-create-issue.yml`, `README.md` |

## Manual-only Findings

- M-01: Protect `main` and require the `CI` status check. The GitHub API reports that `main` is not protected.
- M-02: Configure the organization Actions policy to allow only approved actions and require immutable SHA pinning where supported. The current GitHub token cannot read or change this setting.
33 changes: 33 additions & 0 deletions .planning/phases/01-enterprise-hardening/01-VERIFICATION.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# Enterprise Hardening Verification

**Date:** 2026-06-10
**Branch:** `hardening/enterprise-audit-20260610`
**Result:** passed with manual GitHub administration gaps

## Finding Status

| ID | Result | Evidence |
|---|---|---|
| F-01 | passed | `simulate_failure` is explicit and defaults to false |
| F-02 | passed | invalid YAML now fails CI; dependency is pinned |
| F-03 | passed | third-party Actions use verified 40-character commit SHAs |
| F-04 | passed | routine jobs declare read-only permissions and five-minute timeouts |
| F-05 | passed | seven workflow contract tests run in CI; bytecode is ignored |
| F-06 | passed | intake updates are serialized by head SHA and time-bounded |
| F-07 | passed | issues record attempt, event, actor, and conclusion |
| F-08 | passed after one reverted attempt | matching closed issues reopen; runbook covers prerequisites, reset, and troubleshooting |

## Verification Commands

- `python -m unittest discover -s tests -v`: 7 passed
- strict PyYAML parse of `.github/workflows/*.yml`: 3 validated
- PowerShell parser check of `scripts/record-fix.ps1`: passed
- `git diff --check origin/main...HEAD`: passed
- clean worktree after tests: passed

## Manual Gaps

- Protect `main` and require the `CI` status check.
- Configure the organization Actions allow-list and immutable pinning policy.
- Run the live failure-to-issue-to-repair journey after this branch is merged because `workflow_dispatch` uses the default-branch workflow definition.
- Install `actionlint` in the workstation bootstrap and add it to CI; it was not available for this verification.
17 changes: 15 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# autopilot-demo
# autopilot-demo

[![CI](https://github.com/Coding-Autopilot-System/autopilot-demo/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/Coding-Autopilot-System/autopilot-demo/actions/workflows/ci.yml)
[![Demo CI](https://github.com/Coding-Autopilot-System/autopilot-demo/actions/workflows/demo-ci.yml/badge.svg?branch=main)](https://github.com/Coding-Autopilot-System/autopilot-demo/actions/workflows/demo-ci.yml)
Expand Down Expand Up @@ -47,7 +47,7 @@ gh pr list -R Coding-Autopilot-System/autopilot-demo

## Demo runbook

1. Trigger [`.github/workflows/demo-ci.yml`](.github/workflows/demo-ci.yml) to produce a known failure signal.
1. Trigger [`.github/workflows/demo-ci.yml`](.github/workflows/demo-ci.yml) with `simulate_failure=true` to produce a known failure signal. Pushes and default dispatches remain green.
2. Confirm [`.github/workflows/autopilot-create-issue.yml`](.github/workflows/autopilot-create-issue.yml) creates an `autofix + queued` issue.
3. Watch `autopilot-core` pick up the issue and open a PR back into this repo.
4. Use this repo's issue, branch, and PR history as the audit trail for the demo.
Expand All @@ -72,3 +72,16 @@ gh pr list -R Coding-Autopilot-System/autopilot-demo
- [autopilot-core](https://github.com/Coding-Autopilot-System/autopilot-core) - operator control plane
- [ci-autopilot](https://github.com/Coding-Autopilot-System/ci-autopilot) - worker/runtime reference
- [Coding-Autopilot-System org](https://github.com/Coding-Autopilot-System)
## Prerequisites and expected result

- Authenticate GitHub CLI with `gh auth status` and confirm Actions are enabled for this repository.
- Run the `autopilot-core` operator with access to this repository before triggering the failure.
- Expect Demo CI to fail, Autopilot Issue Intake to create or reopen one `autofix + queued` issue, and the operator to propose a pull request.

## Reset and troubleshooting

1. Close the completed intake issue and merge or close its repair pull request before the next demonstration.
2. Re-run with `simulate_failure=true`; intake reopens the matching issue when the same commit is demonstrated again.
3. If no issue appears, inspect `gh run list -R Coding-Autopilot-System/autopilot-demo --workflow autopilot-create-issue.yml` and confirm the failed run was named `Demo CI`.
4. If the issue remains queued, verify the `autopilot-core` operator is running and can read issues and create branches and pull requests in this repository.
5. If CI fails before the demo step, run `python -m unittest discover -s tests -v` locally and repair the workflow contract violation first.
8 changes: 8 additions & 0 deletions memory/examples/20260610_f01_reproducible-failure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# F-01 reproducible demo failure

Issue Description: The documented demo could not produce a failure because Demo CI always succeeded.
State: Pushes and manual dispatches both ran only a non-blocking step.
Action: Added an explicit `simulate_failure` dispatch input and documented the required command.
Result: Operators can intentionally exercise intake while normal pushes and default dispatches stay green.
Diff Patch: Updated `.github/workflows/demo-ci.yml` and `README.md`.
Rationale: A demo repair pipeline must expose a safe, deliberate, and reproducible failure path.
8 changes: 8 additions & 0 deletions memory/examples/20260610_f02_strict-yaml-validation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# F-02 strict YAML validation

Issue Description: CI printed YAML parsing warnings but always returned success.
State: Invalid workflow YAML could pass the portfolio CI check.
Action: Made parsing errors fatal and pinned the YAML parser dependency.
Result: CI now fails closed when workflow YAML is missing or invalid.
Diff Patch: Updated `.github/workflows/ci.yml` and added `requirements-dev.txt`.
Rationale: Validation jobs must fail on the condition they claim to validate.
8 changes: 8 additions & 0 deletions memory/examples/20260610_f03_immutable-actions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# F-03 immutable Action references

Issue Description: Workflows trusted mutable major-version Action tags.
State: Upstream tag movement could change executed code without a repository diff.
Action: Pinned checkout v4.2.2 and github-script v7.0.1 to verified commit SHAs.
Result: Third-party workflow code is immutable and remains human-readable through version comments.
Diff Patch: Updated CI and intake workflow Action references.
Rationale: Immutable references reduce CI supply-chain risk.
8 changes: 8 additions & 0 deletions memory/examples/20260610_f04_workflow-boundaries.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# F-04 least-privilege workflow boundaries

Issue Description: CI and Demo CI relied on repository-default token permissions and had no execution timeout.
State: Jobs could inherit broader permissions and run without a repository-defined time bound.
Action: Declared read-only contents permission and five-minute job timeouts.
Result: Routine workflows now have explicit least privilege and bounded execution.
Diff Patch: Updated `.github/workflows/ci.yml` and `.github/workflows/demo-ci.yml`.
Rationale: Security boundaries should be visible and enforced in source control.
8 changes: 8 additions & 0 deletions memory/examples/20260610_f05_workflow-contract-tests.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# F-05 workflow contract tests

Issue Description: No automated tests protected workflow behavior and security invariants.
State: Regressions in YAML validity, Action pinning, permissions, timeouts, or demo failure behavior depended on review alone.
Action: Added standard-library workflow contract tests and executed them from CI.
Result: Core demo and security invariants now fail visibly in pull requests.
Diff Patch: Added `tests/test_workflows.py` and updated `.github/workflows/ci.yml`.
Rationale: Workflow code needs executable contracts like application code.
8 changes: 8 additions & 0 deletions memory/examples/20260610_f06_serialized-intake.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# F-06 serialized intake processing

Issue Description: Intake processing had no concurrency guard or execution timeout.
State: Multiple failures for the same commit could race while updating the same signature issue.
Action: Serialized intake by workflow head SHA, preserved queued runs, and bounded the job to five minutes.
Result: Intake updates are deterministic and cannot run indefinitely.
Diff Patch: Updated the intake workflow and its contract tests.
Rationale: Idempotent issue handling still needs race protection at the workflow boundary.
8 changes: 8 additions & 0 deletions memory/examples/20260610_f07_intake-audit-context.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# F-07 richer intake audit evidence

Issue Description: Intake issues omitted operational context needed to reconstruct a failure.
State: Issues recorded workflow, branch, SHA, run URL, and failed steps only.
Action: Added run attempt, triggering event, actor, and conclusion to every intake issue body.
Result: Each issue now carries a stronger self-contained audit trail.
Diff Patch: Updated the intake workflow and its contract tests.
Rationale: Operational evidence should support diagnosis without requiring immediate API access.
8 changes: 8 additions & 0 deletions memory/examples/20260610_f08_repeatable-runbook.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
# F-08 repeatable operator runbook

Issue Description: The runbook lacked prerequisites, reset steps, and troubleshooting, while repeat demonstrations could leave the matching issue closed.
State: Operators had no defined recovery path and intake updated closed matching issues without reopening them.
Action: Documented prerequisites, expected results, reset and troubleshooting; reopen matching intake issues on repeat failures.
Result: The user journey is repeatable and common failure modes have concrete diagnostics.
Diff Patch: Updated the intake workflow, README, UAT record, and contract tests.
Rationale: A portfolio demo must be operable repeatedly by someone other than its author.
1 change: 1 addition & 0 deletions requirements-dev.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
PyYAML==6.0.3
59 changes: 59 additions & 0 deletions tests/test_workflows.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
import pathlib
import re
import unittest

import yaml


ROOT = pathlib.Path(__file__).resolve().parents[1]
WORKFLOWS = ROOT / ".github" / "workflows"


class WorkflowContractTests(unittest.TestCase):
def workflow_text(self, name: str) -> str:
return (WORKFLOWS / name).read_text(encoding="utf-8-sig")

def test_all_workflows_are_valid_yaml(self) -> None:
paths = sorted(WORKFLOWS.glob("*.yml"))
self.assertTrue(paths, "expected at least one workflow")
for path in paths:
with self.subTest(path=path.name):
self.assertIsInstance(yaml.safe_load(path.read_text(encoding="utf-8-sig")), dict)

def test_third_party_actions_are_pinned_to_commits(self) -> None:
action_ref = re.compile(r"uses:\s+[^./\s][^@\s]*@([^\s#]+)")
for path in WORKFLOWS.glob("*.yml"):
for ref in action_ref.findall(path.read_text(encoding="utf-8-sig")):
with self.subTest(path=path.name, ref=ref):
self.assertRegex(ref, r"^[0-9a-f]{40}$")

def test_routine_workflows_are_read_only_and_bounded(self) -> None:
for name in ("ci.yml", "demo-ci.yml"):
text = self.workflow_text(name)
with self.subTest(name=name):
self.assertIn("permissions:\n contents: read", text.replace("\r\n", "\n"))
self.assertIn("timeout-minutes: 5", text)

def test_intake_is_serialized_and_bounded(self) -> None:
text = self.workflow_text("autopilot-create-issue.yml")
self.assertIn("group: autopilot-intake-${{ github.event.workflow_run.head_sha }}", text)
self.assertIn("cancel-in-progress: false", text)
self.assertIn("timeout-minutes: 5", text)
def test_intake_records_audit_context(self) -> None:
text = self.workflow_text("autopilot-create-issue.yml")
for field in ("Attempt:", "Event:", "Actor:", "Conclusion:"):
with self.subTest(field=field):
self.assertIn(field, text)
def test_existing_intake_issue_is_reopened(self) -> None:
text = self.workflow_text("autopilot-create-issue.yml")
self.assertIn("state: 'open'", text)
def test_demo_failure_is_explicitly_opt_in(self) -> None:
text = self.workflow_text("demo-ci.yml")
self.assertIn("simulate_failure:", text)
self.assertIn("default: false", text)
self.assertIn("if: ${{ inputs.simulate_failure }}", text)
self.assertIn("exit 1", text)


if __name__ == "__main__":
unittest.main()
Loading