Skip to content

feat(testing): add cross-platform E2E test suite#78

Merged
wpfleger96 merged 10 commits into
mainfrom
wpfleger/e2e-testing
Jun 3, 2026
Merged

feat(testing): add cross-platform E2E test suite#78
wpfleger96 merged 10 commits into
mainfrom
wpfleger/e2e-testing

Conversation

@wpfleger96
Copy link
Copy Markdown
Owner

@wpfleger96 wpfleger96 commented Jun 3, 2026 •
edited
Loading

Stack: #74 → this PR

Summary

  • Add subprocess-based E2E tests that invoke the real ai-agent-rules CLI binary via python -m ai_rules.cli against an isolated temp HOME, validating that commands actually work on each OS
  • New tests/e2e/ directory with 13 test cases covering --help, status, validate, list-agents, diff, install --dry-run, and completions status
  • New .github/workflows/e2e.yml runs the E2E suite on ubuntu-latest, macos-latest, and windows-latest (separate from the github-config-managed ci.yml)
  • pyproject.toml updated with e2e marker and --ignore=tests/e2e in addopts so just test excludes E2E by default
  • Justfile gains test-e2e recipe using -n 0 (not -p no:xdist, which crashes when -n auto is in addopts)
  • Windows-specific test (TestWindowsSpecific) uses @pytest.mark.skipif(sys.platform != "win32") and exercises PowerShell completions via the Windows support in feat: add native Windows support #74

Key fixture design:

  • e2e_home returns (home_path, env_overrides) dict passed to subprocess.run(env=...) — monkeypatch only affects the current process, not child subprocesses
  • subprocess.run(encoding="utf-8") instead of text=True — on Windows, text=True uses cp1252 on the parent side, crashing on Rich's Unicode box-drawing characters
  • NO_COLOR=1 + PYTHONIOENCODING=utf-8 + PYTHONUTF8=1 in subprocess env for clean cross-platform assertions
  • run_cli extends PYTHONPATH with src/ so importlib.resources resolves the dev package config

Related: shell-configs#65

@wpfleger96 wpfleger96 force-pushed the wpfleger/e2e-testing branch from 907c2dd to 7b99c0e Compare June 3, 2026 20:26
@wpfleger96 wpfleger96 changed the base branch from main to worktree-wpfleger-windows-support June 3, 2026 20:26
@wpfleger96 wpfleger96 force-pushed the wpfleger/e2e-testing branch from 7b99c0e to 4c310f2 Compare June 3, 2026 20:53
wpfleger96 and others added 5 commits June 3, 2026 17:14
@wpfleger96
feat: add native Windows support
0718f50 
Claude Code, Goose, Gemini CLI, and Codex CLI all run natively on Windows.
Amp is WSL-only so it remains Linux-only. This change adds the plumbing to
make all four agents work on native Windows without requiring WSL.

Key design decisions:
- Centralize all platform branching in a new `platform.py` module (single
  monkeypatch target for tests; no scattered sys.platform checks).
- Goose and Statusline use different config dirs on Windows (%APPDATA%/Block
  and %APPDATA%/claude-statusline respectively); helpers return tilde paths
  on Unix for backward compatibility.
- Gemini CLI actively destroys symlinks when rewriting settings.json (GitHub
  issue #10960, closed "not planned"). Windows Gemini gets file copies instead
  of symlinks via a new copy_mode_targets mechanism; preserved fields are read
  from the live target before each cache rebuild to survive Gemini rewrites.
- Windows symlinks require Developer Mode; PermissionError surfaces a
  step-by-step enable guide instead of a generic message.
- PowerShell completions use Register-ArgumentCompleter with bash_source mode,
  setting COMP_WORDS/COMP_CWORD so Click returns contextual completions.
- uv path resolution respects UV_TOOL_DIR override on all platforms.
- CI matrix extended to windows-latest.
@wpfleger96
refactor: upgrade platform detection to 4-value enum with shellingham
af52639 
Replace is_windows() boolean with Platform enum (MACOS, LINUX, WINDOWS,
WSL) and detect_platform() using platform.uname().release for WSL
detection. Adopt shellingham for shell detection on all platforms --
PSModulePath was inherited by every Windows process, falsely identifying
all shells as PowerShell. Query PowerShell for its profile path via
subprocess instead of hardcoding ~/Documents (OneDrive silently
redirects it). Drop click-pwsh (broken under uv).
@wpfleger96
fix: address crossfire review findings for Windows support
daa9c86 
Fix Python 3 SyntaxError from bare-comma except clauses. Rewrite
PowerShell completion to use Click's bash_complete mode with correct
COMP_CWORD cursor-position logic. Add backup and confirmation to
create_file_copy when target has user modifications. Wire copy-mode
awareness into global pending-change and first-run checks. Add APPDATA
fallback for stripped Windows environments, filter Amp skills on
Windows, and add shellingham upper-bound pin.
@wpfleger96
fix(tests): make test suite cross-platform for Windows CI
48402c7 
On Windows, Path.home() reads USERPROFILE (not HOME), and str(Path)
uses backslashes. Tests that only set HOME caused 75 failures on Windows
CI because Path.home() resolved to the real C:\Users\runneradmin.

- Add autouse _patch_home_from_env fixture in conftest.py that makes
  Path.home() follow the HOME env var at call time via a lazy closure,
  so all tests that set HOME get correct isolation without any per-test
  changes
- Update mock_home fixture to also set APPDATA for Windows parity
- Fix test_agents.py: use Path.as_posix() when comparing symlink targets
  so ~\.claude\CLAUDE.md comparisons work cross-platform
- Fix test_bootstrap_installer.py: use UV_TOOL_DIR override instead of
  XDG_DATA_HOME in tool-source tests (UV_TOOL_DIR is the canonical env
  var and takes precedence on all platforms); use as_posix() for path
  assertions; clear UV_TOOL_DIR in tests that check default path structure
- Fix test_bootstrap_updater.py: same UV_TOOL_DIR approach for pyvenv.cfg
  tests
- Fix test_session_search_core.py: use Path.as_posix() when comparing
  session_to_json path output; fix matches_term path test to not rely
  on forward-slash path separators in str(path)
- Fix test_session_search_readers.py: also set USERPROFILE in _patch_home
  helper (belt-and-suspenders alongside the autouse fixture)
@wpfleger96
fix(tests): resolve remaining 14 Windows CI failures
34f44e5 
Four root causes: (1) Click CliRunner uses cp1252 on Windows, causing
UnicodeDecodeError for skill markdown with emoji — fixed via PYTHONUTF8=1
in autouse fixture. (2) Path assertions using forward slashes fail when
Path() stringifies with backslashes — normalized with .replace/as_posix().
(3) repo_score() converts POSIX session paths to backslash form, breaking
all string comparisons — fixed with .as_posix() in production code.
@wpfleger96 wpfleger96 force-pushed the worktree-wpfleger-windows-support branch from a8e6f56 to 34f44e5 Compare June 3, 2026 21:14
@wpfleger96 wpfleger96 force-pushed the wpfleger/e2e-testing branch from 4c310f2 to c04ceb6 Compare June 3, 2026 21:15
@wpfleger96 wpfleger96 mentioned this pull request Jun 3, 2026
Merged
@wpfleger96
fix: use explicit utf-8 encoding for skill file reads
e3af0b8 
Path.read_text() without encoding uses the system default (cp1252 on
Windows). SKILL.md files contain UTF-8 emoji characters that can't be
decoded by cp1252, causing UnicodeDecodeError in all skill CLI commands
on Windows.
@wpfleger96 wpfleger96 force-pushed the wpfleger/e2e-testing branch from c04ceb6 to d921ad9 Compare June 3, 2026 21:21
wpfleger96 added 3 commits June 3, 2026 18:42
@wpfleger96
ci: revert checks job to ubuntu-only
b3713ba 
Cross-platform testing belongs in the E2E workflow (e2e.yml) which runs
on all 3 OSes. The matrix in ci.yml caused the job name to change from
bare "checks" to "checks (ubuntu-latest)" / "checks (windows-latest)",
which no longer matched the ruleset's required status check context.
@wpfleger96
feat(testing): add cross-platform E2E test suite
6f9fa9b 
Subprocess-based E2E tests that invoke the real CLI binary via
`python -m ai_rules.cli` against an isolated temp HOME. Catches the
class of platform-specific bugs (broken imports, wrong paths, encoding
issues) that pass in-process CliRunner tests but fail on real OSes.

New `.github/workflows/e2e.yml` runs the suite on ubuntu, macos, and
windows — separate from the github-config-managed `ci.yml`.
@wpfleger96
fix(e2e): use explicit utf-8 encoding for subprocess pipes
677502d 
text=True uses the parent process's locale encoding (cp1252 on Windows),
not the child's PYTHONIOENCODING. Rich box-drawing characters crash the
cp1252 decoder, leaving stdout as None. Explicit encoding="utf-8"
overrides locale-based pipe decoding.
@wpfleger96 wpfleger96 force-pushed the wpfleger/e2e-testing branch from d921ad9 to 677502d Compare June 3, 2026 22:43
Base automatically changed from worktree-wpfleger-windows-support to main June 3, 2026 22:48
@wpfleger96 wpfleger96 merged commit 11835b2 into main Jun 3, 2026
9 checks passed
@wpfleger96 wpfleger96 deleted the wpfleger/e2e-testing branch June 3, 2026 22:51
@wpfleger-release-bot wpfleger-release-bot Bot mentioned this pull request Jun 3, 2026
Merged
wpfleger96 added a commit that referenced this pull request Jun 3, 2026
@wpfleger96
Merge remote-tracking branch 'origin/main' into worktree-wpfleger+fix…
2f4dad6 
…-managed-component-cleanup

* origin/main:
  feat(testing): add cross-platform E2E test suite (#78)
  feat: add native Windows support (#74)
  chore(main): release 0.59.0 (#75)
  docs(agents): add infrastructure awareness, quality gate, git-pull rule, PR cross-referencing; compress 32% (#77)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@wpfleger96