Skip to content

Add test suite, E2E tests, and CI for live-reload#2

Open
knobo wants to merge 13 commits into
masterfrom
add-tests-ci-e2e
Open

Add test suite, E2E tests, and CI for live-reload#2
knobo wants to merge 13 commits into
masterfrom
add-tests-ci-e2e

Conversation

@knobo

@knobo knobo commented May 30, 2026

Copy link
Copy Markdown
Owner

Summary

Adds a real automated test suite, end-to-end browser/protocol tests, and CI to the live-reload server — plus fixes uncovered along the way. No production .lisp behavior was changed; this is tests + tooling + bug fixes in tests.

The work was developed in parallel across isolated git worktrees, then integrated and verified end-to-end against a real running server.

What's included

FiveAM unit suite (live-reload-test.asd, t/test-live-reload.lisp)

  • New separate ASDF system live-reload-test (:depends-on ("live-reload" "fiveam")) so it never interferes with the main system. (asdf:test-system :live-reload-test) signals failure on a failing run.
  • 19 tests / 36 checks, 100% pass. Covers:
    • send-update-url — exact reload-command JSON format, verbatim url, broadcast to N clients, zero-client no-op
    • add / stop — the inotify mailbox protocol
    • add-file-to-listener*files* pushnew semantics against a real inotify handle
    • mlcar macro
    • the LiveReload v7 hello handshake string
    • the Lack middleware — wrapper is callable, pathname results get registered for watching via add, non-pathname results pass through unchanged
  • Every test rebinds the relevant specials with let so global state is never polluted; no sockets or OS threads are started.

Playwright E2E + WebSocket protocol test (t/e2e/, package.json)

  • A fast, browser-free WebSocket protocol smoke test (ws.spec.js): boots the real Lisp server, performs the v7 handshake, changes a watched file on disk, and asserts a {"command":"reload",...} broadcast arrives. Verified passing against a live server.
  • A full Chromium reload test (reload.spec.js): loads the served page and asserts the browser actually reloads when a watched file changes.
  • t/e2e/server.ros launcher, Playwright config, fixtures, and t/e2e/README.md with prerequisites.

CI (.github/workflows/ci.yml)

  • lisp-tests: installs libfixposix-dev + build-essential (required — inotifyiolib grovel needs lfp.h), sets up Roswell/Quicklisp, loads live-reload, and runs the FiveAM suite with proper non-zero-on-failure exit semantics. Caches Roswell/Quicklisp.
  • node-tests: existence-guarded npm + node --check for the E2E tooling and client script.

Bug fixes found while wiring it up

  • Tests referenced the internal symbol live-reload-ws::*live-reload-hello-string* with a single colon, causing a READ error during COMPILE-FILE so the test system never compiled. Fixed all references.
  • The middleware tests booted Hunchentoot on port 35729 via ensure-running when the wrapper was built, leaking a listening socket after the suite finished. Replaced with three hermetic tests behind a with-stubbed-ensure-running macro — no server, no socket — that also verify the middleware's real watch-registration behavior.
  • .gitignore extended to cover ASDF fasl build artifacts, qlot, and .claude/ agent-artifact data.

Verification (run locally against a real server)

system load            : OK
FiveAM unit suite      : 36 checks, Pass 36 (100%), Fail 0   (exit 0), no socket bound
Playwright WS E2E      : 1 passed                            (exit 0)

Note for reviewers / prerequisites

Running the Lisp tests locally requires libfixposix (sudo apt-get install -y libfixposix-dev build-essential) because inotifyiolib greps lfp.h at build time. CI installs this automatically. For the E2E browser test: npm install && npx playwright install chromium.

knobo added 13 commits May 30, 2026 14:23
Adds .github/workflows/ci.yml with two jobs:
- lisp-tests: installs libfixposix-dev + build-essential (needed for the
  iolib grovel pulled in via inotify), sets up Roswell/SBCL/Quicklisp via
  40ants/setup-lisp, symlinks the checkout into local-projects, smoke-tests
  (ql:quickload :live-reload), and runs the FiveAM suite with a non-zero
  exit on failure (tries live-reload-test system, falls back to loading
  t/test-live-reload.lisp).
- node-tests: setup-node plus existence-guarded npm test and node --check
  on static/js/livereload.js.
Add live-reload-test.asd (system :live-reload-test, depends on live-reload
and fiveam, :pathname "t", test-op runs the suite and errors on failure)
and flesh out t/test-live-reload.lisp into a real FiveAM suite.

Tests cover:
- send-update-url: exact reload-JSON payload, verbatim url, broadcast to
  multiple clients, and zero-client no-op (intercepted via a fake client
  class with a wsd:send method).
- add / stop: correct message enqueued on a rebound file-adder mailbox.
- add-file-to-listener: *files* pushnew semantics (no dups, multi-url)
  exercised against a real inotify handle on a temp file.
- *live-reload-hello-string*: valid JSON handshake with hello command and
  the official LiveReload v7 protocol url.
- middleware: *lack-middleware-live-reload* is a callable app->app wrapper.

All global special vars are rebound with let so process state is not
polluted. The mlcar macro is not tested because it does not exist in the
source tree.
Add an end-to-end test suite under t/e2e/ driving the Common Lisp
LiveReload server (started via Roswell) from Node:

- server.ros: Roswell launcher that quickloads :live-reload, starts the
  WebSocket/HTTP server on port 35729, registers a temp watched file via
  (live-reload:add ...), and serves a test HTML page on port 35730.
- lib/server.js: spawns/stops the Lisp server, waits for readiness +
  port, and exposes touch()/stop(). Skips gracefully (tagged error) when
  ros is missing or the server can't start; fails loudly otherwise.
- ws.spec.js: browser-free protocol smoke test (client hello -> server
  hello, file change -> reload broadcast) using the ws package.
- reload.spec.js: Chromium headless test asserting a real page reload on
  watched-file change.
- playwright.config.js, root package.json, fixtures/page.html, README.md.
mlcar (map-lambda-car) exists in live-reload.lisp, so test it (basic
mapping + empty list). Removed the comment that wrongly claimed it absent.

Verified the suite (17 tests / 32 checks) runs 100% green under SBCL+FiveAM
with stub packages standing in for the unbuildable inotify/iolib dep.
The symbol is not exported from live-reload-ws; single-colon caused a
READ error during COMPILE-FILE so the test system never compiled.
All 4 remaining references used single-colon for the internal symbol.
FiveAM suite now compiles and runs: 49 checks, 100% pass.
The two middleware tests booted Hunchentoot on port 35729 via
ENSURE-RUNNING when the wrapper was built, leaking a listening socket
after the suite finished.

Replace with three hermetic tests behind a WITH-STUBBED-ENSURE-RUNNING
macro (no server, no socket): the wrapper is callable, a pathname body is
registered for watching via ADD with the :path-info url, and a
non-pathname body is passed through unchanged registering nothing.

Suite: 19 tests / 36 checks, 100% pass, no socket bound.
Ignore ASDF fasl output, qlot, and Claude Code agent worktree/session
data so they never land in commits.
- test-live-reload.lisp: use single-colon for exported symbols
  (send-update-url, add, stop, *live-reload-clients*) and correct the
  header comment that misstated which symbols package.lisp exports.
- server.ros: fix 'ningle:app -> 'ningle:<app> (the real class name);
  the previous name would have errored when serving the test page, a
  bug the browser spec now exercises and passes.
- server.ros + e2e/README.md: fix 't/t/test-simple.ros' path typo.
- ci.yml: drive tests solely through (asdf:test-system :live-reload-test),
  whose test-op already signals on FiveAM failure; catch it and exit
  non-zero. Removes the redundant double-run and its fragile
  run!-return-value exit mapping.

Verified: unit 36/36 pass; browser + ws E2E specs pass against a live server.
The browser client static/js/livereload.js is third-party code (the
LiveReload client). It was checked in with no version record, no
integrity check, and an unused byte-identical duplicate at t/livereload.js.

Establish provenance and reproducibility:
- Identify it as published livereload-js@2.3.0 dist/livereload.js
  (byte-identical to npm; the internal Version string still reads 2.2.2,
  a stale upstream constant).
- scripts/livereload.lock records the pinned version + SHA-256.
- scripts/verify-livereload.sh: offline hash check (gates CI against drift).
- scripts/fetch-livereload.sh: re-download a pinned version from npm,
  verify, and vendor it (--check diffs without writing).
- scripts/lock-livereload.sh: regenerate the lock after an upgrade.
- THIRDPARTY.md: attribution, MIT license, upgrade instructions.
- CI: new node-tests step runs verify-livereload.sh.
- Remove the unused duplicate t/livereload.js (only static/js is served).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant