Add test suite, E2E tests, and CI for live-reload#2
Open
knobo wants to merge 13 commits into
Open
Conversation
Adds .github/workflows/ci.yml with two jobs: - lisp-tests: installs libfixposix-dev + build-essential (needed for the iolib grovel pulled in via inotify), sets up Roswell/SBCL/Quicklisp via 40ants/setup-lisp, symlinks the checkout into local-projects, smoke-tests (ql:quickload :live-reload), and runs the FiveAM suite with a non-zero exit on failure (tries live-reload-test system, falls back to loading t/test-live-reload.lisp). - node-tests: setup-node plus existence-guarded npm test and node --check on static/js/livereload.js.
Add live-reload-test.asd (system :live-reload-test, depends on live-reload and fiveam, :pathname "t", test-op runs the suite and errors on failure) and flesh out t/test-live-reload.lisp into a real FiveAM suite. Tests cover: - send-update-url: exact reload-JSON payload, verbatim url, broadcast to multiple clients, and zero-client no-op (intercepted via a fake client class with a wsd:send method). - add / stop: correct message enqueued on a rebound file-adder mailbox. - add-file-to-listener: *files* pushnew semantics (no dups, multi-url) exercised against a real inotify handle on a temp file. - *live-reload-hello-string*: valid JSON handshake with hello command and the official LiveReload v7 protocol url. - middleware: *lack-middleware-live-reload* is a callable app->app wrapper. All global special vars are rebound with let so process state is not polluted. The mlcar macro is not tested because it does not exist in the source tree.
Add an end-to-end test suite under t/e2e/ driving the Common Lisp LiveReload server (started via Roswell) from Node: - server.ros: Roswell launcher that quickloads :live-reload, starts the WebSocket/HTTP server on port 35729, registers a temp watched file via (live-reload:add ...), and serves a test HTML page on port 35730. - lib/server.js: spawns/stops the Lisp server, waits for readiness + port, and exposes touch()/stop(). Skips gracefully (tagged error) when ros is missing or the server can't start; fails loudly otherwise. - ws.spec.js: browser-free protocol smoke test (client hello -> server hello, file change -> reload broadcast) using the ws package. - reload.spec.js: Chromium headless test asserting a real page reload on watched-file change. - playwright.config.js, root package.json, fixtures/page.html, README.md.
mlcar (map-lambda-car) exists in live-reload.lisp, so test it (basic mapping + empty list). Removed the comment that wrongly claimed it absent. Verified the suite (17 tests / 32 checks) runs 100% green under SBCL+FiveAM with stub packages standing in for the unbuildable inotify/iolib dep.
The symbol is not exported from live-reload-ws; single-colon caused a READ error during COMPILE-FILE so the test system never compiled.
All 4 remaining references used single-colon for the internal symbol. FiveAM suite now compiles and runs: 49 checks, 100% pass.
The two middleware tests booted Hunchentoot on port 35729 via ENSURE-RUNNING when the wrapper was built, leaking a listening socket after the suite finished. Replace with three hermetic tests behind a WITH-STUBBED-ENSURE-RUNNING macro (no server, no socket): the wrapper is callable, a pathname body is registered for watching via ADD with the :path-info url, and a non-pathname body is passed through unchanged registering nothing. Suite: 19 tests / 36 checks, 100% pass, no socket bound.
Ignore ASDF fasl output, qlot, and Claude Code agent worktree/session data so they never land in commits.
- test-live-reload.lisp: use single-colon for exported symbols (send-update-url, add, stop, *live-reload-clients*) and correct the header comment that misstated which symbols package.lisp exports. - server.ros: fix 'ningle:app -> 'ningle:<app> (the real class name); the previous name would have errored when serving the test page, a bug the browser spec now exercises and passes. - server.ros + e2e/README.md: fix 't/t/test-simple.ros' path typo. - ci.yml: drive tests solely through (asdf:test-system :live-reload-test), whose test-op already signals on FiveAM failure; catch it and exit non-zero. Removes the redundant double-run and its fragile run!-return-value exit mapping. Verified: unit 36/36 pass; browser + ws E2E specs pass against a live server.
The browser client static/js/livereload.js is third-party code (the LiveReload client). It was checked in with no version record, no integrity check, and an unused byte-identical duplicate at t/livereload.js. Establish provenance and reproducibility: - Identify it as published livereload-js@2.3.0 dist/livereload.js (byte-identical to npm; the internal Version string still reads 2.2.2, a stale upstream constant). - scripts/livereload.lock records the pinned version + SHA-256. - scripts/verify-livereload.sh: offline hash check (gates CI against drift). - scripts/fetch-livereload.sh: re-download a pinned version from npm, verify, and vendor it (--check diffs without writing). - scripts/lock-livereload.sh: regenerate the lock after an upgrade. - THIRDPARTY.md: attribution, MIT license, upgrade instructions. - CI: new node-tests step runs verify-livereload.sh. - Remove the unused duplicate t/livereload.js (only static/js is served).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a real automated test suite, end-to-end browser/protocol tests, and CI to the live-reload server — plus fixes uncovered along the way. No production
.lispbehavior was changed; this is tests + tooling + bug fixes in tests.The work was developed in parallel across isolated git worktrees, then integrated and verified end-to-end against a real running server.
What's included
FiveAM unit suite (
live-reload-test.asd,t/test-live-reload.lisp)live-reload-test(:depends-on ("live-reload" "fiveam")) so it never interferes with the main system.(asdf:test-system :live-reload-test)signals failure on a failing run.send-update-url— exact reload-command JSON format, verbatim url, broadcast to N clients, zero-client no-opadd/stop— the inotify mailbox protocoladd-file-to-listener—*files*pushnew semantics against a real inotify handlemlcarmacroadd, non-pathname results pass through unchangedletso global state is never polluted; no sockets or OS threads are started.Playwright E2E + WebSocket protocol test (
t/e2e/,package.json)ws.spec.js): boots the real Lisp server, performs the v7 handshake, changes a watched file on disk, and asserts a{"command":"reload",...}broadcast arrives. Verified passing against a live server.reload.spec.js): loads the served page and asserts the browser actually reloads when a watched file changes.t/e2e/server.roslauncher, Playwright config, fixtures, andt/e2e/README.mdwith prerequisites.CI (
.github/workflows/ci.yml)lisp-tests: installslibfixposix-dev+build-essential(required —inotify→iolibgrovel needslfp.h), sets up Roswell/Quicklisp, loadslive-reload, and runs the FiveAM suite with proper non-zero-on-failure exit semantics. Caches Roswell/Quicklisp.node-tests: existence-guardednpm+node --checkfor the E2E tooling and client script.Bug fixes found while wiring it up
live-reload-ws::*live-reload-hello-string*with a single colon, causing aREADerror duringCOMPILE-FILEso the test system never compiled. Fixed all references.ensure-runningwhen the wrapper was built, leaking a listening socket after the suite finished. Replaced with three hermetic tests behind awith-stubbed-ensure-runningmacro — no server, no socket — that also verify the middleware's real watch-registration behavior..gitignoreextended to cover ASDF fasl build artifacts, qlot, and.claude/agent-artifact data.Verification (run locally against a real server)
Note for reviewers / prerequisites
Running the Lisp tests locally requires libfixposix (
sudo apt-get install -y libfixposix-dev build-essential) becauseinotify→iolibgrepslfp.hat build time. CI installs this automatically. For the E2E browser test:npm install && npx playwright install chromium.