Add version history page and prosemirror-collab step sync#127
Open
hendriebeats wants to merge 53 commits into
Open
Add version history page and prosemirror-collab step sync#127hendriebeats wants to merge 53 commits into
hendriebeats wants to merge 53 commits into
Conversation
Introduces server-side step storage and version tracking as the
foundation for real-time collaborative editing and version history.
- Migration: adds `version` column to `document`; creates
`document_step` (persistent step log) and `document_snapshot`
(periodic full-doc checkpoints every 500 steps) tables
- Database.hs: Beam types for both new tables; `version :: Int32`
field on DocumentT
- Types.hs: DocVersionNum newtype
- Entity/Document.hs: getDocVersion, updateDocVersion, insertSteps,
getStepsSince, insertSnapshot, getLatestSnapshot, maybeTakeSnapshot;
GetDoc now includes current version (sent to client on open)
- Api/Websocket.hs: InUpdated now carries structured payload
{version, steps, clientId}; handleUpdated performs a transactional
version check — inserts steps and replies OutDocConfirmed on match,
or returns OutDocConflict with steps-since on mismatch; handleSave
triggers a snapshot when version % 500 == 0
Frontend collab integration (prosemirror-collab) comes in a follow-up
batch.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces the ad-hoc step-sending approach with the official collab
plugin, enabling proper version tracking and conflict resolution:
- Add prosemirror-collab dependency; register collab() plugin after
history() with per-tab sessionClientId (computerId + UUID)
- Send {version, steps, clientId} payloads instead of raw step arrays
- Handle DocConfirmed (server ack) via receiveTransaction to clear the
unconfirmed buffer; inflight flag prevents duplicate sends
- Handle DocConflict (version mismatch) by applying server steps and
letting sendableSteps rebase automatically on next dispatch
- Add version field to DocRaw and LastUpdate types
- DocUpdated broadcast now carries {version, steps, clientIds} so the
group-study side panel uses receiveTransaction too
Exposes two authenticated endpoints for the history UI: - GET /api/document/:documentId/history Returns step activity grouped into 5-minute buckets using raw SQL (date_trunc grouping not expressible in Beam DSL); restricted to steps authored by the requesting user. - GET /api/document/:documentId/at-version/:versionNum Returns the nearest snapshot plus steps needed to reconstruct the document at any target version. Both endpoints 403 if the user is not in the document's editor list.
- New /study/:id/history page with sidebar of version groups (5-minute buckets) and read-only ProseMirror preview of any selected version - Reconstruct historical doc from nearest snapshot + steps client-side using Transform; no server-side doc reconstruction needed - "Restore this version" saves the doc JSON to sessionStorage and navigates to /study/:id?restore=true; on next open the editor loads the restored content and saves it as a normal edit - Parcel multi-entry: history.tsx compiled alongside index.tsx - history.tsx loaded via /static/editor/history.js (prod) or Parcel dev server port (local)
- Move history API routes from /api/document/... to /document/... so they are served by the HTMX/Scotty server instead of the Servant API server - Add "Version History" link to the study page nav menu - Add comment explaining why two separate Parcel entry bundles (index.js + history.js) are safe to load independently - Ignore .playwright-mcp and .vscode in git
The history preview was broken for documents that had never triggered a collab snapshot (every 500 versions). It would either fetch a snapshot ahead of the requested version or return an empty object, causing ProseMirror to throw when reconstructing the doc. - Add `getLatestSnapshotBefore` to only consider snapshots at or before the target version, preventing a future snapshot from being used as the replay baseline - Add `getDocBase` as a fallback that reads the document's saved JSON and version directly from the document table - Fix the frontend API route prefix (/document/ not /api/document/) - Guard against an empty snapshot object before calling Node.fromJSON, falling back to an empty top-level node
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Raise ProseMirror newGroupDelay from 133ms to 250ms so undo steps group at a more natural typing pause - Change version history sub-group buckets from 133ms to 30-second windows, giving meaningful granularity within each 5-minute group - Fix baseline snapshot logic: capture a snapshot when none exists yet (not only at version 0), stamped at the current version so history can replay forward correctly
… types - Fix restore not persisting: restored doc is now written to localStorage and saved directly over the WebSocket immediately on page load, so navigating away and back no longer reverts to the pre-restore content. Root cause: onUpdate only fires on user edits, not on editor init, so the restored doc was displayed but never pushed to the server or local cache. - Separate checkRestoreDoc from checkLastUpdateFallback so the restore path is explicit and sessionStorage is consumed exactly once. - Collapse SubHistoryGroup into a type alias for HistoryGroup (identical shape in both Haskell and TypeScript). - Add aria-expanded / aria-controls to accordion buttons for screen reader support. - Suppress decorative CSS triangle characters from the accessibility tree via content alt-text syntax. - Remove dead commented-out visualViewport resize listener.
The top-level accordion items previously showed raw ProseMirror step counts, which were meaningless to users (a single word could be dozens of steps). They now show how many 30-second sub-sessions are inside, which matches exactly the number of items revealed on expand. - SQL: COUNT(DISTINCT floor(epoch/30)) instead of COUNT(*) to count distinct 30-second buckets within each 5-minute group - Label changed from "edits" to "sessions" to match the new meaning
- Replace bare placeholder text with a rich empty state (icon, heading, description, hint arrow) to orient new users - Add a version pill in the preview toolbar showing which snapshot is open; auto-hides via ResizeObserver when the toolbar is too narrow - Add a "← Versions" back button (hidden on desktop, shown on mobile) so users can return to the list after opening a preview - Implement a CSS-only mobile layout: sidebar and preview swap via the `preview-open` class on #historyLayout; JS toggles it when a version is selected or the back button is clicked - Rename `historyPreviewActions` → `previewToolbar` for clarity - Add a hint paragraph below the "Versions" header - Fix triangle expand/collapse glyphs to use Unicode escapes so they render reliably across browsers Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Swaps the janky document-name hyperlink in the history page header for a clean back-arrow icon (arrow2.svg) linking to the document, followed by a static "Version History" heading. Prevents long titles from breaking the mobile layout. Adds hover animation (gray circle + scale) and a keyboard focus ring for accessibility. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
hendriebeats
changed the title
- Show each bucket's latest edit time (endedAt) instead of earliest - Move the chevron expand button to the right side of each accordion row - Widen the expand button from 32px to 40px for easier tap/click target
- Insert a baseline snapshot at version 0 when a document is created, ensuring history replay has a starting point without relying on a lazy first-edit heuristic in the WebSocket handler (which is removed). - Change the top-level "step count" to count distinct 1-minute buckets of activity (i.e. editing sessions), and relabel the pill as "N session(s)" instead of "N edit(s)" to match the new semantics. - Drop `stepCount` from `SubHistoryGroup` — sub-items now show only the time, keeping the expanded view uncluttered. - Sub-item left margin and padding tweaks for better visual alignment.
- Fall back to the document's saved content (getDocBase) when no collab snapshot exists, so legacy documents reconstruct correctly instead of showing a blank document. - Mark loadSubGroups failures as retryable: only set subLoaded=true on success so collapsing and re-expanding retries the network request. - Add focus-visible rings to history sidebar buttons for keyboard accessibility. - Remove duplicate overflow-y scroll on #historyList outer container. - Remove dead .historyStepCount CSS rule (class no longer emitted). - Fix media query indentation for #restoreButton. - Add aria-hidden to decorative clock emoji in empty state. - Correct migration comment: snapshot cadence is every 50 steps, not 500.
hendriebeats
changed the title
Without a baseline snapshot, getDocAtVersion fell back to getDocBase, which returns the document's current (latest) version number and content. Because that version was already past any requested targetVersion, getStepsSince returned nothing and every preview rendered the same latest content. Two fixes: - Websocket.hs: before inserting steps, if no snapshot exists at or before currentVersion, save document.document as a baseline snapshot. This gives the step-replay logic a valid anchor for all future edits, including the first editing session on legacy documents. - DocumentHistory.hs: when the getDocBase fallback returns a version already ahead of targetVersion (legacy doc with no pre-step snapshot), fall back to (0, emptyDoc) instead of silently serving the wrong content for every version. Note: steps already recorded without a baseline (e.g. documents edited before this fix) cannot be reconstructed retroactively, as the pre-step-1 document state was never captured.
Two bugs in the version history feature: 1. The baseline snapshot for legacy documents was being saved in handleUpdated using document.document from the DB, which could be stale relative to what the client was actually editing (e.g. when the client loaded from localStorage). This caused the version history preview to show content that differed from the real document. Fix: move baseline snapshot logic to handleSave, where the client explicitly sends its current document content. This ensures the snapshot always reflects what the client is editing. 2. QuestionsView placed noQuestionsText inside contentDOM. ProseMirror manages contentDOM and when it detects DOM content that does not match the document model it reconciles by generating a replace step, converting the placeholder text into a real question node. Fix: move noQuestionsText outside contentDOM into a wrapper td; make contentDOM a div inside that same cell. The update() method now also toggles placeholder visibility when questions are added or removed. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The version list had no way to view a document's original state before any edits. This was especially confusing for documents that predated version history: opening and immediately editing would lump the baseline and the first session into one indistinguishable group. - Pin a flat "Document created" item at the bottom of the history list, visually styled like a group-level entry but without an accordion - Clicking it previews startVersion-1, the baseline snapshot taken before the very first step, so the pre-edit state is always reachable - The first session's edits remain in the normal accordion list so individual changes within that session are still accessible - Adds .historyFirstVersion CSS class with matching selected/hover/focus styles; deselection logic updated to cover the new class
Move the "capture baseline snapshot" logic from the DocSave handler into the DocUpdated (step-insertion) handler, and execute it before any steps are written. Previously, the snapshot was taken during DocSave using the client-supplied document content, which could differ from the server's stored state (e.g. if the client had localStorage content ahead of the DB). This caused history replay to start from a potentially incorrect baseline. By capturing the snapshot in DocUpdated — before any steps are inserted — and using getDocBase to read the actual stored document, replay is guaranteed to start from the server-authoritative pre-edit state.
Move baseline snapshot creation from the DocUpdated (step-insertion) handler to handleOpenDoc, and use an idempotent INSERT … ON CONFLICT DO NOTHING variant (insertSnapshotIfAbsent) to handle concurrent opens of the same legacy document safely. Previously the snapshot was captured just before the first steps were inserted, which required threading snapshot logic into the hot edit path. Doing it at open time is simpler, happens before any edits arrive, and correctly reflects the server-authoritative state at the moment the user opens the document. - Add insertSnapshotIfAbsent using raw SQL with ON CONFLICT DO NOTHING to avoid duplicate-key errors when multiple users open the same doc simultaneously. - Remove the snapshot check/insert from handleUpdated.
hendriebeats
added
Size: Large
The study page template loads the editor JS from
/static/editor/index.js (served by Scotty's static middleware)
whenever the env is not Dev "local" — which includes CI. The
workflow was starting a Parcel dev server instead of building,
so the file never existed and the editor never initialised.
The "Wait for frontend to be ready" health-check on port 3001
was hitting Scotty (which always runs there), masking the failure.
- Replace `npm run dev` + port-poll with `npm run build`
- Copy frontend/dist/{index,history}.js to backend/static/editor/
Downloading Chromium + font packages on every run was slow. Cache ~/.cache/ms-playwright keyed on e2e/package-lock.json so the browser binary is reused across runs. On cache hit, only system-level deps are (re)installed via playwright install-deps, which is fast since most packages are already present on ubuntu-22.04.
Three additional caches to reduce build time: - ~/.cabal/packages + ~/.cabal/store + backend/dist-newstyle, keyed on backend.cabal with a restore-key fallback. The Cabal store holds all compiled Haskell dependencies, which is the most expensive step to rebuild. A partial hit (source changed, deps unchanged) still reuses the compiled deps. - frontend/node_modules keyed on frontend/package-lock.json - e2e/node_modules keyed on e2e/package-lock.json npm install steps now use --prefer-offline so a warm node_modules cache avoids network fetches entirely.
window.isLocal is only set in Dev "local" mode. In CI (Dev "ci"), it is unset, so the frontend was connecting over wss:// against a plain-HTTP server — the handshake failed and the editor never initialised. Deriving the protocol from window.location.protocol is correct for all environments: http: → ws://, https: → wss://.
nix develop pulls down HLS, hoogle, hlint, fourmolu, lsp, etc. on every run from cache.nixos.org. Adding nix-community/cache-nix-action persists /nix/store between runs, keyed on flake.lock. A restore-key fallback allows partial hits when only some inputs change.
- Add Playwright JSON reporter in CI (writes to test-results/results.json) - After tests (pass or fail), post a formatted markdown table to the PR showing each test's status, name, and duration - Uses a hidden marker comment so re-runs update the same comment rather than creating new ones
E2E Tests ❌ — 6 passed, 8 failed
|
Tests the WebSocket collaboration flow end-to-end using two isolated browser contexts (same user, separate cookie jars): - Tab 1 types → Tab 2 receives via DocUpdated (and vice versa) - Concurrent edits trigger DocConflict → OT rebase → both tabs converge - Cold open of an already-edited document replays steps from DocOpened Adds: - openStudy molecule — navigates a second page to an existing study and waits for the editor to become interactive - EditorAtoms.assertContainsWithTimeout — SLA-bounded sync assertion - collab-sync.spec.ts organism with the four scenarios above - Branch suite trimmed to version-history + collab-sync only Also fixes navigateToHistory to remove the assertPageLoaded call that was causing a layer violation (molecule calling an atom assertion), and adds editor.assertVisible() before editor.assertContains() in the restore-version test so failures identify the correct missing element.
- e2e.yml: ignore PRs targeting main/master/production so tests only run on feature branch PRs, not on merge PRs - production.yml: remove pull_request trigger entirely so deployments only happen on direct push to the production branch
stats.ok is unreliable in Playwright 1.58 — replace with stats.unexpected === 0 as the success condition.
The comment step previously only ran on pull_request events, so pushes to a feature branch (e.g. triggered by a push trigger) silently skipped it. This meant the PR comment was never created or updated when CI ran via push. - Add push trigger to e2e.yml (branches-ignore: master) - Remove pull_request guard from the comment step - On push events, look up the open PR for the branch via the API; skip gracefully if none exists yet
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a version history page for documents and replaces the last-writer-wins sync model with a proper OT (operational transform) collab protocol using prosemirror-collab.
What changed
Version history page (
/document/:id/history)prosemirror-collab OT sync
prosemirror-collabto track a version number and produce/receive typed steps instead of raw JSON blobsdocument_steptable (versioned per doc)DocConflictwith the missing steps to rebase againstDocConfirmed/DocConflict/DocUpdatedreplace the previous singleDocUpdatedeventMap ConnId Connection(replacing the singleopenedByslot) and all receive broadcastsSnapshots
Reviewer Q&A
Why
insertSnapshotIfAbsentwith raw SQL instead of Beam?Beam does not expose
ON CONFLICT DO NOTHING. Two users opening the same legacy document concurrently could both try to insert a baseline snapshot; this prevents a unique-constraint error.Why is
initVersionset toe.doc.version(current server version) on restore instead of the restored content's original version?After a restore, the old content is immediately saved to the server at the current version via
SaveDoc. The collab plugin needs to agree with the server on version number so subsequent steps are accepted — using the old version would cause every step to be treated as a conflict.Why does
handleOpenDocstill send the full document viaSaveDocprotocol on restore?The server has no dedicated "restore" endpoint; reusing
SaveDocis the minimal-change approach that correctly updates thedocumentcolumn and advances the snapshot baseline.Why
replicate n payload.clientIdwhen buildingStepsPayload.clientIds?The collab protocol assigns one
clientIdper step. Since a singleInUpdatedmessage can carry multiple steps but they all come from the same client tab, we broadcast[clientId, clientId, …]to match the per-step shape thatreceiveTransactionon receivers expects.Why was
OutDocOpenedOther(kick-out logic) removed?The old model allowed only one editor at a time per document. With the OT model any number of editors can coexist, so the kick-out message is no longer needed or sent.
Why is
maybeTakeSnapshotcalled insidehandleSaverather thanhandleUpdated?handleSavealready receives the full document JSON (the client sends it).handleUpdatedonly receives steps; reconstructing the document there would require replaying all steps, which is wasteful given that the client does it anyway and sends the result viaSaveDoc.Why
sessionClientId = computerId + "_" + randomStrinstead of justcomputerId?computerIdis stable across tabs; if two tabs are open on the same machine and both send steps, the server would see duplicate clientIds andreceiveTransactionwould reject one as a duplicate. The random suffix makes each tab session unique.