PRD: Real freemocap sidecar service

[rupertgermann]

Problem Statement

Users can select Multi-camera sidecar in the mocap capture UI, but they cannot actually install or run the sidecar described by the docs. The documented command pip install rowing-tracker-sidecar fails because that package is not published, and the repository currently contains only the app-side HTTP/WebSocket client, tests, and a mock sidecar contract. As a result, a user with real cameras and FreeMoCap cannot record a real sidecar-3d mocap session through Rowing Tracker.

Solution

Build a repo-owned Python sidecar service that users can install locally and run as rowing-tracker-sidecar --port 8765. The service wraps FreeMoCap or a narrow FreeMoCap-compatible adapter, exposes the ADR-0005 localhost contract, discovers available cameras, reports readiness, starts and stops capture sessions, streams schema-v2 3D keypoint frames over WebSocket, returns calibration traceability, and gives clear degraded/error states when hardware or calibration is not ready.

From the user's perspective, the flow should become: install the sidecar from the repo-supported package path, run it locally, open /mocap, enable Multi-camera sidecar, see a ready status with camera count/fps, click Start sidecar capture, row the session, click Stop, and open the stored replay/analysis as a normal mocap session with capturePerspective = sidecar-3d.

User Stories

1. As a rower, I want the documented sidecar install command to work or point to a working repo-supported install path, so that I can start setup without guessing.
2. As a rower, I want to run rowing-tracker-sidecar --port 8765, so that the app can connect to a local motion-capture service.
3. As a rower, I want the sidecar to report whether my camera rig is ready, so that I know whether I can record before starting a session.
4. As a rower, I want the app's Multi-camera sidecar toggle to show the real camera count and fps, so that I can confirm it is talking to my hardware.
5. As a rower, I want capture start to arm the sidecar and create a traceable session id, so that the row is recorded as one coherent mocap session.
6. As a rower, I want the sidecar to return a calibration id when calibration is available, so that I can trace which rig calibration produced the session.
7. As a rower, I want the sidecar to refuse capture with a clear error when calibration or cameras are missing, so that bad recordings are not silently stored.
8. As a rower, I want live 3D pose frames to stream into Rowing Tracker during capture, so that the app stores the same PoseFrameStream format it already analyzes.
9. As a rower, I want stopping the capture to flush frames and close the sidecar session, so that the replay contains the complete row.
10. As a rower, I want the stored sidecar session to finalize and analyze like browser mocap sessions, so that I can use the existing replay and posture analysis views.
11. As a rower, I want sidecar sessions to expose sidecar-3d metrics where available, so that multi-camera capture unlocks depth-aware technique feedback.
12. As a rower, I want clear sidecar error messages in the UI when the process is unreachable, incompatible, or not ready, so that I know what to fix.
13. As a rower, I want the sidecar to keep raw video/pose capture local, so that precise body geometry is not sent to cloud services by accident.
14. As a developer, I want the sidecar package to be part of the repository, so that it can be tested, versioned, and installed without relying on a nonexistent PyPI package.
15. As a developer, I want the sidecar to preserve the ADR-0005 contract, so that the existing app-side client and capture lifecycle continue to work.
16. As a developer, I want a hardware-free test source to remain available, so that CI can verify the contract without physical cameras.
17. As a developer, I want a real FreeMoCap adapter boundary, so that FreeMoCap internals can change without spreading churn into the app or sidecar server.
18. As a developer, I want sidecar frames validated before streaming, so that malformed camera/runtime output does not corrupt stored pose blobs.
19. As a developer, I want explicit schema/version reporting, so that the app can reject incompatible sidecar builds before capture.
20. As a developer, I want fixture or smoke-test coverage for the installable command, so that packaging regressions are caught.
21. As a developer, I want the sidecar docs to distinguish mock, real hardware, and release/publishing flows, so that users do not try an unavailable package name again.
22. As a maintainer, I want FreeMoCap licensing and process-boundary assumptions documented, so that the AGPL dependency remains isolated from the Next.js app.
23. As a maintainer, I want the sidecar to fail closed on privacy-sensitive data sharing, so that raw frames and reconstructed body geometry stay on localhost.
24. As a maintainer, I want logs and troubleshooting output for camera discovery, calibration, and streaming failures, so that sidecar support is diagnosable.

Implementation Decisions

• Build a Python package owned by this repository with a console script named rowing-tracker-sidecar.
• Keep the sidecar as a separate local process. Do not link FreeMoCap code into the Next.js app.
• Preserve the existing ADR-0005 HTTP/WebSocket contract: /health, /session/start, /session/stop, and /pose-stream on localhost.
• Continue using keypointSchemaVersion = 2, coordinateSpace = world-mm-3d, capturePerspective = sidecar-3d, 33 BlazePose landmarks, and [x_mm, y_mm, z_mm, confidence] keypoints.
• Treat the existing app-side sidecar client and capture UI as the primary integration surface; change app code only where the real sidecar exposes missing readiness, diagnostics, or port configuration needs.
• Use a narrow sidecar-internal frame source interface. At minimum it should support a synthetic/mock source for tests and a FreeMoCap-backed source for real camera sessions.
• The FreeMoCap-backed source should prefer public/stable FreeMoCap APIs where available. If FreeMoCap only exposes batch or GUI-oriented flows, isolate the workaround behind the frame source boundary rather than leaking it into the server contract.
• Health should report ready, initializing, or error, plus fps, camera count, and schema version. It may include extra diagnostics, but app compatibility depends on the current fields.
• Session start should fail if the sidecar cannot produce schema-v2 frames for the configured rig.
• Session stop should flush and close the stream before returning success.
• Calibration traceability belongs to the sidecar. Rowing Tracker stores the returned calibrationId; it does not own the Charuco calibration workflow.
• Documentation should replace the currently broken public-PyPI install claim with a working repo-local install path. Public PyPI publishing is a separate release activity unless credentials and ownership are provided.
• The sidecar should remain local-only by default: bind to localhost, avoid cloud calls, and document that raw pose/video data stays on the user's machine.

Testing Decisions

• The highest-value test seam is the existing sidecar contract as observed by Rowing Tracker: run a sidecar process, connect through the app-side client, stream schema-v2 frames, persist/finalize the blob, and assert that sidecar-3d analysis produces derived rows.
• Good tests should verify external behavior at the contract boundary, not private camera-loop internals.
• Reuse the existing mock contract tests as prior art: health, session start/stop, WebSocket frames, v2 blob persistence, finalization, and post-session analysis.
• Add Python package tests for CLI startup, health response shape, one-session lifecycle, WebSocket frame shape, and graceful failure states.
• Add at least one hardware-free end-to-end test source that emits deterministic schema-v2 frames through the real sidecar server.
• Add a manual or hardware-gated smoke test for a real FreeMoCap camera rig. It should be documented and skippable in CI.
• Add installation verification for the supported local package command and console script.
• Keep app E2E coverage focused on user-visible behavior: sidecar readiness, start capture, stop/finalize, and replay availability.

Out of Scope

• Retuning sidecar-only posture fault thresholds.
• Replacing the browser MediaPipe capture path.
• Building a full FreeMoCap GUI inside Rowing Tracker.
• Owning or redesigning FreeMoCap's Charuco calibration workflow.
• Sending raw frames, 3D keypoints, or reconstructed body geometry to cloud AI.
• Publishing to public PyPI unless release credentials and package ownership are explicitly provided.
• Supporting arbitrary non-BlazePose landmark schemas before a concrete adapter is designed.

Further Notes

• Current public FreeMoCap packaging exists as freemocap and supports Python 3.10 through 3.12; the repo-specific rowing-tracker-sidecar package currently does not exist on public PyPI.
• FreeMoCap is AGPL-licensed; keep the process boundary and document the licensing assumption before distributing bundled artifacts.
• The existing Rowing Tracker docs and ADR-0005 already define the app-side contract. This PRD makes the missing real sidecar service an owned deliverable rather than a placeholder.