Add FreeMoCap sidecar contract and client integration#75
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 030af5c5a8
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| key = self.headers.get("Sec-WebSocket-Key") | ||
| if self.headers.get("Upgrade", "").lower() != "websocket" or not key: | ||
| self._send_json(HTTPStatus.BAD_REQUEST, {"error": "Expected WebSocket upgrade"}) |
There was a problem hiding this comment.
When the sidecar is running, any webpage the user visits can open ws://localhost:<port>/pose-stream because the WebSocket upgrade accepts requests solely based on the upgrade key, and the HTTP endpoints also advertise Access-Control-Allow-Origin: *. In that scenario a malicious site can start/stop the local session endpoints and read live 3D body keypoints from localhost, bypassing the app's consent/privacy controls; the sidecar should reject unexpected Origin values (for both HTTP CORS and WebSocket upgrades) before streaming frames.
Useful? React with 👍 / 👎.
| tracked = sum(1 for confidence in confidences if confidence >= 0.5) | ||
| mean_confidence = sum(confidences) / len(confidences) |
There was a problem hiding this comment.
When a recorded FreeMoCap file has the rowing-relevant joints tracked but low-confidence unused landmarks such as face or hand points, these lines compute both tracked_count and mean_confidence across all 33 keypoints. The app turns meanConfidence < 0.6 or trackedCount < 13 into LOW_CONFIDENCE, so valid rowing data can be flagged degraded, while high-confidence unused landmarks can also mask missing knees or ankles. Compute these fields over the 13 rowing landmarks defined by the contract.
Useful? React with 👍 / 👎.
| frame = self.sidecar_state.source.frame(frame_index, timestamp_ms) | ||
| _send_websocket_text(self.request, json.dumps(frame, separators=(",", ":"))) |
There was a problem hiding this comment.
When recorded .npy or JSON data contains NaN/Infinity for missed landmarks, the source preserves those floats and this json.dumps call emits non-standard JSON tokens such as NaN. The browser client uses JSON.parse for WebSocket messages, so the first such frame is reported as a parse failure and the sidecar capture errors out; validate or sanitize non-finite coordinates/confidences before reporting the source ready or stream with strict JSON handling.
Useful? React with 👍 / 👎.
| ordered = sorted(keypoints, key=lambda item: int(item["index"])) | ||
| return [ |
There was a problem hiding this comment.
When recorded JSON/JSONL data contains ADR-style keypoint objects with a duplicate, missing, or out-of-range index, this loader sorts by the provided index but then drops it; frame() later re-enumerates the list as indexes 0–32. That makes malformed input report ready while silently assigning coordinates to the wrong BlazePose landmarks, corrupting the stored skeleton and posture analysis instead of failing readiness.
Useful? React with 👍 / 👎.
1 participant
Summary
Testing