Skip to content

Statistics HUD: fix video zoom, two-column layout, drop redundant diagnostics#76

Draft
oliversluke wants to merge 4 commits into
owenselles:mainfrom
oliversluke:fix/stats-hud-video-zoom
Draft

Statistics HUD: fix video zoom, two-column layout, drop redundant diagnostics#76
oliversluke wants to merge 4 commits into
owenselles:mainfrom
oliversluke:fix/stats-hud-video-zoom

Conversation

@oliversluke

@oliversluke oliversluke commented Jul 11, 2026

Copy link
Copy Markdown
Contributor

Three related fixes to the in-stream Statistics HUD, all around the tall Standard + Diagnostics case. (Folds in the two-column work that was previously #77.)

1. Stop the HUD zooming the video

The video and the HUD/pause-menu/time-warning were laid out as siblings of a ZStack. The video is a UIViewControllerRepresentable, which — unlike a plain view — can expand to the union of its ZStack siblings' content; a tall Standard+Diagnostics HUD grew the layout and videoGravity = .resizeAspectFill then zoomed/cropped the picture. Moving the HUD into an .overlay of the video fixes it: an overlay is sized to the view it decorates and can never grow it.

(Note: the earlier "padding inside the flexible frame" fix from #62 wasn't reverted — it just only stopped the padding from growing the layout, not a tall HUD's content height.)

2. Two-column Standard HUD when Diagnostics is on

Standard alone fits the screen, but enabling Diagnostics appends the whole Debug section and pushes the panel off the bottom. With Diagnostics on, the Standard HUD now lays out in two columns — core stats (Network/Video/Audio/Session) on the left, Debug on the right — so everything stays on screen. Compact and Standard-without-Diagnostics are unchanged (single column).

3. Drop redundant free-form diagnostics

The bottom of the Debug section carried six free-form sentence lines left over from before the structured rows existed, largely repeating data now shown structurally (session summary → Resolution/FPS/Codec; color & decoded-video → Video Format; app-queue/display-layer "dropped" → Video Drops/freezes). The German "App N in Warteschlange" also mis-read an ever-growing enqueued counter as a queue backlog. Removed those six lines plus the now-orphaned code: a GFNStreamController property, a StatsHUD helper, six AppLocalization formatters, and their ten dead L10n keys across all 34 locale files. The conditional color-fallback warning and RTC-log indicator stay.

Testing

Device-tested on Apple TV 4K (Standard + Diagnostics): no zoom, two columns fit on screen, lean Debug section. Simulator build + SwiftFormat + SwiftLint (strict) pass.

🤖 Generated with Claude Code

@aarikmudgal

Copy link
Copy Markdown
Collaborator

The game stream zoom issues seems to be fixed but the standard overlay still shifts. And double information also in the right column, we can remove the bottom part and keep the segmented info.

Simulator Screenshot - Apple TV 4K (3rd generation) - 2026-07-11 at 12 56 03 Simulator Screenshot - Apple TV 4K (3rd generation) - 2026-07-11 at 12 55 53

@aarikmudgal

Copy link
Copy Markdown
Collaborator

Text "Session 68672b52 • 80-250-97-37. cloudma...
App 4.005 enqueued • 0 dropped • 0 backpre...
Sample 0.01 ms • convert 0.00 ms
Display layer metrics 60 frames • 0 dropped •...
Color Automatic • requested SDR 8-bit • dete..." at the bottom is obsolete and duplicate.

The in-stream player laid the video and the HUD/pause-menu/time-warning
out as siblings of a ZStack. The video is a UIViewControllerRepresentable,
which — unlike a plain view — can expand to the union of its ZStack
siblings' content. A tall Standard statistics HUD (especially with
Diagnostics enabled) grew the ZStack past the screen, and the video's
resizeAspectFill gravity then zoomed/cropped the picture to fill the
taller bounds.

Move the HUD/menu/warning into an .overlay of the video. An overlay is
sized to the view it decorates and can never grow it, so the video's
bounds stay fixed to the screen regardless of how tall the HUD gets.
Positioning and transitions are unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@oliversluke oliversluke force-pushed the fix/stats-hud-video-zoom branch from 703a722 to 7abf152 Compare July 11, 2026 11:05
@oliversluke oliversluke marked this pull request as draft July 11, 2026 11:06
oliversluke added a commit to oliversluke/CloudNow that referenced this pull request Jul 11, 2026
The Standard statistics HUD fits the screen comfortably on its own, but
enabling Diagnostics appends the whole Debug section (~15-25 extra lines,
several of them wrapping sentences), roughly doubling the panel height so
the lower rows run off the bottom of the screen.

When Diagnostics is on, lay the Standard HUD out in two columns — core
stats (Network/Video/Audio/Session) on the left, Debug on the right —
which roughly halves the height so everything stays visible. Compact and
Standard-without-Diagnostics are unchanged (single column). Splits the
panel/column layout helpers so the fixed 380pt width now lives on each
column rather than the whole panel.

Complements owenselles#76 (which stops the tall HUD from zooming the video); this
keeps the extra Diagnostics rows on screen.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
oliversluke and others added 2 commits July 11, 2026 13:32
The Standard statistics HUD fits the screen comfortably on its own, but
enabling Diagnostics appends the whole Debug section (~15-25 extra lines,
several of them wrapping sentences), roughly doubling the panel height so
the lower rows run off the bottom of the screen.

When Diagnostics is on, lay the Standard HUD out in two columns — core
stats (Network/Video/Audio/Session) on the left, Debug on the right —
which roughly halves the height so everything stays visible. Compact and
Standard-without-Diagnostics are unchanged (single column). Splits the
panel/column layout helpers so the fixed 380pt width now lives on each
column rather than the whole panel.

Complements owenselles#76 (which stops the tall HUD from zooming the video); this
keeps the extra Diagnostics rows on screen.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The bottom of the Debug section carried six free-form sentence lines
left over from before the structured Network/Video/Audio/Session/Debug
rows existed. They largely repeat data now shown in those rows — e.g.
the session-summary line duplicates Resolution/FPS/Codec, the color and
decoded-video lines duplicate the Video Format row, and the app-queue /
display-layer "dropped" counts overlap Video Drops/freezes. The German
"App N in Warteschlange" reading of the ever-growing enqueued counter
was also misleading.

Remove the session-summary, app-queue, sample/convert, display-layer,
color-diagnostic, and decoded-video lines, keeping the conditional
fallback warning and RTC-log status. Also removes the now-orphaned code:
GFNStreamController.diagnosticSessionSummary, the StatsHUDView
detectedColorLabel helper, six AppLocalization formatters
(diagnosticSessionSummary, displayLayerMetrics, colorDiagnosticStatus,
decodedVideoStatus, hdrSupportLabel, decoderPathLabel), and their ten
now-dead L10n keys across all 34 locale files.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@oliversluke oliversluke marked this pull request as ready for review July 11, 2026 11:51
@oliversluke oliversluke changed the title Fix Statistics HUD zooming the video (Standard + Diagnostics) Statistics HUD: fix video zoom, two-column layout, drop redundant diagnostics Jul 11, 2026
@oliversluke

oliversluke commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

@aarikmudgal Updated this PR to cover everything we discussed, and marked it ready for review:

  1. Zoom fixed (the overlay change).
  2. Two-column Standard HUD when Diagnostics is on — the layout you saw running off the bottom now splits into two columns (core stats left, Debug right), so it stays on screen. This was Two-column Standard HUD when Diagnostics is on #77, now folded in here and that PR closed.
  3. Removed the redundant bottom lines — all six free-form sentences are gone (session summary, app-queue, sample/convert, display-layer, color, and decoded-video), keeping only the structured rows plus the conditional fallback warning and RTC-log indicator. Also swept the now-dead code behind them (a controller property, a helper, six localization formatters, and their L10n keys across all 34 files).

Device-tested on Apple TV 4K. Ready for another look.

@aarikmudgal

aarikmudgal commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

@oliversluke I think the debug menu shouldn't be visible in "Release" run. Maybe only when the run is in debug mode.

image

And maybe the processing delay in the session metrics. rest lines only for debug run, what do you think?

Developer diagnostics (the Debug HUD section, RTC event log) are now a
DEBUG-build-only feature:
- SettingsView hides the whole Diagnostics section under #if DEBUG.
- normalizedForClient forces diagnosticsEnabled off under #if !DEBUG, so
  a persisted flag (set on a debug build or carried over) is never
  honored in Release — it runs at launch and before every session, so
  the Debug section, stats collection, and RTC event log all stay off.

Since the Debug section disappears in Release, move the one metric worth
keeping for end users — Processing delay (WebRTC totalProcessingDelay) —
into the Standard HUD's Video section next to Decode time. Compact stays
minimal (unchanged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@oliversluke

Copy link
Copy Markdown
Contributor Author

@aarikmudgal Good calls — both done in the latest commit:

Diagnostics is now DEBUG-build-only. The Settings "Diagnostics" section is hidden under #if DEBUG, and normalizedForClient forces the flag off under #if !DEBUG — so even a value persisted from a debug build is never honored in Release (it runs at launch and before every session, so the Debug HUD section, stats collection, and RTC event log all stay off). A Release build simply has no diagnostics feature.

Processing delay stays visible. Since the whole Debug section disappears in Release, I moved that one metric (WebRTC totalProcessingDelay) into the Standard HUD's Video section, next to Decode time — so end users still see it in Standard. Compact stays minimal, as you suggested.

So this PR now covers: video-zoom fix, two-column Standard HUD, redundant-diagnostics removal, DEBUG-only diagnostics gating, and the processing-delay move. Ready for another look.

@aarikmudgal

aarikmudgal commented Jul 11, 2026

Copy link
Copy Markdown
Collaborator

@oliversluke I think there was a misunderstanding, we need the diagnostics menu in release as well only this below section to be in Debug app mode. Also the processing delay can be moved to session on left column bottom.

image

So in release mode the standard diagnostic view will be one column (addition of Processing delay at the bottom) and when I debug mode we should have second column and in future extend more debug specific metrics in there.

@oliversluke

oliversluke commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

@aarikmudgal To make sure I build exactly what you have in mind (I think I over-gated it — killed the whole diagnostics feature in Release rather than just the low-level section), here's our current behavior per decision point. Could you fill in the right-hand column (or just correct my italic guesses)? Then I'll realign.

# Aspect Our current implementation What you want
1 Standard statistics HUD (Network / Video / Audio / Session rows) Available in both Release and Debug (driven by the pause-menu Statistics: Compact/Standard, not gated) This behaviour is correct
2 Debug section (2nd column: NACK/PLI/FIR, Retransmits, Input queue, Input buffer, Decoder, ICE path) Shown only when the user's Diagnostics toggle is ON; that toggle is force-disabled in Release, so it never appears in Release and appears in Debug only if the user turns it on This behaviour is correct
3 What gates the Debug section A user setting (diagnosticsEnabled), forced off in Release (sounds like you want: #if DEBUG build mode — auto in Debug, absent in Release?) No this is correct, we do not want Debug section of standard diagnostic view in release mode
4 Two-column layout Appears whenever the Debug section is shown → never in Release, only in Debug-with-toggle This behaviour is correct
5 Settings → "Diagnostics" menu (the toggle + RTC-log toggle) Hidden in Release (#if DEBUG), visible in Debug (you said keep it visible in Release too?) This in-app setting reflects if diagnostics menu is rendered at all in the options overlay during stream, we should keep it as-is unless we have better idea.(so yes these settings in app stays)
6 What the "Diagnostics" toggle controls if the Debug section becomes build-mode-gated Currently enables the Debug section + RTC event log + tracing (unclear in your model — nothing? just RTC-log / tracing?) Yes this was my fault, I created more confusion. We should only gate the Debug column of overlay based on app deployment mode, rest stays as per current behaviour in main branch
7 Processing delay row In the Video section (visible in Standard, Release + Debug) (you said: move to Session, left column bottom?) Correct, processing delay is something that is not debug specific rather a metric for the run so we should move it under session part of left column
8 Persisted diagnosticsEnabled = true opened in a Release build Ignored — forced to false yes this settings should stay in-app and also toggled on by default, the off, compact and standard mode should be persisted over it based on user's selection
9 RTC event log capture Under the Diagnostics section; gated on the toggle → off in Release Stays as it is

Once I have your answers I'll adjust the PR accordingly.

@aarikmudgal

Copy link
Copy Markdown
Collaborator

@oliversluke Updated

@aarikmudgal aarikmudgal self-assigned this Jul 11, 2026
@aarikmudgal aarikmudgal added bug Something isn't working enhancement New feature or request labels Jul 11, 2026
@oliversluke

Copy link
Copy Markdown
Contributor Author

@aarikmudgal Thanks for the answers — before I realign, one important thing to flag, because it changes what "keep it in Release" actually costs.

The "Diagnostics" toggle isn't the 2nd column — the column is just its visible readout. Flipping it on turns on developer instrumentation (the Settings subtitle even says "adds receiver timing, renderer metrics, frame counters and instruments signposts"). Concretely, ON does:

  • Per-frame renderer instrumentation — every decoded frame is measured (sample-creation/conversion time, display-layer frame/dropped/corrupted counters, backpressure, os_signposts). Off = zero cost; on = per-frame at 60 fps.
  • 1-second full WebRTC stats polling — forces a full RTCStatistics report every tick even when the HUD is off/compact (normally only every 5th tick), plus audio-sync parsing regardless of HUD level.
  • Continuous AudioSync logging — a detailed per-second line to the unified log.
  • Unlocks the RTC-event-log sub-toggle → writes a bounded 15 MB WebRTC protobuf trace to disk for the whole session.

So the implication of "keep the toggle in Release, only build-gate the column": a Release user who turns Diagnostics on pays for all that instrumentation (+ optional 15 MB disk capture) and sees nothing change on screen, because the column they'd be enabling is build-gated away. It's not "the toggle does nothing" — it's "the toggle silently adds overhead for no visible benefit." This is really a developer-instrumentation feature, not a user display option.

So I'd frame the choice as:

  • A — build-gate the whole bundle to Debug (column + per-frame metrics + 1 s stats + AudioSync log + RTC-log). Release has no Diagnostics section; end users never pay for dev telemetry. (Basically my earlier commit, minus the confusion.)
  • B — keep it in Release as you described, accepting the toggle is overhead-only there (no visible change), where the one tangible artifact is the RTC event log — genuinely useful if you want users to attach a WebRTC trace to a bug report.

If you want something user-facing in Release, the RTC event log is the piece worth surfacing; the rest is developer-only by nature. Happy to go either way — I just want the tradeoff to be deliberate.

To unblock what we already agree on: the video-zoom fix (7abf152) is fully independent and you've confirmed it works. I can split just that into its own small PR so you can merge it now, and keep this PR open for the diagnostics-gating decision. Want me to carve it out?

@aarikmudgal

Copy link
Copy Markdown
Collaborator

@oliversluke Indeed great points, lets go this way, by default the "Diagnostics" and "RTC Logging" are "Off" in the app but they are available. Even in Release bundle.
And rest of the behaviour stays the same. If the user enabled the diagnostics in the app only then he/she get "Compact" and "Standard" menu which shows different level of details only if the user wants it.
What do you think?
I surely cannot recommend removing entire diagnostic menu from release. This is sometimes better for users to figure out the issues on their network, etc than anything else. (And for nerds like me, sometimes even play while watching the stats)

@oliversluke

Copy link
Copy Markdown
Contributor Author

Thanks for all the back-and-forth on this! To keep things moving, I've split the video-zoom fix into its own PR — it's independent and you've already confirmed it works: #78. That one's ready to merge on its own whenever you like.

I'm marking this PR as draft for now so we can take our time getting the diagnostics side right. While implementing I noticed the boundary between the user-facing stats and the developer instrumentation is a little tangled in the current code (the "Debug" column is actually driven by the same lightweight once-a-second stats as the rest of the HUD, and some of the heavier per-frame capture isn't wired to anything anymore). I'd like to untangle that and write up a clearer proposal before we lock in the Release/Debug behaviour, then re-open for review. Appreciate your patience!

@aarikmudgal

Copy link
Copy Markdown
Collaborator

No worries @oliversluke We work for the best of community :) So all good. Thank you for your investments...

oliversluke added a commit to oliversluke/CloudNow that referenced this pull request Jul 11, 2026
The Standard statistics HUD fits the screen comfortably on its own, but
enabling Diagnostics appends the whole Debug section (~15-25 extra lines,
several of them wrapping sentences), roughly doubling the panel height so
the lower rows run off the bottom of the screen.

When Diagnostics is on, lay the Standard HUD out in two columns — core
stats (Network/Video/Audio/Session) on the left, Debug on the right —
which roughly halves the height so everything stays visible. Compact and
Standard-without-Diagnostics are unchanged (single column). Splits the
panel/column layout helpers so the fixed 380pt width now lives on each
column rather than the whole panel.

Complements owenselles#76 (which stops the tall HUD from zooming the video); this
keeps the extra Diagnostics rows on screen.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants