Skip to content

popup: rewrite + revisit TTS highlight with focus-toggle to originating window #19

Description

@davidj4tech

Background

The TTS popup (tts-popup, bound to a tmux display-popup -E) has accreted a lot of behaviour in packages/audio-relay/src/agent_audio_relay/shell/bin/tts-popup — single-key hotkeys for transport, seek, speed, volume, channel-cycling, history walk, sentence highlight, etc. It works, but the script has grown to ~200+ lines of shell driving tmux primitives and tts-ctl calls, and several features (channels, highlight, history) were bolted on over multiple commits.

This issue tracks a rewrite of the popup as a proper module rather than a shell pile, plus revisiting a couple of the more interesting features along the way.

How the popup previously worked

Anchored by commit 6f3a75c ("popup: sentence-level highlight in caller pane during TTS playback"):

  • Bound to display-popup -E -w 20 -h 3 -x R -y 0 — a tiny top-right key-capture overlay; the visible UI is in tmux's status-right (progress bar).
  • The tmux binding sets TTS_POPUP_SESSION and TTS_POPUP_PANE='#{pane_id}' so the popup knows which session it's controlling and which pane invoked it.
  • Single-key hotkeys drive tts-ctl underneath (Space, r, h/l, H/L, -/=, [/], </>, 1-9, m, p, i, ?, q/Esc).
  • Sentence-level highlight (v): paints a tmux copy-mode selection on the sentence currently being spoken, in the originating pane (TTS_POPUP_PANE). tmux's copy-mode auto-scrolls to keep the selection visible — gives auto-scroll-with-speech UX for free. Source text comes from a <stem>.txt sidecar written by tts-drop alongside each mp3; segmentation is done by aar-spoken-text (spoken_text.py) which preserves char offsets so the popup can map mpv time-pos to a slice.
  • Auto-closes TTS_POPUP_AUTO_CLOSE seconds after playback ends (default 10; 0 disables); only arms once mpv has been observed playing at least once.

Relevant commits (chronological):

  • 6f3a75c — sentence-level highlight + sidecar text
  • c3a51af / bb382b2 / 1cf2441 / dde8d38 — multi-channel support (music + voice), channel re-probe on keypress, background fetcher
  • ec65bae — cross-channel cycle + session overlay fallback
  • 4b3c9a0 — chapter-aware label + marquee scroll
  • 1b24c62 — per-channel history walk
  • 6cf40a5 — speed control [ / ]
  • 38a1b96 — p / 1-9 / i additions
  • c433019f hotkey → tts-pick fzf picker
  • d8f3554 — prefer non-idle channel + probe stdin-close race fix
  • 435c6fd — Space replays global latest
  • 511938c — popup Tab sees local Termux channels

Goals of the rewrite

  1. Move the popup from a sprawling shell script to a proper Python module in packages/audio-relay (sibling to spoken_text.py, etc.) — easier to test, easier to extend.
  2. Preserve all existing hotkeys and behaviour.
  3. Cleaner separation between state model (what's playing, where, on which channel) and render (the 3-line popup + status-right).
  4. Set up the originator-metadata pattern so future features (see below) aren't bolted on.

Re-address: TTS text highlighting + focus-toggle

Currently v toggles the sentence highlight in the originating pane. As part of the rewrite, extend this so toggling highlight on also switches the tmux client to the originating window, and toggling it off returns to the previously-focused pane.

Design notes (worked out in conversation):

  • Bidirectional: toggle-on → tmux switch-client/select-window to TTS_POPUP_PANE; capture the prior pane so toggle-off restores it.
  • Current tmux session only. If the originating pane is in a different session, do not auto-switch — show a transient status/toast like 🔊 speaking in session: <name> and let the user switch-client themselves.
  • Originator metadata: speech-queue entries should record where the utterance came from (tmux pane id today; eventually X11/Wayland window id, or "cron"/headless). The toggle dispatches to the right focus handler based on that metadata.
  • Edge cases:
    • Originating pane has been killed → toast "originating pane gone", no-op.
    • Non-tmux originators (cron, daemon-triggered notifications) → toast only.
  • Future / out of scope for v1: pluggable focus backends (wmctrl / xdotool / hyprctl) for switching to arbitrary windowed apps when the originator wasn't a tmux pane at all. Worth designing the originator-metadata shape with this in mind even though only the tmux backend will ship initially.

Acceptance

  • Popup rewritten as a Python module under packages/audio-relay; shell tts-popup becomes a thin entrypoint (or is retired in favour of python -m ...).
  • All existing hotkeys preserved with parity tests where feasible.
  • Speech queue entries carry originator metadata (tmux pane id at minimum).
  • v toggle: highlight-on jumps to originator pane in current session; highlight-off restores prior pane.
  • Cross-session originator → transient toast, no auto-switch.
  • Killed-pane and non-tmux-originator cases handled gracefully.

Related: #7 (Mopidy YouTube backend), agent-media monorepo restructure (#11 and phase PRs).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions