Summary
Consolidate falcon monitor functionality into falcon launch by adding attach/detach capability. This would allow users to:
- Detach from a running
falcon launch session while keeping training running
- Re-attach later to see current progress
- Eliminate the need for a separate
falcon monitor command
Background
The interactive TUI mode was added to falcon launch (see recent commits on feat/improved_output branch). Currently, falcon monitor is a separate Textual-based TUI that connects to running falcon processes via Ray actors (MonitorBridge).
Proposed Design
Detach
- User presses
d key during interactive mode
- Training continues in background (Ray actors keep running)
- Terminal is released back to user
MonitorBridge actor remains accessible for re-attachment
Attach
falcon launch --attach or falcon attach
- Discovers running falcon process via
ray.get_actor("falcon:monitor_bridge")
- Reconnects interactive TUI to show current state
- Resumes keyboard control (j/k for node switching, Ctrl+C for stop)
Implementation Notes
- The
MonitorBridge actor already provides all necessary status methods
- Interactive display already reads node logs from disk (works for attach)
- May need to persist some state (e.g., which nodes exist) for clean attach
- Consider: should attach support filtering to specific run-dir?
Related
falcon monitor can be deprecated once this is implemented
- Currently hidden from help via recent commit
Summary
Consolidate
falcon monitorfunctionality intofalcon launchby adding attach/detach capability. This would allow users to:falcon launchsession while keeping training runningfalcon monitorcommandBackground
The interactive TUI mode was added to
falcon launch(see recent commits onfeat/improved_outputbranch). Currently,falcon monitoris a separate Textual-based TUI that connects to running falcon processes via Ray actors (MonitorBridge).Proposed Design
Detach
dkey during interactive modeMonitorBridgeactor remains accessible for re-attachmentAttach
falcon launch --attachorfalcon attachray.get_actor("falcon:monitor_bridge")Implementation Notes
MonitorBridgeactor already provides all necessary status methodsRelated
falcon monitorcan be deprecated once this is implemented