Summary
Port CloudNow to visionOS so users can stream PC games via GeForce NOW in a visionOS window using a connected controller or keyboard/mouse.
What works without changes
- All networking:
SignalingClient, CloudMatchClient, GamesClient, ZoneClient, auth
SDPMunger (pure Foundation)
GCExtendedGamepad / GCMouse input paths in InputSender
- All non-streaming SwiftUI views (Home, Library, Store, Settings, Login)
- LiveKitWebRTC xcframework — already ships
xros-arm64 slices
Required changes
VideoSurfaceView.swift — Full rewrite
layerClass override (AVSampleBufferDisplayLayer as backing layer) is tvOS/iOS-only. Replace with UIView + layer.addSublayer(displayLayer) + layoutSubviews frame management. WebRTCFrameRenderer core is unchanged.
InputSender.swift — Gate Siri Remote path
Wrap GCMicroGamepad, handleMicroGamepad, RemoteInputMode, and touchpad-as-mouse in #if os(tvOS). Extended gamepad + keyboard/mouse paths already work on visionOS.
StreamView.swift — Replace Siri Remote dismissal
.onExitCommand is silently ignored on visionOS. Replace with a SwiftUI .ornament toolbar or tap gesture. Evaluate fullScreenCover → dedicated WindowGroup for the stream.
GFNStreamController.swift — AVAudioSession guards
Gate .moviePlayback / .allowBluetooth options and deprecated requestRecordPermission callback behind #if os(tvOS).
QueueAdPlayerView.swift — Optional
Switch UIViewControllerRepresentable + AVPlayerViewController to VideoPlayer(player:) (5 lines, more idiomatic).
Xcode project + SPM
- Add visionOS deployment target
- Fork
livekit/webrtc-xcframework, add .visionOS(.v1) to Package.swift platforms
- Add
NSMicrophoneUsageDescription to visionOS Info.plist
Verification
- Build for xrOS Simulator — zero errors
- End-to-end GFN session: video renders, controller input registers
- Overlay show/dismiss works without Siri Remote
- On-device AVSampleBufferDisplayLayer frame rendering confirmed
Summary
Port CloudNow to visionOS so users can stream PC games via GeForce NOW in a visionOS window using a connected controller or keyboard/mouse.
What works without changes
SignalingClient,CloudMatchClient,GamesClient,ZoneClient, authSDPMunger(pure Foundation)GCExtendedGamepad/GCMouseinput paths inInputSenderxros-arm64slicesRequired changes
VideoSurfaceView.swift— Full rewritelayerClassoverride (AVSampleBufferDisplayLayeras backing layer) is tvOS/iOS-only. Replace withUIView+layer.addSublayer(displayLayer)+layoutSubviewsframe management.WebRTCFrameRenderercore is unchanged.InputSender.swift— Gate Siri Remote pathWrap
GCMicroGamepad,handleMicroGamepad,RemoteInputMode, and touchpad-as-mouse in#if os(tvOS). Extended gamepad + keyboard/mouse paths already work on visionOS.StreamView.swift— Replace Siri Remote dismissal.onExitCommandis silently ignored on visionOS. Replace with a SwiftUI.ornamenttoolbar or tap gesture. EvaluatefullScreenCover→ dedicatedWindowGroupfor the stream.GFNStreamController.swift— AVAudioSession guardsGate
.moviePlayback/.allowBluetoothoptions and deprecatedrequestRecordPermissioncallback behind#if os(tvOS).QueueAdPlayerView.swift— OptionalSwitch
UIViewControllerRepresentable + AVPlayerViewControllertoVideoPlayer(player:)(5 lines, more idiomatic).Xcode project + SPM
livekit/webrtc-xcframework, add.visionOS(.v1)toPackage.swiftplatformsNSMicrophoneUsageDescriptionto visionOS Info.plistVerification