Skip to content

Add visionOS support #2

Description

@owenselles

Summary

Port CloudNow to visionOS so users can stream PC games via GeForce NOW in a visionOS window using a connected controller or keyboard/mouse.

What works without changes

  • All networking: SignalingClient, CloudMatchClient, GamesClient, ZoneClient, auth
  • SDPMunger (pure Foundation)
  • GCExtendedGamepad / GCMouse input paths in InputSender
  • All non-streaming SwiftUI views (Home, Library, Store, Settings, Login)
  • LiveKitWebRTC xcframework — already ships xros-arm64 slices

Required changes

VideoSurfaceView.swift — Full rewrite

layerClass override (AVSampleBufferDisplayLayer as backing layer) is tvOS/iOS-only. Replace with UIView + layer.addSublayer(displayLayer) + layoutSubviews frame management. WebRTCFrameRenderer core is unchanged.

InputSender.swift — Gate Siri Remote path

Wrap GCMicroGamepad, handleMicroGamepad, RemoteInputMode, and touchpad-as-mouse in #if os(tvOS). Extended gamepad + keyboard/mouse paths already work on visionOS.

StreamView.swift — Replace Siri Remote dismissal

.onExitCommand is silently ignored on visionOS. Replace with a SwiftUI .ornament toolbar or tap gesture. Evaluate fullScreenCover → dedicated WindowGroup for the stream.

GFNStreamController.swift — AVAudioSession guards

Gate .moviePlayback / .allowBluetooth options and deprecated requestRecordPermission callback behind #if os(tvOS).

QueueAdPlayerView.swift — Optional

Switch UIViewControllerRepresentable + AVPlayerViewController to VideoPlayer(player:) (5 lines, more idiomatic).

Xcode project + SPM

  • Add visionOS deployment target
  • Fork livekit/webrtc-xcframework, add .visionOS(.v1) to Package.swift platforms
  • Add NSMicrophoneUsageDescription to visionOS Info.plist

Verification

  • Build for xrOS Simulator — zero errors
  • End-to-end GFN session: video renders, controller input registers
  • Overlay show/dismiss works without Siri Remote
  • On-device AVSampleBufferDisplayLayer frame rendering confirmed

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions