An experimental application developed for the IMMAA project. This application is designed to record all failures related to VR during self or controller tracking.
The application aims to allow users to navigate various environments (indoor or outdoor) while capturing all parameters necessary to identify complete or degraded VR failures. It focuses specifically on Meta Quest 2, 3, and Pro. To determine when and under what conditions failures occur, the user's Passthrough view (captured via HMD) is recorded using SideQuest and OBS
The parameters are categorized into two parts: HMD and controller tracking, and Wi-Fi connection.
For HMD and Controllers tracking
- Tracking status and its validation
- Positions and orientation (quaternion)
These parameters help identify drifts, jitters and more — i.e., discrepancies in position or orientation between frames over time.
For WIFI connection
- Signal strength (RSSI)
- Frequency
- Link speed (including Tx/Rx link speed)
For more details, refer to past experiment resources here: URL.
To modify the application, ensure:
- The editor is set up with Oculus SDK, Android, and XR integration.
- All incorrect settings (e.g., Player settings) are fixed.
- An alternative IDE (e.g., Android Studio) is available for editing Android native plugins.
- The Barrier system is disabled via Developer Hub while using Passthrough.
- The application works exclusively with Oculus devices (1, 2, 3, Pro, and 3S).
- Native plugin connections are unsupported in the Editor; the script must be built and deployed to Oculus.
- The Oculus SDK may not function correctly in the Editor—always validate using a built app.
- Passthrough data is protected by Meta. The only method to access "raw graphic data" is via SideQuest.
- Streaming behavior differs by device (e.g., avoid recording both eyes on MQ3 to minimize latency).
- Some parameters are deprecated or limited except Rift S. Tools like the Oculus Debugger (via Quest Link) or Profiler Analyzer in Developer Hub may offer partial access, but availability varies between HMDs.
This Git repository provides three different architecture options. We are using the architecture from the WifiDataPage branch. This branch introduces a feature to send messages about changes (e.g., disconnections, reconnections, and measurement data outside the HMD view) to a logging website. We used our own server to collect these logs. If you wish to use this feature yourself, you’ll need to set up your own server and run the code provided in index.php. Then, update the server URL in the configuration file located at: C:/User/<user>/AppData/localLow/OrryInuCompany/failure_VR_project/Inputs/settings.config (This file is created automatically the first time you launch the application.) Alternatively, you can modify the URL directly within the Unity project. Now I will focus on explaining this architecture. If you're interested in exploring the other options, you can review their respective scripts, as they follow a similar structure.
The above schematic outlines the entire structure, which operates as a state machine. To initiate and measure, first connect all devices (HMD and controllers). Wi-Fi is optional. All connection data can be monitored via the website, although this is not the messenger's primary function.
Once all VR components are connected, measurement begins by pressing A/X. At this point, a red notification panel will appear for one second to help synchronize the application with OBS recordings. During this phase, users can perform various actions. The application cycles between tracking and export states until B/Y is pressed. During the cycle, data is measured and saved efficiently, minimizing unnecessary storage and avoiding data loss during failures. Two .csv files are created initially to store tracking and Wi-Fi data. Pressing B/Y ends the session, closes the files, resets values, and prepares for the next measurement. (Data can be retrieved via the project's source files using SideQuest.)
Additional functions:
- Control Panel: Displays parameters using the primary trigger.
- Messenger: Reports application status and errors on the linked webpage.
- Note: Messenger requires a Wi-Fi connection to send data.
- Other UI enhancements.
The experiment design and data preprocessing details can be found here.