Skip to content

ALXR Client

Jump to bottom
korejan edited this page Oct 4, 2024 · 120 revisions

About & Status

ALXR is a cross-platform/device OpenXR based client for ALVR. It can be used to stream to a variety of standalone, wired, smartphone, DIY VR and AR headsets with support for features such as hand-tracking, facial & eye tracking, Quest/Pico/Focus3/XR Elite/Lynx R-1 passthrough modes, AR non-opaque environmental blend modes.

Project development videos can found here

Stable release builds are here, Nightly builds are here

This project is still a work in progress, audio & microphone support is currently android (all flavors) only.

Pico users please note only Pico 4 and Pico Neo 3 (any variant) have OpenXR runtimes.

HoloLens (1st gen) does not have an OpenXR runtime, this device is not supported, only HoloLens 2 has an OpenXR runtime and is supported.

Tested OpenXR Runtimes

  • Lynx R-1
  • Magic Leap 2 (device/standalone runtime)
  • Magic Leap Remote Rendering
  • Windows Mixed Reality
    • Hololens 2 (Arm64, x64-emulator), Windows-x64
  • Oculus PC and Mobile
  • Pico OpenXR Mobile
  • Vive Wave OpenXR
  • VirtualDesktop-OpenXR (VDXR)
  • SteamVR (OpenXR) - Linux and Windows
  • Monado
    • Linux, Windows, Android
  • Meta XR Simulator

Tested HMDs / Devices

  • Lynx R-1
  • Magic Leap 2
  • Quest 1 / 2 / Pro (Both Mobile & PC Oculus OpenXR run-times)
  • Pico 4
  • Pico Neo 3 Link - Pro, Pro Eye with PUI v4.8.x and above
  • XR Elite
  • Vive Focus 3
  • Rift CV1
  • HTC Vive
  • Valve Index
  • Pimax 5k
  • Lenovo Explorer
  • Hololens 2 (arm64-device & x64-Emulator)
  • Leap Motion Controller (Ultraleap's OpenXR API Layer for EXT_hand_tracking extension)
  • Asus Rog 2 Phone
  • Samsung Galaxy S10/S20/S22
  • Xiaomi Mi 10T Pro
  • Poco X3 Pro

Usage

All Versions

Download the ALVR server, follow the steps to install, setup and run on any of the supported platforms.

Non Android Platforms (Linux, Windows, etc)

  1. Set an active OpenXR runtime for the headset you wish the client to use. If you do not know how to do this you can use the tool OpenXR explorer.
  2. Start the ALVR server/SteamVR
  3. Run alxr-client(.exe), some of the available flags:
    • -g, --graphics Options: D3D11/D3D12/Vulkan/Vulkan2/auto default is auto when not specificed.
    • d, --decoder Options: NVDEC/CUVID/D3D11VA/VAAPI/CPU default is D3D11VA on windows and VAAPI on linux
    • --decoder-thread-count=N Only applies to CPU based decoding.
    • --localhost When running the client and server on the same PC (check below).
    • -v, --verbose Enables verbose output to the command line.
    • --no-linearize-srgb Disables sRGB linerization, use this if the output in your headset looks to "dark"
  4. The first time the client runs, a new connection should be made to the server, click on the trust button.
  5. To exit the client, hold the controller menu button for 4 seconds.

UWP (x64/arm64) / Hololens 2 (Device / Emulator)

  1. For Hololens 2/WMR users, make sure WMR OpenXR runtime is up to date, you can obtain the latest version from the Microsoft Store.
  2. Install the uwp app bundle (alxr-client-uwp_x.x.x.x_x64_arm64(_debug).msixbundle)
    • Prior to this you may need to install the app certifcate alxr_client_TemporaryKey.pfx (obtained from the release page).
  3. For windows desktop users, set an active OpenXR runtime for the headset you wish the client to use. If you do not know how to do this you can use the tool OpenXR explorer.
  4. For Hololens 2 Emulator users only:
    • Server client discovery may not work in which case you can manually add an entry to the trusted client list with the emulator adapter address (note this address always changes between sesions).
    • You will need to forward ports in the VM for the server to establish a connection: emulator_port_fordwarding
  5. Run alxr-client-uwp from the start menu.
  6. The first time the client runs, a new connection should be made to the server, click on the trust button.

Left closing/ed hand shape maps to grip value/click. Right closing/ed hand shape maps to trigger value/click.

Android (All Flavors - Quest/Pico/Generic/etc)

Requirements:

  • Min Android Version: Android 9
  • Vulkan support, required extensions (check on vulkan.gpuinfo and/or with VulkanCapsViewer):
    • VK_KHR_get_physical_device_properties2
    • VK_ANDROID_external_memory_android_hardware_buffer
    • VK_EXT_queue_family_foreign
    • VK_KHR_sampler_ycbcr_conversion
  1. Install alxr-client(-platform-specific).apk via sidequest or adb
  2. run alxr-client
  3. The first time the client runs, a new connection should be made to the server, click on the trust button.

If you would like to use a wired USB connection, please follow these instructions

Android - Generic

The generic version (alxr-client.apk) uses the cross-vendor android OpenXR loader.

If the device has a system installed OpenXR runtime compitable with the cross-vendor android loader (such as the Vive Focus 3) no futher action is required, if the runtime is not system/vendor provided (e.g. Monado for Android) you must do the following:

  1. Install runtime_broker (available on google playstore), this is the OpenXR runtime broker.
  2. Run the runtime broker and in the UI select an OpenXR runtime to make active:

Screenshot_20220608-093245235

  1. If installing on an android device without a vendor specifc OpenXR runtime (such as a smartphone), your only option is to use Monado for Android

Monado Android Users

Before starting please make sure that your device has the minimum required Vulkan extensions to run Monado here. To check this you can either check if your device details has already been uploaded to vulkan.gpuinfo or install a tool such as VulkanCapsViewer.

  1. Obtain a build of Monado for android, either by building from source or download a CI build image
  2. Install openxr_android-(inProcess|outOfProcess)-(debug|release).apk
    • outOfProcess is a library & background running service that enables features such system overlays on top of OpenXR apps.
    • inProcess is a shared library based OXR runtime only, currently this may work best for ALXR, especially on lower-end devices.
  3. Open Monado app and check it runs:

Screenshot_20220608-103204174

  1. While the Monado app is running, if a button is shown to enable vr mode/services please press it, you may still have "vr helper services" available even if Monado does not show this by going into the system settings and search for "vr helper", then enable Monado:

Screenshot_20220608-103603353

Screenshot_20220608-103615597

  1. Run OpenXR runtime broker and select Monado as the active OpenXR runtime.
  2. Test everything works by installing and running the hello_xr app
  3. If your device supports selecting diffent display refresh rates, Monado has partial support for the XR_FB_display_refresh_rate OpenXR extension. Before running ALXR (or any OpenXR android app that supports using this extension) select your desired refresh rate.
  4. Just before running alxr-client (or other OpenXR apps), place/hold the device in the desired orientation as Monado will use this as the base orientation on start-up.
  5. Run alxr-client
  6. The first time the client runs, a new connection should be made to the server, click on the trust button.

Client & Server on the same PC.

To run the client on the same device / host OS as the server, use the --localhost flag. Please note, currently this only works with a TCP connection.

If you plan to use the "Tracking Only Ref" option with two headsets on the same host OS, this will only work under these conditions:

  • SteamVR's OpenXR runtime must not be the active runtime (headset such as Vive (Pro), Valve Index only have OpenXR runtimes via SteamVR).
  • The first, main headset must run an OpenVR app

Build From Soure

Prerequisites

All Target Platform Clients

Windows Client

  • Intall Visual Studio 2022, make sure the VS packages for the C++ compiler, cmake and Ninja are selected.
  • Install Vulkan SDK (make sure glslangValidator is selected)
  • Optional CUDA-toolkit (for cude-interop)

UWP/Hololens 2 Client

  • Intall Visual Studio 2022, make sure the VS packages for the C++ compiler and cmake are selected.
  • Install the base UWP installer packages, make sure the following are selected:
    • "C++ (v143) Universal Windows Platform tools" (for x64 targets such as Hololens2 emulator)
      • vs installer component: Microsoft.VisualStudio.ComponentGroup.UWP.VC
    • "C++ (v143) Universal Windows Platform tools (AMR64)"
      • vs installer component: Microsoft.VisualStudio.Component.UWP.VC.ARM64
    • "Windows 10 SDK (10.0.20348.0)"
  • In addition to the base rust(up) install, you will need to install nightly rust builds for UWP builds, you can run the following:
    1. rustup toolchain install nightly
    2. rustup component add rust-src --toolchain nightly-x86_64-pc-windows-msvc
    3. rustup update

Linux/Other Client

  • Clang(-C++)/gcc(g++)
  • CMake
  • Ninja (preferred, cmake generator is overridable)
  • Vulkan SDK (make sure glslangValidator is installed)
  • Optional CUDA-toolkit (for cude-interop)
  • ffmpeg

Android Clients (All Flavors)

  • If host is Linux you may need to install the gcc-multilib package.
  • Install Android Studio,
    • JDK v17 is required.
  • Open the SDK Manager and install:
    • SDK Platforms -> Android API 34
    • SDK Tools -> SDK Build-Tools 34.0.0, NDK v26.0.10792818, CMAKE 3.22.1,
  • Check JDK v17 is installed/selected in sdkmanager/android-studo.
  • Set the environment variables ANDROID_SDK_ROOT, ANDROID_NDK_ROOT
  • You may also need to set JAVA_HOME, you can refer to the jre included with android studio, typically the path for this is:
    • Windows: C:\Program Files\Android\Android Studio\jbr
    • Linux: /opt/android-studio/jbr
  • If already defined, unset ANDROID_HOME prior to building ALXR.
  • Install rust components:
    • rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android i686-linux-android
    • cargo install cargo-apk --git https://github.com/korejan/cargo-apk.git --branch android-manifest-entries
      • Do not install other versions, they currently do not generate some of the required Android manifest entries.

Pre-Build

If the repository has been cloned with git clone only, before you begin making a build you will need to pull the ALXR-engine git-submodule:

  1. CD to root of ALVR folder
  2. run git submodule update --init --recursive

Deploy Builds for Non-Android Platforms (Linux/Win/etc)

  1. CD to root of ALVR folder
  2. cargo xtask build-alxr-client (--release) (--reproducible) (--fetch) (--bundle-ffmpeg*) (--no-nvidia**)
    • --bundle-ffmpeg is linux only
    • --no-nvidia disables CUDA interop.
  3. Check the folder *build/(debug/release)/alxr_client_(windows|linux)(_cuda)

Deploy Builds for UWP/Hololens 2

  1. CD to root of ALVR folder
  2. cargo xtask build-alxr-uwp(-x64* | -arm64*) (--release) (--reproducible) (--fetch)
  3. Check the folder *build/(debug/release)/alxr_client_uwp
  • Without the arch suffixes all arch targets are built into a single installable app bundle (.msixbundle).

Deploy Builds for Android (Any Flavour)

  1. CD to root of ALVR folder
  2. cargo xtask build-alxr-android (--release) (--reproducible) (--fetch) (--generic) (--oculus-quest) (--pico-neo) (--all-flavors)
    • OR: cargo xtask build-alxr-(quest|pico)
  3. Check the folder build/(debug/release)/alxr_client_android
  4. Install via Sidequest or ADB

Please note there is an issue with building android clients on windows, cargo-apk/xtask will hang waiting on the jdk compiler process to exit and never does, if you would like to still build on windows there are 2 workarounds you can either do:

Open openxr-client\alxr-engine-sys\android in android studio, pick the appriopriate "active build variant", build once and leave android-studio open and then run cargo apk/xtask as many times you want.

Or

Once the build reaches the point of hanging indefinitely, open task manager, search for OpenJDK Platform binary process (or whichever jdk compiler you maybe using) and when you see 0% CPU activity manually kill the process.

The first method is a lot simpler and you only need to build in android-studio once and leave it running.

Dev Builds for Non-Android Platforms (Linux/Win/etc)

  1. CD to the root of ALVR folder
  2. Optional: cargo update
  3. cargo build -p alxr-client (--release) (--features=cuda-interop)
  4. and/or cargo run -p alxr-client (--release) (--features=cuda-interop) (-- *run-flags*)

Dev Builds for UWP/Hololens 2

  1. In the command line run the vcvarsall.bat with either the following arguments:
    • For x64 builds: "vs-path/vcvarsall.bat" x64 uwp 10.0.20348.0 -vcvars_ver=14.32.31326
    • For arm64 builds: "vs-path/vcvarsall.bat" amd64_arm64 uwp 10.0.20348.0 -vcvars_ver=14.32.31326
  2. If you have the standalone version of cmake installed and in the system path you may run into issues making a compatible UWP build, in this case change your system path to make visual studio's version of cmake be used.
  3. CD to the root of ALVR folder
  4. Optional: cargo update
  5. cargo +nightly build -Z build-std=std,panic_abort -p alxr-client-uwp --target (x64|aarch64)-uwp-windows-msvc (--release)

Dev Builds for Android (Any Flavor)

  1. CD to root of ALVR folder
  2. Optional: cargo update
  3. cargo apk build -p alxr-(android-client|client-quest|client-pico) (--release) (--target-dir=target/(quest|pico)*)
  4. and/or cargo apk run -p alxr-(android-client|client-quest|client-pico) (--release) (--target-dir=target/(quest|pico)*)
  • If testing multiple android flavors please set --target-dir option as workaround to a limitation of cargo-apk, if not incorrect shared libraries for specific platform will be packaged.

Please note there is an issue with building android clients on windows, cargo-apk/xtask will hang waiting on the jdk compiler process to exit and never does, if you would like to still build on windows there are 2 workarounds you can either do:

Open openxr-client\alxr-engine-sys\android in android studio, pick the appriopriate "active build variant", build once and leave android-studio open and then run cargo apk/xtask as many times you want.

Or

Once the build reaches the point of hanging indefinitely, open task manager, search for OpenJDK Platform binary process (or whichever jdk compiler you maybe using) and when you see 0% CPU activity manually kill the process.

The first method is a lot simpler and you only need to build in android-studio once and leave it running.

Clone this wiki locally