XR Robotics is an open-source initiative for developers and professionals in XR, robotics, and AI. We provide XRoboToolkit, a suite of cross-device SDKs, frameworks, and sample applications that enable real-time remote robot teleoperation through XR devices.
For more video demos, please follow our YouTube channel @XRRobotics.
| Repository | Description |
|---|---|
| XRoboToolkit-Teleop-Sample-Python | simulation and real robot teleoperation sample written in python |
| XRoboToolkit-Teleop-Sample-Cpp | Bimanuel robot platform teleoperation sample written in C++ |
| Repository | Description |
|---|---|
| XRoboToolkit-Unity-Client | Unity client source code for XR interface (PICO supported, more coming) |
| BRobotAssistantLib | Android library component for Unity robot interface |
| XRoboToolkit-PC-Service | Core C++ PC-side robotics service |
| XRoboToolkit-PC-Service-Pybind | Python binding for PC service SDK |
📄 Each repository contains its own README and setup instructions.
robot_teleoperation.mp4 |
simulation_teleoperation.mp4 |
Follow these steps to build and run a full XR-to-robot teleoperation sample on a PICO 4 Ultra headset and a Linux x86 PC. This sample has been tested only under the below system OS requirements: Linux x86 PC: Ubuntu 22.04 PICO 4 Ultra: User OS >5.12. Special permission with enterprise version and VST camera permission is required for headset camera access.
- Install XRoboToolkit-PC-Service
- Download deb package for ubuntu 22.04, or build from the repo source.
- To install, use command
sudo dpkg -i XRoboToolkit-PC-Service_1.0.0_ubuntu_22.04_amd64.deb
- Clone & Set Up Python Teleop Sample
- Install the XR App on Headset
- Turn on developer mode on Pico 4 Ultra headset first (Enable developer mode on Pico 4 Ultra), and make sure that adb is installed properly.
- Download XRoboToolkit-PICO.apk on a PC with adb installed.
- To install apk on the headset, use command
adb install -g XRoboToolkit-PICO.apk
- Run Sample in Simulated Environment or on Real Robot
- Connect robot PC and Pico 4 Ultra under the same network
- On robot PC, double click app icon of
XRoboToolkit-PC-Serviceor run service/opt/apps/roboticsservice/runService.sh - Open app
XRoboToolkiton the Pico headset. Details of the Unity app can be found in the Unity source repo. - Follow instructions on XRoboToolkit-Teleop-Sample-Python to run test in simulated environment or on real robot.
- 🌍 Cross-device XR support via OpenXR
- 📡 Low-latency video streaming
- <100ms latency, <1% packet loss for both video and command
- 🎮 Robot control via XR headset & controllers
- 🧪 Developer-friendly SDKs
- Native C++ and Unity (URP/Android) clients
- 🧱 Modular, expandable architecture for future robot and headset support
This project is initiated and maintained by PICO to open up its XR robotics framework for community use.
We welcome contributors from academia, industry, and the open-source robotics community.
- 💬 Join the XR Robotics Discord
- 🐞 Use Issues & Discussions in each repo
- 📩 Contact: Ning Yang (yangning726@gmail.com), Ke Jing (drkejing@gmail.com).
⚠️ This project is under active development. Contributions and feedback are highly encouraged.