Handheld xR Controller
A 3D-printed grip for phone-in-headset VR and AR, carrying a tracking image and a Bluetooth gamepad.
- Role
- Designer and builder
- Areas
- XR, Hardware, Computer Vision

A small prototype built to fill a gap in Google Cardboard-style headsets. In those headsets a phone slides into a simple viewer and does everything: display, tracking and, through its rear camera, augmented reality. What they lack is a decent way to interact. At most there's a single button on the viewer. We designed this controller to supplement that phone-in-headset setup for both VR and AR, without special electronics.
The grip is 3D printed and holds a square, high-contrast printed image above the hand. That's the kind of feature-rich picture image-target tracking locks onto well, so the phone's own camera, looking out through the viewer, can follow the image to know where the controller is and how it's turned, whether the app is showing the camera view for AR or a fully rendered scene for VR. For buttons and a thumbstick, a small off-the-shelf Bluetooth gamepad clips onto the front of the handle and pairs with the phone.
It belongs with our other phone-headset work: Cardforia, which combines Google Cardboard and Vuforia image tracking in one Unity package, and the phone-based AR headset that began HoloFlect. The approach also fits a conclusion from HoloFlect: for a lot of interaction, handheld controllers worked better than gesture.




