Joystick Dungeon

💡 Inspiration

Joystick Dungeon is a continuation of Not Suspicious' exploration of hand interactions in tabletop mixed reality. It's a simple dungeon crawler tech demo with no obvious game design inspirations. It is primarily an exploration of the implications of a virtual joystick in hand-tracking, and an implementation of continuous microgesture input, in contrast to Meta's swipe-based discrete microgestures.

🚧 How we built it

This prototype was built in Unity over the course of 3 days. It's a custom "microgesture" solution built on top of OpenXR.

🎓 What we learned

This project explores indirect hand interactions, in contrast to our previous titles - Airspace Defender and Banners & Bastions - which rely on direct hand interactions. With indirect input, players manipulate virtual objects through a proxy mechanism (such as a virtual joystick) rather than interacting with the virtual objects directly. In this experiment, the virtual joystick provides fine-grained control over character rotation and introduces a more subtle, low-effort input method that may benefit users with limited mobility. The trade-off, however, is that indirect input is inherently more abstract and less intuitive, resulting in a less immersive experience compared with the natural pinch-and-grab gestures used in our previous titles. Additionally, we observed that indirect interaction encourages players to rest their hands in their lap, since no explicit contact with virtual objects is required. This posture, however, leads to diminished hand-tracking accuracy, highlighting the need for robust downward-facing cameras or neural wristband input devices to maintain consistent tracking quality.

🚀 What comes next?

If we were to continue developing a tabletop mixed reality dungeon crawler, our first step would be to return to direct hand interactions, as used in Airspace Defender and Banners & Bastions. While the virtual joystick's indirect input offered some potential accessibility benefits for users with limited mobility, those advantages can be replicated - and in many cases exceeded - using Touch controllers, which provide more reliable tracking and greater precision. For the vast majority of users, however, direct hand interactions would be the most natural form of input. If this project were to get funded, we would add multiplayer to it, because dungeon crawlin' is more fun with friends. Naturally, we would also build an actual game - not just a research tech demo, as is currently the case with this prototype.