Hand Interfaces: Using Hands to Imitate Objects in AR/VR for Expressive Interactions

University of California, Los Angeles Background
Virtual and Augmented Reality (AR/VR) systems allow users to interact with a virtual environment in an immersive and intuitive way. Typically, a headset provides visual and auditory input to this virtual environment while physical controllers are used as means to interact with this virtual environment. These controllers are often expensive and are by far the least immersive and intuitive aspect of AR/VR. The use of interactive gloves has been explored as an alternative, and while they are intuitive, they are prohibitively expensive and often are uncomfortable to wear for extended periods. AR/VR systems are in desperate need of an interface that provides the intuitive aspect of the gloves with a significant price reduction.
One promising solution is known as freehand AR/VR which uses cameras to visually track the user’s hands so the user is not restricted by clunky controllers or restrictive gloves. Prior work has been focused on modeling the user’s hands in the virtual experience so that they may directly interact with objects as they would in the real world. While this has been promising, it is not very intuitive due to the lack of haptic feedback. One alternative approach is to treat the user’s hands as objects rather than as hands interacting with virtual objects. The idea is rather than having a user hold a virtual wrench, the user could make a shape with their hands which will then act as the virtual wrench. There is a clear and present need for the development of such an intuitive approach to interacting with objects that is low cost and is significantly more comfortable than gloves and controllers.
Technology Overview
Researchers at UCLA have developed software to allow freehand AR/VR interface that relies on using the user’s hands as objects within the virtual environment. In this instance, the user can mimic an object by changing the orientation of their fingers which the software will then interpret. This allows the user’s hands to act as that object within the virtual environment and enables intuitive interaction with a virtual environment. The software eliminates the requirement of physical controllers and clunky gloves which reduces costs and barriers to widespread implementation.
Further Details
Hand Interfaces: Using Hands to Imitate Objects in AR/VR for Expressive Interactions.
Stage of Development
Software has been developed and successfully demonstrated.

Low Cost
No Physical Controller
Intuitive Use


Virtual Reality
Augmented Reality
Creative Design

Related Blog

Smart, interactive desk

Get ready to take your space management game to the next level with the University of Glasgow’s innovative project! By combining the

Mechanical Hamstring™

University of Delaware Technology Overview This device was created to allow athletes who suffer a hamstring strain to return to the field

Join Our Newsletter

                                                   Receive Innovation Updates, New Listing Highlights And More