The Effects of Hand Tracking on User Performance: an experimental study of an object selection based memory game

Jamalian, Nima; Gillies, Marco; Leymarie, Frederic Fol and Pan, Xueni. 2022. 'The Effects of Hand Tracking on User Performance: an experimental study of an object selection based memory game'. In: 2022 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Singapore, Singapore 17-21 October 2022. [Conference or Workshop Item]

The_Effects_of_Hand_Tracking_on_User_Performance__an_experimentstudy_ofan_object_selection_based_memory_game__ISMAR_.pdf - Accepted Version

Download (17MB) | Preview
maxresdefault.jpg - Supplemental Material

Download (137kB) | Preview
[img] Video
The Effects of Hand Tracking on User Performance (VR, Memory Based Game) ISMAR Presentation-wZ-W0-NKI2w.webm - Supplemental Material

Download (37MB)

Abstract or Description

Until recently, Virtual Reality (VR) applications relied on controllers to enable user interaction in virtual environments. With advances in tracking technology, HMDs are now able to track the movements of users’ hands in real-time with significantly greater accuracy, allowing us to interact with the digital world directly with our hands. However, it is not entirely clear how hand tracking affects users’ performance. In this study, we investigate user performance using an in-game analytics-based assessment methodology for a VR memory puzzle task. We conducted a within-subjects experiment with 30 participants in three conditions: 1- Hand-tracking, 2- Controller Without Haptics, and 3- Controller With Haptics. In all our measurements (correct order and pattern, correct pattern only, and trial completion) except for the initial selection time, participants performed best with hand tracking. The use of controllers with haptics did not outperform controllers without haptics in most measures, possibly because other feedback cues compensated for the lack of haptics. This study helps us better understand the three selected interactivity methods when used in VR, as well as the importance of naturalistic experience in interaction design.

Item Type:

Conference or Workshop Item (Paper)

Identification Number (DOI):

Additional Information:

“© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.”

Xueni Pan is funded by the AHRC Project “Immersive, Innovative, and Interactive Experience” AH/T011416/1.


Virtual Reality, Hand Tracking, Cognition, Haptic, Multisensory Feedback

Related URLs:

Departments, Centres and Research Units:



27 December 2022Published

Event Location:

Singapore, Singapore

Date range:

17-21 October 2022

Item ID:


Date Deposited:

02 Feb 2023 09:26

Last Modified:

02 Feb 2023 10:30


View statistics for this item...

Edit Record Edit Record (login required)