Tap the ShapeTones: Exploring the Effects of Crossmodal Congruence in an Audio-Visual Interface

Tools

Metatla, Oussama; Correia, Nuno N.; Martin, Fiore; Bryan-Kinns, Nick and Stockman, Tony. 2016. 'Tap the ShapeTones: Exploring the Effects of Crossmodal Congruence in an Audio-Visual Interface'. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. San Jose, United States. [Conference or Workshop Item]

Preview

Text
COM_Correia_2016.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial.
Download (942kB) | Preview

Official URL: http://doi.acm.org/10.1145/2858036.2858456

Abstract or Description

There is growing interest in the application of crossmodal perception to interface design. However, most research has focused on task performance measures and often ignored user experience and engagement. We present an examination of crossmodal congruence in terms of performance and engagement in the context of a memory task of audio, visual, and audio-visual stimuli. Participants in a first study showed improved performance when using a visual congruent mapping that was cancelled by the addition of audio to the baseline conditions, and a subjective preference for the audio-visual stimulus that was not reflected in the objective data. Based on these findings, we designed an audio-visual memory game to examine the effects of crossmodal congruence on user experience and engagement. Results showed higher engagement levels with congruent displays with some reported preference for potential challenge and enjoyment that an incongruent display may support, particularly for increased task complexity.

Item Type:

Conference or Workshop Item (Paper)

Identification Number (DOI):

https://doi.org/10.1145/2858036.2858456

Additional Information:

This work was supported by EPSRC grant EP/J017205/1 and EU Marie Curie fellowship FP7 REA grant 627922.

Keywords:

audio-visual display, crossmodal congruence, Games, spatial mappings, User engagement, user experience

Departments, Centres and Research Units:

Computing > Embodied AudioVisual Interaction Group (EAVI)

Dates: