MetaVR: Understanding metaphors in the mind and relation to emotion through immersive, spatial interaction

Tools

G. Djokic, Vesna; Shutova, Ekaterina and Fiebrink, Rebecca. 2021. 'MetaVR: Understanding metaphors in the mind and relation to emotion through immersive, spatial interaction'. In: CHI 2021 – Interactivity. Yokohama, Japan 8 – 13 May 2021. [Conference or Workshop Item]

Preview

Text
DjokicShutovaFiebrink_CHI2021Interactivity.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial.
Download (1MB) | Preview

Official URL: https://dl.acm.org/doi/10.1145/3411763.3451565

Abstract or Description

Metaphorical thinking acts as a bridge between embodiment and abstraction and helps to flexibly organize human knowledge and behavior. Yet its role in embodied human-computer interface de- sign, and its potential for supporting goals such as self-awareness and well-being, have not been extensively explored in the HCI community. We have designed a system called MetaVR to support the creation and exploration of immersive, multimodal, metaphoric experiences, in which people’s bodily actions in the physical world are linked to metaphorically relevant actions in a virtual reality world.

As a team of researchers in interaction, neuroscience, and linguistics, we have created MetaVR to support research exploring the impact of such metaphoric interactions on human emotion and well-being. We have used MetaVR to create a proof-of-concept interface for immersive, spatial interactions underpinned by the WELL-BEING is VERTICALITY conceptual mapping—the known association of ‘good’=‘up’ and ‘bad’=‘down’. Researchers and developers can currently interact with this proof of concept to configure various metaphoric interactions or personifications that have positive associations (e.g., ‘being like a butterfly’ or ‘being like a flower’) and also involve vertical motion (e.g., a butterfly might fly upwards, or a flower might bloom upwards). Importantly, the metaphoric interactions supported in MetaVR do not link human movement to VR actions in one-to-one ways, but rather use abstracted relational mappings in which events in VR (e.g., the blooming of a virtual flower) are contingent not merely on a “correct” gesture being per- formed, but on aspects of verticality exhibited in human movement (e.g., in a very simple case, the time a person’s hands spend above some height threshold).

This work thus serves as a small-scale vehicle for us to re- search how such interactions may impact well-being. Relatedly, it highlights the potential of using virtual embodied interaction as a tool to study cognitive processes involved in more deliberate/functional uses of metaphor and how this relates to emotion processing. By demonstrating MetaVR and metaphoric interactions designed with it at CHI Interactivity, and by offering the MetaVR tool to other researchers, we hope to inspire new perspectives, dis- cussion, and research within the HCI community about the role that such metaphoric interaction may play, in interfaces designed for well-being and beyond.

Item Type:

Conference or Workshop Item (Other)

Identification Number (DOI):

https://doi.org/10.1145/3411763.3451565