The Role of Sound Source Perception in Gestural Sound Description

Caramiaux, Baptiste; Bevilacqua, Frédéric; Bianco, Tommaso; Schnell, Norbert; Houix, Olivier and Susini, Patrick. 2014. The Role of Sound Source Perception in Gestural Sound Description. ACM Transactions on Applied Perception, 11(1), pp. 1-19. ISSN 1544-3558 [Article]

caramiaux2014role.pdf - Accepted Version

Download (1MB) | Preview

Abstract or Description

We investigated gesture description of sound stimuli performed during a listening task. Our hypothesis is that the strategies in gestural responses depend on the level of identification of the sound source, and specifically on the identification of the action causing the sound. To validate our hypothesis, we conducted two experiments. In the first experiment, we built two corpora of sounds. The first corpus contains sounds with identifiable causal actions. The second contains sounds where no causal actions could be identified. These corpora properties were validated through a listening test. In the second experiment, participants performed arm and hand gestures synchronously while listening to sounds taken from these corpora. Afterwards, we conducted interviews asking participants to verbalize their experience, watching their own video recordings. They were questioned on their perception of the listened sounds and on their gestural strategies. We showed that for the sounds where causal action can be identified, participants mainly mimic the action that has produced the sound. In the other case, when no action can be associated to the sound, participants trace contours related to sound acoustic features. We also found that the inter-participants gesture variability is higher for causal sounds compared to non-causal sounds. Variability demonstrates that in the first case, participants have several ways of producing the same action whereas in the second case, the sound features tend to make the gesture responses consistent.

Item Type:


Identification Number (DOI):

Additional Information:

© ACM, 2014. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in ACM Transactions on Applied Perception, {VOL11, ISS1, (2014)}

Departments, Centres and Research Units:

Computing > Embodied AudioVisual Interaction Group (EAVI)


April 2014Published

Item ID:


Date Deposited:

21 Jan 2015 21:14

Last Modified:

29 Apr 2020 16:05

Peer Reviewed:

Yes, this version has been peer-reviewed.


View statistics for this item...

Edit Record Edit Record (login required)