Tanaka, Atau; Di Donato, Balandino; Zbyszynski, Michael and Roks, Geert. 2019. 'Designing Gestures for Continuous Sonic Interaction'. In: The International Conference on New Interfaces for Musical Expression. Porto Alegre, Brazil 3-6 June 2019. [Conference or Workshop Item]

Preview

Text nime_19 (3).pdf - Published Version Available under License Creative Commons Attribution. Download (6MB) | Preview

Abstract or Description

We present a system that allows users to try different ways to train neural networks and temporal modelling to asso- ciate gestures with time-varying sound. We created a soft- ware framework for this and evaluated it in a workshop- based study. We build upon research in sound tracing and mapping-by-demonstration to ask participants to de- sign gestures for performing time-varying sounds using a multimodal, inertial measurement (IMU) and muscle sens- ing (EMG) device. We presented the user with two classical techniques from the literature, Static Position regression and Hidden Markov based temporal modelling, and pro- pose a new technique for capturing gesture anchor points on the fly as training data for neural network based regression, called Windowed Regression. Our results show trade- offs between accurate, predictable reproduction of source sounds and exploration of the gesture-sound space. Several users were attracted to our windowed regression technique. This paper will be of interest to musicians engaged in going from sound design to gesture design and offers a workflow for interactive machine learning.

Item Type:	Conference or Workshop Item (Paper)
Keywords:	Sonic Interaction Design, Interactive Machine Learning, Gestural Interaction
Related URLs:	Organisation
Departments, Centres and Research Units:	Computing > Embodied AudioVisual Interaction Group (EAVI)
Dates:	Date Event 1 March 2019 Submitted 20 March 2019 Accepted 3 June 2019 Published
Event Location:	Porto Alegre, Brazil
Date range:	3-6 June 2019
Item ID:	26475
Date Deposited:	19 Jun 2019 13:07
Last Modified:	13 Jun 2021 11:13
URI:	https://research.gold.ac.uk/id/eprint/26475

View statistics for this item...

Edit Record (login required)