Van Zandt-Escobar, Alejandro; Caramiaux, Baptiste and Tanaka, Atau. 2014. 'PiaF: A Tool for Augmented Piano Performance Using Gesture Variation Following'. In: New Interfaces for Musical Expression. Goldsmiths, United Kingdom. [Conference or Workshop Item]

Preview

Text vanzandtescobar2014piaf.pdf - Published Version Available under License Creative Commons Attribution Non-commercial No Derivatives. Download (737kB) | Preview

Abstract or Description

When performing a piece, a pianist's interpretation is communicated both through the sound produced and through body gestures. We present PiaF (Piano Follower), a prototype for augmenting piano performance by measuring gesture variations. We survey other augmented piano projects, several of which focus on gestural recognition, and present our prototype which uses machine learning techniques for
gesture classification and estimation of gesture variations in
real-time. Our implementation uses the Kinect depth sensor to track body motion in space, which is used as input data. During an initial learning phase, the system is taught a set of reference gestures, or templates. During performance, the live gesture is classi�ed in real-time, and variations with respect to the recognized template are computed. These values can then be mapped to audio processing parameters, to control digital effects which are applied to the acoustic output of the piano in real-time. We discuss initial tests using PiaF with a pianist, as well as potential applications beyond live performance, including pedagogy and embodiment of recorded performance.

Item Type:	Conference or Workshop Item (Poster)
Keywords:	Augmented piano, gesture recognition, machine learning
Departments, Centres and Research Units:	Computing > Embodied AudioVisual Interaction Group (EAVI)
Dates:	Date Event June 2014 Published
Event Location:	Goldsmiths, United Kingdom
Item ID:	11202
Date Deposited:	23 Jan 2015 12:21
Last Modified:	29 Apr 2020 16:05
URI:	https://research.gold.ac.uk/id/eprint/11202

View statistics for this item...

Edit Record (login required)