Reflections on Eight Years of Instrument Creation with Machine Learning

Fiebrink, Rebecca and Sonami, Laetitia. 2020. 'Reflections on Eight Years of Instrument Creation with Machine Learning'. In: International Conference on New Interfaces for Musical Expression (NIME). Birmingham, United Kingdom 21-25 July 2020. [Conference or Workshop Item]

FiebrinkSonami_NIME2020.pdf - Published Version
Available under License Creative Commons Attribution.

Download (1MB) | Preview

Abstract or Description

Machine learning (ML) has been used to create mappings for digital musical instruments for over twenty-five years, and numerous ML toolkits have been developed for the NIME community. However, little published work has studied how ML has been used in sustained instrument building and performance practices. This paper examines the experiences of instrument builder and performer Laetitia Sonami, who has been using ML to build and refine her Spring Spyre instrument since 2012. Using Sonami’s current practice as a case study, this paper explores the utility, opportunities, and challenges involved in using ML in practice over many years. This paper also reports the perspective of Rebecca Fiebrink, the creator of the Wekinator ML tool used by Sonami, revealing how her work with Sonami has led to changes to the software and to her teaching. This paper thus contributes a deeper understanding of the value of ML for NIME practitioners, and it can inform design considerations for future ML toolkits as well as NIME pedagogy. Further, it provides new perspectives on familiar NIME conversations about mapping strategies, expressivity, and control, informed by a dedicated practice over many years.

Item Type:

Conference or Workshop Item (Paper)

Departments, Centres and Research Units:

Computing > Embodied AudioVisual Interaction Group (EAVI)


30 March 2020Accepted
25 July 2020Published

Event Location:

Birmingham, United Kingdom

Date range:

21-25 July 2020

Item ID:


Date Deposited:

01 May 2020 14:01

Last Modified:

30 Mar 2021 15:38


View statistics for this item...

Edit Record Edit Record (login required)