Innermost Echoes: Integrating Real-Time Physiology into Live Music Performances

Hynds, Danny; Chernyshov, George; Zheng, Dingding; Uyama, Aoi; Li, Juling; Matsumoto, Kozue; Pogorzhelskiy, Michael; Kunze, Kai; Ward, Jamie A and Minamizawa, Kouta. 2024. 'Innermost Echoes: Integrating Real-Time Physiology into Live Music Performances'. In: Eighteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI ’24). Cork, Ireland 11–14 February 2024. [Conference or Workshop Item]

Innermost_Echoes_2023_TEI.pdf - Accepted Version

Download (9MB) | Preview

Abstract or Description

In this paper, we propose a method for utilizing musical artifacts and physiological data as a means for creating a new form of live music experience that is rooted in the physiology of the perform- ers and audience members. By utilizing physiological data (namely Electrodermal Activity (EDA) and Heart Rate Variability (HRV)) and applying this data to musical artifacts including a robotic koto (a traditional 13-string Japanese instrument fitted with solenoids and linear actuators), a Eurorack synthesizer, and Max/MSP software, we aim to develop a new form of semi-improvisational and signif- icantly indeterminate performance practice. It has since evolved into a multi-modal methodology which honors improvisational performance practices and utilizes physiological data which of- fers both performers and audiences an ever-changing and intimate experience.

In our first exploratory phase, we focused on the development of a means for controlling a bespoke robotic koto in conjunction with a Eurorack synthesizer system and Max/MSP software for controlling the incoming data. We integrated a reliance on physiological data to infuse a more directly human elements into this artifact system. This allows a significant portion of the decision-making to be directly controlled by the incoming physiological data in real-time, thereby affording a sense of performativity within this non-living system. Our aim is to continue the development of this method to strike a novel balance between intentionality and impromptu performative results.

Item Type:

Conference or Workshop Item (Paper)

Identification Number (DOI):

Additional Information:

"© 2024 Copyright held by the owner/author(s). This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record is available at,"

This project was supported by the JST Moonshot R&D Program “Cybernetic Being” Project (Grant number JPMJMS2013).


liveness, sonic art, physiological sensing, music composition, public art, improvisation

Departments, Centres and Research Units:



17 December 2023Accepted
11 February 2024Published

Event Location:

Cork, Ireland

Date range:

11–14 February 2024

Item ID:


Date Deposited:

02 Feb 2024 11:08

Last Modified:

03 Feb 2024 01:32


View statistics for this item...

Edit Record Edit Record (login required)