Yee-King, Matthew and McCallum, Louis. 2021. 'Studio report: sound synthesis with DDSP and network bending techniques'. In: 2nd Conference on AI Music Creativity (MuMe + CSMC). Graz, Austria 18 – 22 July 2021. [Conference or Workshop Item] (Forthcoming)

Preview

Text myk_CSMC_MUME_2021.pdf - Accepted Version Available under License Creative Commons Attribution Non-commercial. Download (486kB) | Preview

Abstract or Description

This paper reports on our experiences synthesizing sounds and building network bending functionality onto the Differentiable Digital Signal Processing (DDSP) system. DDSP is an extension to the TensorFlow API with which we can embed trainable signal processing nodes in neural networks. Comparing DDSP sound synthesis networks to preset finding networks and sample level synthesis networks, we argue that it offers a third mode of working, providing continuous control in real-time of high fidelity synthesizers using low numbers of control parameters. We describe two phases of our experimentation. Firstly we worked with a composer to explore different training datasets and parameters. Secondly, we extended DDSP models with network bending functionality, which allows us to feed additional control data into the network's hidden layers and achieve new timbral effects. We describe several possible network bending techniques and how they affect the sound.

Item Type:	Conference or Workshop Item (Paper)
Keywords:	Sound synthesis, neural networks, network bending
Departments, Centres and Research Units:	Computing
Dates:	Date Event 20 May 2021 Accepted 18 July 2021 Published
Event Location:	Graz, Austria
Date range:	18 – 22 July 2021
Item ID:	30217
Date Deposited:	23 Jun 2021 09:30
Last Modified:	20 Dec 2023 11:35
URI:	https://research.gold.ac.uk/id/eprint/30217

View statistics for this item...

Edit Record (login required)