Studio report: sound synthesis with DDSP and network bending techniques

Yee-King, Matthew and McCallum, Louis. 2021. 'Studio report: sound synthesis with DDSP and network bending techniques'. In: 2nd Conference on AI Music Creativity (MuMe + CSMC). Graz, Austria 18 – 22 July 2021. [Conference or Workshop Item] (Forthcoming)

[img]
Preview
Text
myk_CSMC_MUME_2021.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial.

Download (486kB) | Preview
Official URL: https://aimc2021.iem.at/

Abstract or Description

This paper reports on our experiences synthesizing sounds and building network bending functionality onto the Differentiable Digital Signal Processing (DDSP) system. DDSP is an extension to the TensorFlow API with which we can embed trainable signal processing nodes in neural networks. Comparing DDSP sound synthesis networks to preset finding networks and sample level synthesis networks, we argue that it offers a third mode of working, providing continuous control in real-time of high fidelity synthesizers using low numbers of control parameters. We describe two phases of our experimentation. Firstly we worked with a composer to explore different training datasets and parameters. Secondly, we extended DDSP models with network bending functionality, which allows us to feed additional control data into the network's hidden layers and achieve new timbral effects. We describe several possible network bending techniques and how they affect the sound.

Item Type:

Conference or Workshop Item (Paper)

Keywords:

Sound synthesis, neural networks, network bending

Departments, Centres and Research Units:

Computing

Dates:

DateEvent
20 May 2021Accepted
18 July 2021Published

Event Location:

Graz, Austria

Date range:

18 – 22 July 2021

Item ID:

30217

Date Deposited:

23 Jun 2021 09:30

Last Modified:

20 Dec 2023 11:35

URI:

https://research.gold.ac.uk/id/eprint/30217

View statistics for this item...

Edit Record Edit Record (login required)