Broad, Terence; Leymarie, Frederic Fol and Grierson, Mick. 2021. 'Network Bending: Expressive Manipulation of Deep Generative Models'. In: 10th International Conference on Artificial Intelligence in Music, Sound, Art and Design (EvoMUSART 2021). Seville, Spain 7 – 9 April 2021. [Conference or Workshop Item]

No full text available

Text network_bending_expressive_manipulation_of_deep_generative_models.pdf - Accepted Version Permissions: Administrator Access Only Available under License Creative Commons Attribution Non-commercial. Download (19MB)

Abstract or Description

We introduce a new framework for manipulating and interacting with deep generative models that we call network bending. We present a comprehensive set of deterministic transformations that can be inserted as distinct layers into the computational graph of a trained generative neural network and applied during inference. In addition, we present a novel algorithm for analysing the deep generative model and clustering features based on their spatial activation maps. This allows features to be grouped together based on spatial similarity in an unsupervised fashion. This results in the meaningful manipulation of sets of features that correspond to the generation of a broad array of semantically significant features of the generated images. We outline this framework, demonstrating our results on state-of-the-art deep generative models trained on several image datasets. We show how it allows for the direct manipulation of semantically meaningful aspects of the generative process as well as allowing for a broad range of expressive outcomes.

Item Type:	Conference or Workshop Item (Paper)
Keywords:	neural networks, generative models, expressive generation
Departments, Centres and Research Units:	Computing
Dates:	Date Event 20 January 2021 Accepted 7 April 2021 Published
Event Location:	Seville, Spain
Date range:	7 – 9 April 2021
Item ID:	29829
Date Deposited:	19 Mar 2021 16:35
Last Modified:	11 Jun 2021 04:58
URI:	https://research.gold.ac.uk/id/eprint/29829

View statistics for this item...

Edit Record (login required)