Broad, Terence; Leymarie, Frederic Fol and Grierson, Mick. 2022. Network Bending: Expressive Manipulation of Generative Models in Multiple Domains. Entropy, 24(1), 28. ISSN 1099-4300 [Article]

Preview

Text network-bending_entropy.pdf - Published Version Available under License Creative Commons Attribution. Download (33MB) | Preview

Abstract or Description

This paper presents the network bending framework, a new approach for manipulating and interacting with deep generative models. We present a comprehensive set of deterministic transformations that can be inserted as distinct layers into the computational graph of a trained generative neural network and applied during inference. In addition, we present a novel algorithm for analysing the deep generative model and clustering features based on their spatial activation maps. This allows features to be grouped together based on spatial similarity in an unsupervised fashion. This results in the meaningful manipulation of sets of features that correspond to the generation of a broad array of semantically significant features of the generated results. We outline this framework, demonstrating our results on deep generative models for both image and audio domains. We show how it allows for the direct manipulation of semantically meaningful aspects of the generative process as well as allowing for a broad range of expressive outcomes.

Item Type:	Article
Identification Number (DOI):	https://doi.org/10.3390/e24010028
Additional Information:	This work has been supported by UK’s EPSRC Centre for Doctoral Training in Intelligent Games and Game Intelligence (IGGI; grant EP/L015846/1). The generated datasets of activation maps have been made publicly available and can be found at: https://github.com/terrybroad/network-bending
Keywords:	deep generative models; expressive manipulation; active divergence
Departments, Centres and Research Units:	Computing
Dates:	Date Event 17 December 2021 Accepted 24 December 2021 Published Online 2022 Published
Item ID:	31378
Date Deposited:	07 Feb 2022 10:20
Last Modified:	07 Feb 2022 10:29
Peer Reviewed:	Yes, this version has been peer-reviewed.
URI:	https://research.gold.ac.uk/id/eprint/31378

View statistics for this item...

Edit Record (login required)