AVUI: Designing a Toolkit for Audiovisual Interfaces

Correia, Nuno N. and Tanaka, Atau. 2017. 'AVUI: Designing a Toolkit for Audiovisual Interfaces'. In: ACM CHI. Denver, CO, United States 5 - 11 June 2017. [Conference or Workshop Item]

AVUI-manuscript.pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial.

Download (5MB) | Preview
AVUI-CHI-preview3.mp4 - Supplemental Material
Available under License Creative Commons Attribution.

Download (13MB) | Preview
[img] Text
paper4846.pdf - Published Version
Permissions: Administrator Access Only

Download (4MB)
Official URL: https://chi2017.acm.org

Abstract or Description

The combined use of sound and image has a rich history, from audiovisual artworks to research exploring the potential of data visualization and sonification. However, we lack standard tools or guidelines for audiovisual (AV) interaction design, particularly for live performance. We propose the AVUI (AudioVisual User Interface), where sound and image are used together in a cohesive way in the interface; and an enabling technology, the ofxAVUI toolkit. AVUI guidelines and ofxAVUI were developed in a three-stage process, together with AV producers: 1) participatory design activities; 2) prototype development; 3) encapsulation of prototype as a plug-in, evaluation, and roll out. Best practices identified include: reconfigurable interfaces and mappings; object-oriented packaging of AV and UI; diverse sound visualization; flexible media manipulation and management. The toolkit and a mobile app developed using it have been released as open-source. Guidelines and toolkit demonstrate the potential of AVUI and offer designers a convenient framework for AV interaction design.

Item Type:

Conference or Workshop Item (Paper)

Identification Number (DOI):



H.5.2 User Interfaces: Auditory (non-speech) feedback; H.5.2 User Interfaces: Graphical user interfaces (GUI); H.5.2 User Interfaces: Prototyping; H.5.2 User Interfaces: User-centered design User interface; toolkit; interface builder; audiovisual; crossmodal interaction; prototyping; participatory design; interaction design; hackathons.

Related URLs:

Departments, Centres and Research Units:



12 January 2017Accepted
2 May 2017Published

Event Location:

Denver, CO, United States

Date range:

5 - 11 June 2017

Item ID:


Date Deposited:

24 Jan 2017 15:07

Last Modified:

16 Oct 2020 14:26



View statistics for this item...

Edit Record Edit Record (login required)