Adieu Features? End-to-End Speech Emotion Recognition using a Deep Convolutional Recurrent Network

Trigeorgis, George; Ringeval, Fabien; Brueckner, Raymond; Marchi, Erik; Nicolaou, Mihalis; Schuller, Björn and Zafeiriou, Stefanos. 2016. Adieu Features? End-to-End Speech Emotion Recognition using a Deep Convolutional Recurrent Network. Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2016, pp. 5200-5204. ISSN 2379-190X [Article]

learning_audio_paralinguistics_from_the_raw_waveform.pdf - Accepted Version

Download (683kB) | Preview

Abstract or Description

The automatic recognition of spontaneous emotions from speech is a challenging task. On the one hand, acoustic features need to be robust enough to capture the emotional content for various styles of speaking, and while on the other, machine learning algorithms need to be insensitive to outliers while being able to model the context. Whereas the latter has been tackled by the use of Long Short-Term Memory (LSTM) networks, the former is still under very active investigations, even though more than a decade of research has provided a large set of acoustic descriptors. In this paper, we propose a solution to the problem of `context-aware' emotional relevant feature extraction, by combining Convolutional Neural Networks (CNNs) with LSTM networks, in order to automatically learn the best representation of the speech signal directly from the raw time representation. In this novel work on the so-called end-to-end speech emotion recognition, we show that the use of the proposed topology significantly outperforms the traditional approaches based on signal processing techniques for the prediction of spontaneous and natural emotions on the RECOLA database.

Item Type:


Identification Number (DOI):


LSTM, end-to-end learning, raw waveform, emotion recognition, deep learning, CNN

Departments, Centres and Research Units:



25 March 2016Accepted
19 May 2016Published Online

Item ID:


Date Deposited:

22 Mar 2016 08:39

Last Modified:

24 Mar 2021 17:59

Peer Reviewed:

Yes, this version has been peer-reviewed.


View statistics for this item...

Edit Record Edit Record (login required)