Research Online


Goldsmiths - University of London

Deep Canonical Time Warping

George, Trigeorgis; Nicolaou, Mihalis; Zafeiriou, Stefanos and Schuller, Bjorn. 2016. 'Deep Canonical Time Warping'. In: Proceedings of IEEE International Conference on Computer Vision & Pattern Recognition (CVPR'16). Las Vegas, United States. [Conference or Workshop Item] (Forthcoming)

0005[1].pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (3MB) | Preview

Abstract or Description

Machine learning algorithms for the analysis of timeseries often depend on the assumption that the utilised data are temporally aligned. Any temporal discrepancies arising in the data is certain to lead to ill-generalisable models, which in turn fail to correctly capture the properties of the task at hand. The temporal alignment of time-series is thus a crucial challenge manifesting in a multitude of applications. Nevertheless, the vast majority of algorithms oriented towards the temporal alignment of time-series are applied directly on the observation space, or utilise simple linear projections. Thus, they fail to capture complex, hierarchical non-linear representations which may prove to be beneficial towards the task of temporal alignment, particularly when dealing with multi-modal data (e.g., aligning visual and acoustic information). To this end, we present the Deep Canonical Time Warping (DCTW), a method which automatically learns complex non-linear representations of multiple time-series, generated such that (i) they are highly
correlated, and (ii) temporally in alignment. By means of
experiments on four real datasets, we show that the representations
learnt via the proposed DCTW significantly outperform
state-of-the-art methods in temporal alignment, elegantly
handling scenarios with highly heterogeneous features,
such as the temporal alignment of acoustic and visual

Item Type:

Conference or Workshop Item (Paper)

Identification Number (DOI):

Additional Information:

George Trigeorgis is a recipient of the fellowship of the
Department of Computing, Imperial College London, and
this work was partially funded by it. The work of Stefanos
Zafeiriou was partially funded by the EPSRC project
EP/J017787/1 (4D-FAB) and by the the FiDiPro program
of Tekes (project number: 1849/31/2015). The work of
Bjorn W. Schuller was partially funded by the European ¨
Community’s Horizon 2020 Framework Programme under
grant agreement No. 645378 (ARIA-VALUSPA). We
would like to thank the NVIDIA Corporation for donating
a Tesla K40 GPU used in this work.

Departments, Centres and Research Units:



June 2016Published
March 2016Accepted

Event Location:

Las Vegas, United States

Item ID:


Date Deposited:

08 Jun 2016 13:38

Last Modified:

12 Jul 2018 05:51


View statistics for this item...

Edit Record Edit Record (login required)