Deep Canonical Time Warping

Tools

George, Trigeorgis; Nicolaou, Mihalis; Zafeiriou, Stefanos and Schuller, Bjorn. 2016. 'Deep Canonical Time Warping'. In: Proceedings of IEEE International Conference on Computer Vision & Pattern Recognition (CVPR'16). Las Vegas, United States 27-30 June 2016. [Conference or Workshop Item]

Preview

Text
0005[1].pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.
Download (3MB) | Preview

Abstract or Description

Machine learning algorithms for the analysis of timeseries often depend on the assumption that the utilised data are temporally aligned. Any temporal discrepancies arising in the data is certain to lead to ill-generalisable models, which in turn fail to correctly capture the properties of the task at hand. The temporal alignment of time-series is thus a crucial challenge manifesting in a multitude of applications. Nevertheless, the vast majority of algorithms oriented towards the temporal alignment of time-series are applied directly on the observation space, or utilise simple linear projections. Thus, they fail to capture complex, hierarchical non-linear representations which may prove to be beneficial towards the task of temporal alignment, particularly when dealing with multi-modal data (e.g., aligning visual and acoustic information). To this end, we present the Deep Canonical Time Warping (DCTW), a method which automatically learns complex non-linear representations of multiple time-series, generated such that (i) they are highly correlated, and (ii) temporally in alignment. By means of experiments on four real datasets, we show that the representations learnt via the proposed DCTW significantly outperform state-of-the-art methods in temporal alignment, elegantly handling scenarios with highly heterogeneous features, such as the temporal alignment of acoustic and visual features.

Item Type:

Conference or Workshop Item (Paper)

Identification Number (DOI):

https://doi.org/10.1109/CVPR.2016.552

Additional Information:

George Trigeorgis is a recipient of the fellowship of the Department of Computing, Imperial College London, and this work was partially funded by it. The work of Stefanos Zafeiriou was partially funded by the EPSRC project EP/J017787/1 (4D-FAB) and by the the FiDiPro program of Tekes (project number: 1849/31/2015). The work of Bjorn W. Schuller was partially funded by the European Community’s Horizon 2020 Framework Programme under grant agreement No. 645378 (ARIA-VALUSPA). We would like to thank the NVIDIA Corporation for donating a Tesla K40 GPU used in this work.

Keywords:

time warping, cca, lda, dcca, dda, deep learning, shared representations, dctw

Departments, Centres and Research Units:

Computing

Dates: