Mnemonic Descent Method: A recurrent process applied for end-to-end face alignment

Trigeorgis, G.; Snape, P.; Nicolaou, M. A.; Antonakos, E. and Zafeiriou, S.. 2016. 'Mnemonic Descent Method: A recurrent process applied for end-to-end face alignment'. In: Proceedings of IEEE International Conference on Computer Vision & Pattern Recognition (CVPR'16). Las Vegas, United States. [Conference or Workshop Item]

trigeorgis2016mnemonic[1].pdf - Accepted Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (5MB) | Preview

Abstract or Description

Cascaded regression has recently become the method of choice for solving non-linear least squares problems such as deformable image alignment. Given a sizeable training set, cascaded regression learns a set of generic rules that are sequentially applied to minimise the least squares problem. Despite the success of cascaded regression for problems such as face alignment and head pose estimation, there are several shortcomings arising in the strategies proposed thus far. Specifically, (a) the regressors are learnt independently, (b) the descent directions may cancel one another out and (c) handcrafted features (e.g., HoGs, SIFT etc.) are mainly used to drive the cascade, which may be sub-optimal for the task at hand. In this paper, we propose a combined and jointly trained convolutional recurrent neural network architecture that allows the training of an end-to-end to system that attempts to alleviate the aforementioned drawbacks. The recurrent module facilitates the joint optimisation of the regressors by assuming the cascades form a nonlinear dynamical system, in effect fully utilising the information between all cascade levels by introducing a memory unit that shares information across all levels. The convolutional module allows the network to extract features that are specialised for the task at hand and are experimentally shown to outperform hand-crafted features. We show that the application of the proposed architecture for the problem of face alignment results in a strong improvement over the current state-of-the-art.

Item Type:

Conference or Workshop Item (Poster)

Departments, Centres and Research Units:



June 2016Published
March 2016Accepted

Event Location:

Las Vegas, United States

Item ID:


Date Deposited:

08 Jun 2016 13:44

Last Modified:

29 Apr 2020 16:18


View statistics for this item...

Edit Record Edit Record (login required)