Abstract
Learning a disentangled representation is essential to build 3D face models that accurately capture identity and expression. We propose a novel variational autoencoder (VAE) framework to disentangle identity and expression from 3D input faces that have a wide variety of expressions. Specifically, we design a system that has two decoders: one for neutral-expression faces (i.e. identity-only faces) and one for the original (expressive) input faces respectively. Crucially, we have an additional mutual-information regulariser applied on the identity part to solve the issue of imbalanced information over the expressive input faces and the reconstructed neutral faces. Our evaluations on two public datasets (CoMA and BU-3DFE) show that this model achieves competitive results on the 3D face reconstruction task and state-of-the-art results on identity-expression disentanglement. We also show that by updating to a conditional VAE, we have a system that generates different levels of expressions from semantically meaningful variables.
Original language | English |
---|---|
Title of host publication | Winter Conference on Applications in Computer Vision, Proceedings |
Publisher | IEEE |
Publication status | Accepted/In press - 4 Oct 2021 |
Event | Winter Conference on Applications in Computer Vision - Waikoloa, Waikoloa, United States Duration: 4 Jan 2022 → 8 Jan 2022 https://wacv2022.thecvf.com/home |
Conference
Conference | Winter Conference on Applications in Computer Vision |
---|---|
Abbreviated title | WACV |
Country/Territory | United States |
City | Waikoloa |
Period | 4/01/22 → 8/01/22 |
Internet address |
Bibliographical note
This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for detailsKeywords
- 3D Face Modelling, 3D facial expression modelling, 3D face disentanglement