Adversarial 3D Face Disentanglement of Identity and Expression

Yajie Gu; N. E. Pears; Hao Sun

Adversarial 3D Face Disentanglement of Identity and Expression

Yajie Gu, N. E. Pears, Hao Sun

Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We propose a new framework to decompose 3D facial shape into identity and expression.
Existing 3D face disentanglement methods assume the presence of a corresponding neutral (i.e. identity) face for each subject. Our method designs an identity discriminator to obviate this requirement. This is a binary classifier that determines if two input faces are from the same identity, and encourages the synthesised identity face to have the same identity features as the input face and to approach the `apathy' expression.
To this end, we take advantage of adversarial learning to train a PointNet-based variational auto-encoder and discriminator. Comprehensive experiments are employed on CoMA, BU3DFE, and FaceScape datasets. Results demonstrate state-of-the-art performance with the option of operating in a more versatile application setting of no known neutral ground truths. Code is available at \url{https://github.com/rmraaron/FaceExpDisentanglement}.

Original language	English
Title of host publication	International Conference on Automatic Face and Gesture Recognition 2023
Publisher	IEEE
Publication status	Accepted/In press - 11 Sep 2022
Event	International Conference on Automatic Face and Gesture Recognition 2023 - Waikoloa Beach Marriott Resort, Waikoloa, Hawaii, United States Duration: 5 Jan 2023 → 8 Jan 2023 https://fg2023.ieee-biometrics.org/

Conference

Conference	International Conference on Automatic Face and Gesture Recognition 2023
Abbreviated title	FG-2023
Country/Territory	United States
City	Waikoloa, Hawaii
Period	5/01/23 → 8/01/23
Internet address	https://fg2023.ieee-biometrics.org/

Bibliographical note

This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details

Access to Document

FG2023_disentanglement_ACCEPTED
This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details
Accepted author manuscript, 2.42 MB

Cite this

@inproceedings{258b79a60bf041b191f93fb87e0f736c,

title = "Adversarial 3D Face Disentanglement of Identity and Expression",

abstract = "We propose a new framework to decompose 3D facial shape into identity and expression. Existing 3D face disentanglement methods assume the presence of a corresponding neutral (i.e. identity) face for each subject. Our method designs an identity discriminator to obviate this requirement. This is a binary classifier that determines if two input faces are from the same identity, and encourages the synthesised identity face to have the same identity features as the input face and to approach the `apathy' expression. To this end, we take advantage of adversarial learning to train a PointNet-based variational auto-encoder and discriminator. Comprehensive experiments are employed on CoMA, BU3DFE, and FaceScape datasets. Results demonstrate state-of-the-art performance with the option of operating in a more versatile application setting of no known neutral ground truths. Code is available at \url{https://github.com/rmraaron/FaceExpDisentanglement}.",

author = "Yajie Gu and Pears, {N. E.} and Hao Sun",

note = "This is an author-produced version of the published paper. Uploaded in accordance with the publisher{\textquoteright}s self-archiving policy. Further copying may not be permitted; contact the publisher for details; International Conference on Automatic Face and Gesture Recognition 2023, FG-2023 ; Conference date: 05-01-2023 Through 08-01-2023",

year = "2022",

month = sep,

day = "11",

language = "English",

booktitle = "International Conference on Automatic Face and Gesture Recognition 2023",

publisher = "IEEE",

address = "United States",

url = "https://fg2023.ieee-biometrics.org/",

}

TY - GEN

T1 - Adversarial 3D Face Disentanglement of Identity and Expression

AU - Gu, Yajie

AU - Pears, N. E.

AU - Sun, Hao

N1 - This is an author-produced version of the published paper. Uploaded in accordance with the publisher’s self-archiving policy. Further copying may not be permitted; contact the publisher for details

PY - 2022/9/11

Y1 - 2022/9/11

N2 - We propose a new framework to decompose 3D facial shape into identity and expression. Existing 3D face disentanglement methods assume the presence of a corresponding neutral (i.e. identity) face for each subject. Our method designs an identity discriminator to obviate this requirement. This is a binary classifier that determines if two input faces are from the same identity, and encourages the synthesised identity face to have the same identity features as the input face and to approach the `apathy' expression. To this end, we take advantage of adversarial learning to train a PointNet-based variational auto-encoder and discriminator. Comprehensive experiments are employed on CoMA, BU3DFE, and FaceScape datasets. Results demonstrate state-of-the-art performance with the option of operating in a more versatile application setting of no known neutral ground truths. Code is available at \url{https://github.com/rmraaron/FaceExpDisentanglement}.

AB - We propose a new framework to decompose 3D facial shape into identity and expression. Existing 3D face disentanglement methods assume the presence of a corresponding neutral (i.e. identity) face for each subject. Our method designs an identity discriminator to obviate this requirement. This is a binary classifier that determines if two input faces are from the same identity, and encourages the synthesised identity face to have the same identity features as the input face and to approach the `apathy' expression. To this end, we take advantage of adversarial learning to train a PointNet-based variational auto-encoder and discriminator. Comprehensive experiments are employed on CoMA, BU3DFE, and FaceScape datasets. Results demonstrate state-of-the-art performance with the option of operating in a more versatile application setting of no known neutral ground truths. Code is available at \url{https://github.com/rmraaron/FaceExpDisentanglement}.

M3 - Conference contribution

BT - International Conference on Automatic Face and Gesture Recognition 2023

PB - IEEE

T2 - International Conference on Automatic Face and Gesture Recognition 2023

Y2 - 5 January 2023 through 8 January 2023

ER -