E-Thesis 357 views 194 downloads
Face Reenactment with Generative Landmark Guidance / CHEN HU
Swansea University Author: CHEN HU
Abstract
Face reenactment is a task aiming for transferring the expression and head pose from one face image to another. Recent studies mainly focus on estimating optical flows to warp input images’ feature maps to reenact expressions and head poses in synthesized images. However, the identity preserving pro...
| Published: |
Swansea
2023
|
|---|---|
| Institution: | Swansea University |
| Degree level: | Master of Research |
| Degree name: | MSc by Research |
| Supervisor: | Xie, Xianghua |
| URI: | https://cronfa.swan.ac.uk/Record/cronfa62597 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract: |
Face reenactment is a task aiming for transferring the expression and head pose from one face image to another. Recent studies mainly focus on estimating optical flows to warp input images’ feature maps to reenact expressions and head poses in synthesized images. However, the identity preserving problem is one of the major obstacles in these methods. The problem occurs when the model fails to preserve the detailed information of the source identity, namely the identity of the face we wish to synthesize, and especially obvious when reenacting different identities. The underlying factors may include unseen the leaking of driving identity. The driving identity stands for the identity of the face that provides the desired expression and head pose. When the source and the driving hold different identities, the model tends to mix the driving’s facial features with those of the source, resulting in inaccurate optical flow estimation and subsequently causing the identity of the synthesized face to deviate from the source.In this paper, we propose a novel face reenactment approach via generative land-mark coordinates. Specifically, a conditional generative adversarial network is devel-oped to estimate reenacted landmark coordinates for the driving image, which success-fully excludes its identity information. We then use generated coordinates to guide the alignment of individually reenacted facial landmarks. These coordinates are also injected into the style transferal module to increase the realism of face images. We evaluated our method on the VoxCeleb1 dataset for self-reenactment and the CelebV dataset for reenacting different identities. Extensive experiments demonstrate that our method can produce realistic reenacted face images by lowering the error in head pose and enhancing our models’ identity preserving capability.In addition to the conventional centralized learning, we deployed our model and used the CelebV dataset for federated learning in an aim to mitigate potential privacy issues involved in research on face images. We show that the proposed method is capable of showing competitive performance in the setting of federated learning. |
|---|---|
| Keywords: |
face reeanctment, optical flow, style transfer, federated learning, generative adversarial network |
| College: |
Faculty of Science and Engineering |