No Cover Image

E-Thesis 357 views 194 downloads

Face Reenactment with Generative Landmark Guidance / CHEN HU

Swansea University Author: CHEN HU

Abstract

Face reenactment is a task aiming for transferring the expression and head pose from one face image to another. Recent studies mainly focus on estimating optical flows to warp input images’ feature maps to reenact expressions and head poses in synthesized images. However, the identity preserving pro...

Full description

Published: Swansea 2023
Institution: Swansea University
Degree level: Master of Research
Degree name: MSc by Research
Supervisor: Xie, Xianghua
URI: https://cronfa.swan.ac.uk/Record/cronfa62597
Tags: Add Tag
No Tags, Be the first to tag this record!
first_indexed 2023-02-07T11:03:49Z
last_indexed 2023-02-08T04:17:09Z
id cronfa62597
recordtype RisThesis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2023-02-07T11:12:10.5284680</datestamp><bib-version>v2</bib-version><id>62597</id><entry>2023-02-07</entry><title>Face Reenactment with Generative Landmark Guidance</title><swanseaauthors><author><sid>f166c1a306bc35bfaef8c8a4189752d0</sid><firstname>CHEN</firstname><surname>HU</surname><name>CHEN HU</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2023-02-07</date><abstract>Face reenactment is a task aiming for transferring the expression and head pose from one face image to another. Recent studies mainly focus on estimating optical flows to warp input images&#x2019; feature maps to reenact expressions and head poses in synthesized images. However, the identity preserving problem is one of the major obstacles in these methods. The problem occurs when the model fails to preserve the detailed information of the source identity, namely the identity of the face we wish to synthesize, and especially obvious when reenacting different identities. The underlying factors may include unseen the leaking of driving identity. The driving identity stands for the identity of the face that provides the desired expression and head pose. When the source and the driving hold different identities, the model tends to mix the driving&#x2019;s facial features with those of the source, resulting in inaccurate optical flow estimation and subsequently causing the identity of the synthesized face to deviate from the source.In this paper, we propose a novel face reenactment approach via generative land-mark coordinates. Specifically, a conditional generative adversarial network is devel-oped to estimate reenacted landmark coordinates for the driving image, which success-fully excludes its identity information. We then use generated coordinates to guide the alignment of individually reenacted facial landmarks. These coordinates are also injected into the style transferal module to increase the realism of face images. We evaluated our method on the VoxCeleb1 dataset for self-reenactment and the CelebV dataset for reenacting different identities. Extensive experiments demonstrate that our method can produce realistic reenacted face images by lowering the error in head pose and enhancing our models&#x2019; identity preserving capability.In addition to the conventional centralized learning, we deployed our model and used the CelebV dataset for federated learning in an aim to mitigate potential privacy issues involved in research on face images. We show that the proposed method is capable of showing competitive performance in the setting of federated learning.</abstract><type>E-Thesis</type><journal/><volume/><journalNumber/><paginationStart/><paginationEnd/><publisher/><placeOfPublication>Swansea</placeOfPublication><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic/><keywords>face reeanctment, optical flow, style transfer, federated learning, generative adversarial network</keywords><publishedDay>1</publishedDay><publishedMonth>2</publishedMonth><publishedYear>2023</publishedYear><publishedDate>2023-02-01</publishedDate><doi/><url/><notes/><college>COLLEGE NANME</college><CollegeCode>COLLEGE CODE</CollegeCode><institution>Swansea University</institution><supervisor>Xie, Xianghua</supervisor><degreelevel>Master of Research</degreelevel><degreename>MSc by Research</degreename><apcterm/><funders/><projectreference/><lastEdited>2023-02-07T11:12:10.5284680</lastEdited><Created>2023-02-07T11:01:16.8969064</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>CHEN</firstname><surname>HU</surname><order>1</order></author></authors><documents><document><filename>62597__26501__b4c07b6fbca941178c7c94df3c8f1d0a.pdf</filename><originalFilename>Hu_Chen_MSc_Research_Thesis_Final_Redacted_Signature.pdf</originalFilename><uploaded>2023-02-07T11:07:08.6781923</uploaded><type>Output</type><contentLength>6740201</contentLength><contentType>application/pdf</contentType><version>E-Thesis &#x2013; open access</version><cronfaStatus>true</cronfaStatus><documentNotes>Copyright: The author, Chen Hu, 2023.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language></document></documents><OutputDurs/></rfc1807>
spelling 2023-02-07T11:12:10.5284680 v2 62597 2023-02-07 Face Reenactment with Generative Landmark Guidance f166c1a306bc35bfaef8c8a4189752d0 CHEN HU CHEN HU true false 2023-02-07 Face reenactment is a task aiming for transferring the expression and head pose from one face image to another. Recent studies mainly focus on estimating optical flows to warp input images’ feature maps to reenact expressions and head poses in synthesized images. However, the identity preserving problem is one of the major obstacles in these methods. The problem occurs when the model fails to preserve the detailed information of the source identity, namely the identity of the face we wish to synthesize, and especially obvious when reenacting different identities. The underlying factors may include unseen the leaking of driving identity. The driving identity stands for the identity of the face that provides the desired expression and head pose. When the source and the driving hold different identities, the model tends to mix the driving’s facial features with those of the source, resulting in inaccurate optical flow estimation and subsequently causing the identity of the synthesized face to deviate from the source.In this paper, we propose a novel face reenactment approach via generative land-mark coordinates. Specifically, a conditional generative adversarial network is devel-oped to estimate reenacted landmark coordinates for the driving image, which success-fully excludes its identity information. We then use generated coordinates to guide the alignment of individually reenacted facial landmarks. These coordinates are also injected into the style transferal module to increase the realism of face images. We evaluated our method on the VoxCeleb1 dataset for self-reenactment and the CelebV dataset for reenacting different identities. Extensive experiments demonstrate that our method can produce realistic reenacted face images by lowering the error in head pose and enhancing our models’ identity preserving capability.In addition to the conventional centralized learning, we deployed our model and used the CelebV dataset for federated learning in an aim to mitigate potential privacy issues involved in research on face images. We show that the proposed method is capable of showing competitive performance in the setting of federated learning. E-Thesis Swansea face reeanctment, optical flow, style transfer, federated learning, generative adversarial network 1 2 2023 2023-02-01 COLLEGE NANME COLLEGE CODE Swansea University Xie, Xianghua Master of Research MSc by Research 2023-02-07T11:12:10.5284680 2023-02-07T11:01:16.8969064 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science CHEN HU 1 62597__26501__b4c07b6fbca941178c7c94df3c8f1d0a.pdf Hu_Chen_MSc_Research_Thesis_Final_Redacted_Signature.pdf 2023-02-07T11:07:08.6781923 Output 6740201 application/pdf E-Thesis – open access true Copyright: The author, Chen Hu, 2023. true eng
title Face Reenactment with Generative Landmark Guidance
spellingShingle Face Reenactment with Generative Landmark Guidance
CHEN HU
title_short Face Reenactment with Generative Landmark Guidance
title_full Face Reenactment with Generative Landmark Guidance
title_fullStr Face Reenactment with Generative Landmark Guidance
title_full_unstemmed Face Reenactment with Generative Landmark Guidance
title_sort Face Reenactment with Generative Landmark Guidance
author_id_str_mv f166c1a306bc35bfaef8c8a4189752d0
author_id_fullname_str_mv f166c1a306bc35bfaef8c8a4189752d0_***_CHEN HU
author CHEN HU
author2 CHEN HU
format E-Thesis
publishDate 2023
institution Swansea University
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 1
active_str 0
description Face reenactment is a task aiming for transferring the expression and head pose from one face image to another. Recent studies mainly focus on estimating optical flows to warp input images’ feature maps to reenact expressions and head poses in synthesized images. However, the identity preserving problem is one of the major obstacles in these methods. The problem occurs when the model fails to preserve the detailed information of the source identity, namely the identity of the face we wish to synthesize, and especially obvious when reenacting different identities. The underlying factors may include unseen the leaking of driving identity. The driving identity stands for the identity of the face that provides the desired expression and head pose. When the source and the driving hold different identities, the model tends to mix the driving’s facial features with those of the source, resulting in inaccurate optical flow estimation and subsequently causing the identity of the synthesized face to deviate from the source.In this paper, we propose a novel face reenactment approach via generative land-mark coordinates. Specifically, a conditional generative adversarial network is devel-oped to estimate reenacted landmark coordinates for the driving image, which success-fully excludes its identity information. We then use generated coordinates to guide the alignment of individually reenacted facial landmarks. These coordinates are also injected into the style transferal module to increase the realism of face images. We evaluated our method on the VoxCeleb1 dataset for self-reenactment and the CelebV dataset for reenacting different identities. Extensive experiments demonstrate that our method can produce realistic reenacted face images by lowering the error in head pose and enhancing our models’ identity preserving capability.In addition to the conventional centralized learning, we deployed our model and used the CelebV dataset for federated learning in an aim to mitigate potential privacy issues involved in research on face images. We show that the proposed method is capable of showing competitive performance in the setting of federated learning.
published_date 2023-02-01T04:22:20Z
_version_ 1763754471939112960
score 11.012924

Similar Items