No Cover Image

Conference Paper/Proceeding/Abstract 230 views 22 downloads

Depth-Aware Endoscopic Video Inpainting

Francis Xiatian Zhang Orcid Logo, Shuang Chen Orcid Logo, Xianghua Xie Orcid Logo, Hubert P. H. Shum Orcid Logo

Lecture Notes in Computer Science, Volume: 15006, Pages: 143 - 153

Swansea University Author: Xianghua Xie Orcid Logo

  • 66924.pdf

    PDF | Accepted Manuscript

    Author accepted manuscript document released under the terms of a Creative Commons CC-BY licence using the Swansea University Research Publications Policy (rights retention).

    Download (10.96MB)

Check full text

DOI (Published version): 10.1007/978-3-031-72089-5_14

Abstract

Video inpainting fills in corrupted video content with plausible replacements. While recent advances in endoscopic video inpainting have shown potential for enhancing the quality of endoscopic videos,they mainly repair 2D visual information without effectively preserving crucial 3D spatial details f...

Full description

Published in: Lecture Notes in Computer Science
ISBN: 9783031720888 9783031720895
ISSN: 0302-9743 1611-3349
Published: Cham Springer Nature Switzerland 2024
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa66924
first_indexed 2024-07-02T12:58:50Z
last_indexed 2024-11-26T19:44:46Z
id cronfa66924
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2024-11-26T15:05:23.9228464</datestamp><bib-version>v2</bib-version><id>66924</id><entry>2024-07-02</entry><title>Depth-Aware Endoscopic Video Inpainting</title><swanseaauthors><author><sid>b334d40963c7a2f435f06d2c26c74e11</sid><ORCID>0000-0002-2701-8660</ORCID><firstname>Xianghua</firstname><surname>Xie</surname><name>Xianghua Xie</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2024-07-02</date><deptcode>MACS</deptcode><abstract>Video inpainting fills in corrupted video content with plausible replacements. While recent advances in endoscopic video inpainting have shown potential for enhancing the quality of endoscopic videos,they mainly repair 2D visual information without effectively preserving crucial 3D spatial details for clinical reference. Depth-aware inpainting methods attempt to preserve these details by incorporating depth information. Still, in endoscopic contexts, they face challenges including reliance on pre-acquired depth maps, less effective fusion designs, and ignorance of the fidelity of 3D spatial details. To address them, we introduce a novel Depth-aware Endoscopic Video Inpainting (DAEVI) framework. It features a Spatial-Temporal Guided Depth Estimation module for direct depth estimation from visual features, a Bi-Modal Paired Channel Fusion module for effective channel-by-channel fusion of visual and depth information, and a Depth Enhanced Discriminator to assess the fidelity of the RGB-D sequence comprised of the inpainted frames and estimated depth images. Experimental evaluations on established benchmarks demonstrate our framework&#x2019;s superiority, achieving a 2% improvementin PSNR and a 6% reduction in MSE compared to state-of-the-art methods. Qualitative analyses further validate its enhanced ability to inpaint fine details, highlighting the benefits of integrating depth information into endoscopic inpainting.</abstract><type>Conference Paper/Proceeding/Abstract</type><journal>Lecture Notes in Computer Science</journal><volume>15006</volume><journalNumber/><paginationStart>143</paginationStart><paginationEnd>153</paginationEnd><publisher>Springer Nature Switzerland</publisher><placeOfPublication>Cham</placeOfPublication><isbnPrint>9783031720888</isbnPrint><isbnElectronic>9783031720895</isbnElectronic><issnPrint>0302-9743</issnPrint><issnElectronic>1611-3349</issnElectronic><keywords/><publishedDay>3</publishedDay><publishedMonth>10</publishedMonth><publishedYear>2024</publishedYear><publishedDate>2024-10-03</publishedDate><doi>10.1007/978-3-031-72089-5_14</doi><url>http://dx.doi.org/10.1007/978-3-031-72089-5_14</url><notes/><college>COLLEGE NANME</college><department>Mathematics and Computer Science School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MACS</DepartmentCode><institution>Swansea University</institution><apcterm/><funders>This research is supported in part by the EPSRC NortHFutures project (ref: EP/X031012/1).</funders><projectreference/><lastEdited>2024-11-26T15:05:23.9228464</lastEdited><Created>2024-07-02T13:54:54.0019893</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Francis Xiatian</firstname><surname>Zhang</surname><orcid>0000-0003-0228-6359</orcid><order>1</order></author><author><firstname>Shuang</firstname><surname>Chen</surname><orcid>0000-0002-6879-7285</orcid><order>2</order></author><author><firstname>Xianghua</firstname><surname>Xie</surname><orcid>0000-0002-2701-8660</orcid><order>3</order></author><author><firstname>Hubert P. H.</firstname><surname>Shum</surname><orcid>0000-0001-5651-6039</orcid><order>4</order></author></authors><documents><document><filename>66924__30795__cf423692cd264ac9ac0deb5d523c9e93.pdf</filename><originalFilename>66924.pdf</originalFilename><uploaded>2024-07-02T13:58:48.1010254</uploaded><type>Output</type><contentLength>11490256</contentLength><contentType>application/pdf</contentType><version>Accepted Manuscript</version><cronfaStatus>true</cronfaStatus><documentNotes>Author accepted manuscript document released under the terms of a Creative Commons CC-BY licence using the Swansea University Research Publications Policy (rights retention).</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/deed.en</licence></document></documents><OutputDurs/></rfc1807>
spelling 2024-11-26T15:05:23.9228464 v2 66924 2024-07-02 Depth-Aware Endoscopic Video Inpainting b334d40963c7a2f435f06d2c26c74e11 0000-0002-2701-8660 Xianghua Xie Xianghua Xie true false 2024-07-02 MACS Video inpainting fills in corrupted video content with plausible replacements. While recent advances in endoscopic video inpainting have shown potential for enhancing the quality of endoscopic videos,they mainly repair 2D visual information without effectively preserving crucial 3D spatial details for clinical reference. Depth-aware inpainting methods attempt to preserve these details by incorporating depth information. Still, in endoscopic contexts, they face challenges including reliance on pre-acquired depth maps, less effective fusion designs, and ignorance of the fidelity of 3D spatial details. To address them, we introduce a novel Depth-aware Endoscopic Video Inpainting (DAEVI) framework. It features a Spatial-Temporal Guided Depth Estimation module for direct depth estimation from visual features, a Bi-Modal Paired Channel Fusion module for effective channel-by-channel fusion of visual and depth information, and a Depth Enhanced Discriminator to assess the fidelity of the RGB-D sequence comprised of the inpainted frames and estimated depth images. Experimental evaluations on established benchmarks demonstrate our framework’s superiority, achieving a 2% improvementin PSNR and a 6% reduction in MSE compared to state-of-the-art methods. Qualitative analyses further validate its enhanced ability to inpaint fine details, highlighting the benefits of integrating depth information into endoscopic inpainting. Conference Paper/Proceeding/Abstract Lecture Notes in Computer Science 15006 143 153 Springer Nature Switzerland Cham 9783031720888 9783031720895 0302-9743 1611-3349 3 10 2024 2024-10-03 10.1007/978-3-031-72089-5_14 http://dx.doi.org/10.1007/978-3-031-72089-5_14 COLLEGE NANME Mathematics and Computer Science School COLLEGE CODE MACS Swansea University This research is supported in part by the EPSRC NortHFutures project (ref: EP/X031012/1). 2024-11-26T15:05:23.9228464 2024-07-02T13:54:54.0019893 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Francis Xiatian Zhang 0000-0003-0228-6359 1 Shuang Chen 0000-0002-6879-7285 2 Xianghua Xie 0000-0002-2701-8660 3 Hubert P. H. Shum 0000-0001-5651-6039 4 66924__30795__cf423692cd264ac9ac0deb5d523c9e93.pdf 66924.pdf 2024-07-02T13:58:48.1010254 Output 11490256 application/pdf Accepted Manuscript true Author accepted manuscript document released under the terms of a Creative Commons CC-BY licence using the Swansea University Research Publications Policy (rights retention). true eng https://creativecommons.org/licenses/by/4.0/deed.en
title Depth-Aware Endoscopic Video Inpainting
spellingShingle Depth-Aware Endoscopic Video Inpainting
Xianghua Xie
title_short Depth-Aware Endoscopic Video Inpainting
title_full Depth-Aware Endoscopic Video Inpainting
title_fullStr Depth-Aware Endoscopic Video Inpainting
title_full_unstemmed Depth-Aware Endoscopic Video Inpainting
title_sort Depth-Aware Endoscopic Video Inpainting
author_id_str_mv b334d40963c7a2f435f06d2c26c74e11
author_id_fullname_str_mv b334d40963c7a2f435f06d2c26c74e11_***_Xianghua Xie
author Xianghua Xie
author2 Francis Xiatian Zhang
Shuang Chen
Xianghua Xie
Hubert P. H. Shum
format Conference Paper/Proceeding/Abstract
container_title Lecture Notes in Computer Science
container_volume 15006
container_start_page 143
publishDate 2024
institution Swansea University
isbn 9783031720888
9783031720895
issn 0302-9743
1611-3349
doi_str_mv 10.1007/978-3-031-72089-5_14
publisher Springer Nature Switzerland
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
url http://dx.doi.org/10.1007/978-3-031-72089-5_14
document_store_str 1
active_str 0
description Video inpainting fills in corrupted video content with plausible replacements. While recent advances in endoscopic video inpainting have shown potential for enhancing the quality of endoscopic videos,they mainly repair 2D visual information without effectively preserving crucial 3D spatial details for clinical reference. Depth-aware inpainting methods attempt to preserve these details by incorporating depth information. Still, in endoscopic contexts, they face challenges including reliance on pre-acquired depth maps, less effective fusion designs, and ignorance of the fidelity of 3D spatial details. To address them, we introduce a novel Depth-aware Endoscopic Video Inpainting (DAEVI) framework. It features a Spatial-Temporal Guided Depth Estimation module for direct depth estimation from visual features, a Bi-Modal Paired Channel Fusion module for effective channel-by-channel fusion of visual and depth information, and a Depth Enhanced Discriminator to assess the fidelity of the RGB-D sequence comprised of the inpainted frames and estimated depth images. Experimental evaluations on established benchmarks demonstrate our framework’s superiority, achieving a 2% improvementin PSNR and a 6% reduction in MSE compared to state-of-the-art methods. Qualitative analyses further validate its enhanced ability to inpaint fine details, highlighting the benefits of integrating depth information into endoscopic inpainting.
published_date 2024-10-03T08:32:12Z
_version_ 1821393638297436160
score 11.047501

Similar Items