No Cover Image

Conference Paper/Proceeding/Abstract 148 views 21 downloads

Depth-Aware Endoscopic Video Inpainting

Francis Xiatian Zhang, Shuang Chen, Xianghua Xie Orcid Logo, Hubert P. H. Shum

27th INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION (MICCAI), 2024

Swansea University Author: Xianghua Xie Orcid Logo

  • 66924.pdf

    PDF | Accepted Manuscript

    Author accepted manuscript document released under the terms of a Creative Commons CC-BY licence using the Swansea University Research Publications Policy (rights retention).

    Download (10.96MB)

Abstract

Video inpainting fills in corrupted video content with plausible replacements. While recent advances in endoscopic video inpainting have shown potential for enhancing the quality of endoscopic videos,they mainly repair 2D visual information without effectively preserving crucial 3D spatial details f...

Full description

Published in: 27th INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION (MICCAI), 2024
Published:
URI: https://cronfa.swan.ac.uk/Record/cronfa66924
Tags: Add Tag
No Tags, Be the first to tag this record!
first_indexed 2024-07-02T12:58:50Z
last_indexed 2024-07-02T12:58:50Z
id cronfa66924
recordtype SURis
fullrecord <?xml version="1.0" encoding="utf-8"?><rfc1807 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><bib-version>v2</bib-version><id>66924</id><entry>2024-07-02</entry><title>Depth-Aware Endoscopic Video Inpainting</title><swanseaauthors><author><sid>b334d40963c7a2f435f06d2c26c74e11</sid><ORCID>0000-0002-2701-8660</ORCID><firstname>Xianghua</firstname><surname>Xie</surname><name>Xianghua Xie</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2024-07-02</date><deptcode>MACS</deptcode><abstract>Video inpainting fills in corrupted video content with plausible replacements. While recent advances in endoscopic video inpainting have shown potential for enhancing the quality of endoscopic videos,they mainly repair 2D visual information without effectively preserving crucial 3D spatial details for clinical reference. Depth-aware inpainting methods attempt to preserve these details by incorporating depth information. Still, in endoscopic contexts, they face challenges including reliance on pre-acquired depth maps, less effective fusion designs, and ignorance of the fidelity of 3D spatial details. To address them, we introduce a novel Depth-aware Endoscopic Video Inpainting (DAEVI) framework. It features a Spatial-Temporal Guided Depth Estimation module for direct depth estimation from visual features, a Bi-Modal Paired Channel Fusion module for effective channel-by-channel fusion of visual and depth information, and a Depth Enhanced Discriminator to assess the fidelity of the RGB-D sequence comprised of the inpainted frames and estimated depth images. Experimental evaluations on established benchmarks demonstrate our framework’s superiority, achieving a 2% improvementin PSNR and a 6% reduction in MSE compared to state-of-the-art methods. Qualitative analyses further validate its enhanced ability to inpaint fine details, highlighting the benefits of integrating depth information into endoscopic inpainting.</abstract><type>Conference Paper/Proceeding/Abstract</type><journal>27th INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION (MICCAI), 2024</journal><volume/><journalNumber/><paginationStart/><paginationEnd/><publisher/><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic/><keywords/><publishedDay>0</publishedDay><publishedMonth>0</publishedMonth><publishedYear>0</publishedYear><publishedDate>0001-01-01</publishedDate><doi/><url/><notes/><college>COLLEGE NANME</college><department>Mathematics and Computer Science School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MACS</DepartmentCode><institution>Swansea University</institution><apcterm/><funders>This research is supported in part by the EPSRC NortHFutures project (ref: EP/X031012/1).</funders><projectreference/><lastEdited>2024-09-16T16:13:44.6893790</lastEdited><Created>2024-07-02T13:54:54.0019893</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Francis Xiatian</firstname><surname>Zhang</surname><order>1</order></author><author><firstname>Shuang</firstname><surname>Chen</surname><order>2</order></author><author><firstname>Xianghua</firstname><surname>Xie</surname><orcid>0000-0002-2701-8660</orcid><order>3</order></author><author><firstname>Hubert P. H.</firstname><surname>Shum</surname><order>4</order></author></authors><documents><document><filename>66924__30795__cf423692cd264ac9ac0deb5d523c9e93.pdf</filename><originalFilename>66924.pdf</originalFilename><uploaded>2024-07-02T13:58:48.1010254</uploaded><type>Output</type><contentLength>11490256</contentLength><contentType>application/pdf</contentType><version>Accepted Manuscript</version><cronfaStatus>true</cronfaStatus><documentNotes>Author accepted manuscript document released under the terms of a Creative Commons CC-BY licence using the Swansea University Research Publications Policy (rights retention).</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/deed.en</licence></document></documents><OutputDurs/></rfc1807>
spelling v2 66924 2024-07-02 Depth-Aware Endoscopic Video Inpainting b334d40963c7a2f435f06d2c26c74e11 0000-0002-2701-8660 Xianghua Xie Xianghua Xie true false 2024-07-02 MACS Video inpainting fills in corrupted video content with plausible replacements. While recent advances in endoscopic video inpainting have shown potential for enhancing the quality of endoscopic videos,they mainly repair 2D visual information without effectively preserving crucial 3D spatial details for clinical reference. Depth-aware inpainting methods attempt to preserve these details by incorporating depth information. Still, in endoscopic contexts, they face challenges including reliance on pre-acquired depth maps, less effective fusion designs, and ignorance of the fidelity of 3D spatial details. To address them, we introduce a novel Depth-aware Endoscopic Video Inpainting (DAEVI) framework. It features a Spatial-Temporal Guided Depth Estimation module for direct depth estimation from visual features, a Bi-Modal Paired Channel Fusion module for effective channel-by-channel fusion of visual and depth information, and a Depth Enhanced Discriminator to assess the fidelity of the RGB-D sequence comprised of the inpainted frames and estimated depth images. Experimental evaluations on established benchmarks demonstrate our framework’s superiority, achieving a 2% improvementin PSNR and a 6% reduction in MSE compared to state-of-the-art methods. Qualitative analyses further validate its enhanced ability to inpaint fine details, highlighting the benefits of integrating depth information into endoscopic inpainting. Conference Paper/Proceeding/Abstract 27th INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION (MICCAI), 2024 0 0 0 0001-01-01 COLLEGE NANME Mathematics and Computer Science School COLLEGE CODE MACS Swansea University This research is supported in part by the EPSRC NortHFutures project (ref: EP/X031012/1). 2024-09-16T16:13:44.6893790 2024-07-02T13:54:54.0019893 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Francis Xiatian Zhang 1 Shuang Chen 2 Xianghua Xie 0000-0002-2701-8660 3 Hubert P. H. Shum 4 66924__30795__cf423692cd264ac9ac0deb5d523c9e93.pdf 66924.pdf 2024-07-02T13:58:48.1010254 Output 11490256 application/pdf Accepted Manuscript true Author accepted manuscript document released under the terms of a Creative Commons CC-BY licence using the Swansea University Research Publications Policy (rights retention). true eng https://creativecommons.org/licenses/by/4.0/deed.en
title Depth-Aware Endoscopic Video Inpainting
spellingShingle Depth-Aware Endoscopic Video Inpainting
Xianghua Xie
title_short Depth-Aware Endoscopic Video Inpainting
title_full Depth-Aware Endoscopic Video Inpainting
title_fullStr Depth-Aware Endoscopic Video Inpainting
title_full_unstemmed Depth-Aware Endoscopic Video Inpainting
title_sort Depth-Aware Endoscopic Video Inpainting
author_id_str_mv b334d40963c7a2f435f06d2c26c74e11
author_id_fullname_str_mv b334d40963c7a2f435f06d2c26c74e11_***_Xianghua Xie
author Xianghua Xie
author2 Francis Xiatian Zhang
Shuang Chen
Xianghua Xie
Hubert P. H. Shum
format Conference Paper/Proceeding/Abstract
container_title 27th INTERNATIONAL CONFERENCE ON MEDICAL IMAGE COMPUTING AND COMPUTER ASSISTED INTERVENTION (MICCAI), 2024
institution Swansea University
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 1
active_str 0
description Video inpainting fills in corrupted video content with plausible replacements. While recent advances in endoscopic video inpainting have shown potential for enhancing the quality of endoscopic videos,they mainly repair 2D visual information without effectively preserving crucial 3D spatial details for clinical reference. Depth-aware inpainting methods attempt to preserve these details by incorporating depth information. Still, in endoscopic contexts, they face challenges including reliance on pre-acquired depth maps, less effective fusion designs, and ignorance of the fidelity of 3D spatial details. To address them, we introduce a novel Depth-aware Endoscopic Video Inpainting (DAEVI) framework. It features a Spatial-Temporal Guided Depth Estimation module for direct depth estimation from visual features, a Bi-Modal Paired Channel Fusion module for effective channel-by-channel fusion of visual and depth information, and a Depth Enhanced Discriminator to assess the fidelity of the RGB-D sequence comprised of the inpainted frames and estimated depth images. Experimental evaluations on established benchmarks demonstrate our framework’s superiority, achieving a 2% improvementin PSNR and a 6% reduction in MSE compared to state-of-the-art methods. Qualitative analyses further validate its enhanced ability to inpaint fine details, highlighting the benefits of integrating depth information into endoscopic inpainting.
published_date 0001-01-01T16:13:43Z
_version_ 1810366068777025536
score 11.037603

Similar Items