Conference Paper/Proceeding/Abstract 230 views 22 downloads
Depth-Aware Endoscopic Video Inpainting
,
Shuang Chen ,
Xianghua Xie ,
Hubert P. H. Shum
Lecture Notes in Computer Science, Volume: 15006, Pages: 143 - 153
Swansea University Author:
Xianghua Xie
-
PDF | Accepted Manuscript
Author accepted manuscript document released under the terms of a Creative Commons CC-BY licence using the Swansea University Research Publications Policy (rights retention).
Download (10.96MB)
DOI (Published version): 10.1007/978-3-031-72089-5_14
Abstract
Video inpainting fills in corrupted video content with plausible replacements. While recent advances in endoscopic video inpainting have shown potential for enhancing the quality of endoscopic videos,they mainly repair 2D visual information without effectively preserving crucial 3D spatial details f...
| Published in: | Lecture Notes in Computer Science |
|---|---|
| ISBN: | 9783031720888 9783031720895 |
| ISSN: | 0302-9743 1611-3349 |
| Published: |
Cham
Springer Nature Switzerland
2024
|
| Online Access: |
Check full text
|
| URI: | https://cronfa.swan.ac.uk/Record/cronfa66924 |
| Abstract: |
Video inpainting fills in corrupted video content with plausible replacements. While recent advances in endoscopic video inpainting have shown potential for enhancing the quality of endoscopic videos,they mainly repair 2D visual information without effectively preserving crucial 3D spatial details for clinical reference. Depth-aware inpainting methods attempt to preserve these details by incorporating depth information. Still, in endoscopic contexts, they face challenges including reliance on pre-acquired depth maps, less effective fusion designs, and ignorance of the fidelity of 3D spatial details. To address them, we introduce a novel Depth-aware Endoscopic Video Inpainting (DAEVI) framework. It features a Spatial-Temporal Guided Depth Estimation module for direct depth estimation from visual features, a Bi-Modal Paired Channel Fusion module for effective channel-by-channel fusion of visual and depth information, and a Depth Enhanced Discriminator to assess the fidelity of the RGB-D sequence comprised of the inpainted frames and estimated depth images. Experimental evaluations on established benchmarks demonstrate our frameworkâs superiority, achieving a 2% improvementin PSNR and a 6% reduction in MSE compared to state-of-the-art methods. Qualitative analyses further validate its enhanced ability to inpaint fine details, highlighting the benefits of integrating depth information into endoscopic inpainting. |
|---|---|
| College: |
Faculty of Science and Engineering |
| Funders: |
This research is supported in part by the EPSRC NortHFutures project (ref: EP/X031012/1). |
| Start Page: |
143 |
| End Page: |
153 |