An end-to-end dynamic point cloud geometry compression in latent space

Jiang, Zhaoyi; Wang, Guoliang; Tam, Gary; Song, Chao; Li, Frederick W.B.; Yang, Bailin

doi:10.1016/j.displa.2023.102528

Journal article 594 views 90 downloads

An end-to-end dynamic point cloud geometry compression in latent space

Zhaoyi Jiang

, Guoliang Wang, Gary Tam

, Chao Song, Frederick W.B. Li, Bailin Yang

Displays, Volume: 80, Start page: 102528

Swansea University Author: Gary Tam

PDF | Accepted Manuscript

For the purpose of Open Access the author has applied a CC BY copyright licence to any Author Accepted Manuscript version arising from this submission.
Download (1.43MB)

Check full text

DOI (Published version): 10.1016/j.displa.2023.102528

Abstract

Dynamic point clouds are widely used for 3D data representation in various applications such as immersive and mixed reality, robotics and autonomous driving. However, their irregularity and large scale make efficient compression and transmission a challenge. Existing methods require high bitrates to...

Full description

Published in:	Displays
ISSN:	0141-9382
Published:	Elsevier BV 2023
Online Access:	Check full text
URI:	https://cronfa.swan.ac.uk/Record/cronfa64182

first_indexed	2023-08-30T15:03:59Z
last_indexed	2024-11-25T14:13:35Z
id	cronfa64182
recordtype	SURis
fullrecord	<?xml version="1.0"?><rfc1807><datestamp>2024-09-24T16:02:00.2022183</datestamp><bib-version>v2</bib-version><id>64182</id><entry>2023-08-30</entry><title>An end-to-end dynamic point cloud geometry compression in latent space</title><swanseaauthors><author><sid>e75a68e11a20e5f1da94ee6e28ff5e76</sid><ORCID>0000-0001-7387-5180</ORCID><firstname>Gary</firstname><surname>Tam</surname><name>Gary Tam</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2023-08-30</date><deptcode>MACS</deptcode><abstract>Dynamic point clouds are widely used for 3D data representation in various applications such as immersive and mixed reality, robotics and autonomous driving. However, their irregularity and large scale make efficient compression and transmission a challenge. Existing methods require high bitrates to encode point clouds since temporal correlation is not well considered. This paper proposes an end-to-end dynamic point cloud compression network that operates in latent space, resulting in more accurate motion estimation and more effective motion compensation. Specifically, a multi-scale motion estimation network is introduced to obtain accurate motion vectors. Motion information computed at a coarser level is upsampled and warped to the finer level based on cost volume analysis for motion compensation. Additionally, a residual compression network is designed to mitigate the effects of noise and inaccurate predictions by encoding latent residuals, resulting in smaller conditional entropy and better results. The proposed method achieves an average 12.09% and 14.76% (D2) BD-Rate gain over state-of-the-art Deep Dynamic Point Cloud Compression (D-DPCC) in experimental results. Compared to V-PCC, our framework showed an average improvement of 81.29% (D1) and 77.57% (D2).</abstract><type>Journal Article</type><journal>Displays</journal><volume>80</volume><journalNumber/><paginationStart>102528</paginationStart><paginationEnd/><publisher>Elsevier BV</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint>0141-9382</issnPrint><issnElectronic/><keywords>Dynamic point clouds compression, Geometry encoding, Latent scene flow, Deep entropy model</keywords><publishedDay>1</publishedDay><publishedMonth>12</publishedMonth><publishedYear>2023</publishedYear><publishedDate>2023-12-01</publishedDate><doi>10.1016/j.displa.2023.102528</doi><url/><notes/><college>COLLEGE NANME</college><department>Mathematics and Computer Science School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MACS</DepartmentCode><institution>Swansea University</institution><apcterm/><funders>This research was partially supported by Zhejiang Province Natural Science Foundation No. LY21F020013, LY22F020013, the National Natural Science Foundation of China No. 62172366. Gary Tam is supported by the Royal Society grant IEC/NSFC/211159. For the purpose of Open Access the author has applied a CC BY copyright licence to any Author Accepted Manuscript version arising from this submission.</funders><projectreference/><lastEdited>2024-09-24T16:02:00.2022183</lastEdited><Created>2023-08-30T15:53:10.5321838</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Zhaoyi</firstname><surname>Jiang</surname><orcid>0000-0001-5347-7935</orcid><order>1</order></author><author><firstname>Guoliang</firstname><surname>Wang</surname><order>2</order></author><author><firstname>Gary</firstname><surname>Tam</surname><orcid>0000-0001-7387-5180</orcid><order>3</order></author><author><firstname>Chao</firstname><surname>Song</surname><order>4</order></author><author><firstname>Frederick W.B.</firstname><surname>Li</surname><order>5</order></author><author><firstname>Bailin</firstname><surname>Yang</surname><order>6</order></author></authors><documents><document><filename>64182__28401__369152392d854ae1897036e8bd1c4a54.pdf</filename><originalFilename>cgi2023_elsarticle_DISPLA__Copy_accepted.pdf</originalFilename><uploaded>2023-08-30T15:59:11.1996250</uploaded><type>Output</type><contentLength>1502337</contentLength><contentType>application/pdf</contentType><version>Accepted Manuscript</version><cronfaStatus>true</cronfaStatus><documentNotes>For the purpose of Open Access the author has applied a CC BY copyright licence to any Author Accepted Manuscript version arising from this submission.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>http://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling	2024-09-24T16:02:00.2022183 v2 64182 2023-08-30 An end-to-end dynamic point cloud geometry compression in latent space e75a68e11a20e5f1da94ee6e28ff5e76 0000-0001-7387-5180 Gary Tam Gary Tam true false 2023-08-30 MACS Dynamic point clouds are widely used for 3D data representation in various applications such as immersive and mixed reality, robotics and autonomous driving. However, their irregularity and large scale make efficient compression and transmission a challenge. Existing methods require high bitrates to encode point clouds since temporal correlation is not well considered. This paper proposes an end-to-end dynamic point cloud compression network that operates in latent space, resulting in more accurate motion estimation and more effective motion compensation. Specifically, a multi-scale motion estimation network is introduced to obtain accurate motion vectors. Motion information computed at a coarser level is upsampled and warped to the finer level based on cost volume analysis for motion compensation. Additionally, a residual compression network is designed to mitigate the effects of noise and inaccurate predictions by encoding latent residuals, resulting in smaller conditional entropy and better results. The proposed method achieves an average 12.09% and 14.76% (D2) BD-Rate gain over state-of-the-art Deep Dynamic Point Cloud Compression (D-DPCC) in experimental results. Compared to V-PCC, our framework showed an average improvement of 81.29% (D1) and 77.57% (D2). Journal Article Displays 80 102528 Elsevier BV 0141-9382 Dynamic point clouds compression, Geometry encoding, Latent scene flow, Deep entropy model 1 12 2023 2023-12-01 10.1016/j.displa.2023.102528 COLLEGE NANME Mathematics and Computer Science School COLLEGE CODE MACS Swansea University This research was partially supported by Zhejiang Province Natural Science Foundation No. LY21F020013, LY22F020013, the National Natural Science Foundation of China No. 62172366. Gary Tam is supported by the Royal Society grant IEC/NSFC/211159. For the purpose of Open Access the author has applied a CC BY copyright licence to any Author Accepted Manuscript version arising from this submission. 2024-09-24T16:02:00.2022183 2023-08-30T15:53:10.5321838 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Zhaoyi Jiang 0000-0001-5347-7935 1 Guoliang Wang 2 Gary Tam 0000-0001-7387-5180 3 Chao Song 4 Frederick W.B. Li 5 Bailin Yang 6 64182__28401__369152392d854ae1897036e8bd1c4a54.pdf cgi2023_elsarticle_DISPLA__Copy_accepted.pdf 2023-08-30T15:59:11.1996250 Output 1502337 application/pdf Accepted Manuscript true For the purpose of Open Access the author has applied a CC BY copyright licence to any Author Accepted Manuscript version arising from this submission. true eng http://creativecommons.org/licenses/by/4.0/
title	An end-to-end dynamic point cloud geometry compression in latent space
spellingShingle	An end-to-end dynamic point cloud geometry compression in latent space Gary Tam
title_short	An end-to-end dynamic point cloud geometry compression in latent space
title_full	An end-to-end dynamic point cloud geometry compression in latent space
title_fullStr	An end-to-end dynamic point cloud geometry compression in latent space
title_full_unstemmed	An end-to-end dynamic point cloud geometry compression in latent space
title_sort	An end-to-end dynamic point cloud geometry compression in latent space
author_id_str_mv	e75a68e11a20e5f1da94ee6e28ff5e76
author_id_fullname_str_mv	e75a68e11a20e5f1da94ee6e28ff5e76_***_Gary Tam
author	Gary Tam
author2	Zhaoyi Jiang Guoliang Wang Gary Tam Chao Song Frederick W.B. Li Bailin Yang
format	Journal article
container_title	Displays
container_volume	80
container_start_page	102528
publishDate	2023
institution	Swansea University
issn	0141-9382
doi_str_mv	10.1016/j.displa.2023.102528
publisher	Elsevier BV
college_str	Faculty of Science and Engineering
hierarchytype
hierarchy_top_id	facultyofscienceandengineering
hierarchy_top_title	Faculty of Science and Engineering
hierarchy_parent_id	facultyofscienceandengineering
hierarchy_parent_title	Faculty of Science and Engineering
department_str	School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str	1
active_str	0
description	Dynamic point clouds are widely used for 3D data representation in various applications such as immersive and mixed reality, robotics and autonomous driving. However, their irregularity and large scale make efficient compression and transmission a challenge. Existing methods require high bitrates to encode point clouds since temporal correlation is not well considered. This paper proposes an end-to-end dynamic point cloud compression network that operates in latent space, resulting in more accurate motion estimation and more effective motion compensation. Specifically, a multi-scale motion estimation network is introduced to obtain accurate motion vectors. Motion information computed at a coarser level is upsampled and warped to the finer level based on cost volume analysis for motion compensation. Additionally, a residual compression network is designed to mitigate the effects of noise and inaccurate predictions by encoding latent residuals, resulting in smaller conditional entropy and better results. The proposed method achieves an average 12.09% and 14.76% (D2) BD-Rate gain over state-of-the-art Deep Dynamic Point Cloud Compression (D-DPCC) in experimental results. Compared to V-PCC, our framework showed an average improvement of 81.29% (D1) and 77.57% (D2).
published_date	2023-12-01T08:05:47Z
_version_	1827914957116669952
score	11.055693

An end-to-end dynamic point cloud geometry compression in latent space

Similar Items