Predator-Prey Q-Learning Based Collaborative Coverage Path Planning for Swarm Robotics

Watson, Michael; Ren, Hanchi; Arvin, Farshad; Hu, Junyan

doi:10.1007/978-3-031-72062-8_28

Conference Paper/Proceeding/Abstract 598 views 88 downloads

Predator-Prey Q-Learning Based Collaborative Coverage Path Planning for Swarm Robotics

Michael Watson, Hanchi Ren, Farshad Arvin, Junyan Hu

Towards Autonomous Robotic Systems: 25th Annual Conference, TAROS 2024, London, UK, August 21–23, 2024, Proceedings, Part II, Pages: 320 - 332

Swansea University Author: Hanchi Ren

PDF | Accepted Manuscript

Author accepted manuscript document released under the terms of a Creative Commons CC-BY licence using the Swansea University Research Publications Policy (rights retention).
Download (627KB)

Check full text

DOI (Published version): 10.1007/978-3-031-72062-8_28

Abstract

Coverage Path Planning (CPP) is an effective approach to let intelligent robots cover an area by finding feasible paths through the environment. In this paper, we focus on using reinforcement learning to learn about a given environment and find the most efficient path that explores all target points...

Full description

Published in:	Towards Autonomous Robotic Systems: 25th Annual Conference, TAROS 2024, London, UK, August 21–23, 2024, Proceedings, Part II
ISBN:	9783031720611 9783031720628
ISSN:	0302-9743 1611-3349
Published:	Cham Springer 2025
Online Access:	Check full text
URI:	https://cronfa.swan.ac.uk/Record/cronfa66908

first_indexed	2024-06-28T18:29:17Z
last_indexed	2025-02-11T05:48:27Z
id	cronfa66908
recordtype	SURis
fullrecord	<?xml version="1.0"?><rfc1807><datestamp>2025-02-10T15:03:31.2720763</datestamp><bib-version>v2</bib-version><id>66908</id><entry>2024-06-28</entry><title>Predator-Prey Q-Learning Based Collaborative Coverage Path Planning for Swarm Robotics</title><swanseaauthors><author><sid>9e043b899a2b786672a28ed4f864ffcc</sid><firstname>Hanchi</firstname><surname>Ren</surname><name>Hanchi Ren</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2024-06-28</date><deptcode>MACS</deptcode><abstract>Coverage Path Planning (CPP) is an effective approach to let intelligent robots cover an area by finding feasible paths through the environment. In this paper, we focus on using reinforcement learning to learn about a given environment and find the most efficient path that explores all target points. To overcome the limitations caused by standard Q-learning based CPP that often fall into a local optimum and may be in-efficient in large-scale environments, two methods of improvement are considered, i.e., the use of a robot swarm working towards the same goal and the augmenting of the Q-learning algorithm to include a predator-prey based reward system. Existing predator-prey based reward systems provide rewards the further away an agent is from its predator, the paper adapts this concept to work within a robot swarm by simulating each agent of the swarm as both predator and prey. Simulation case studies and comparisons with the standard Q-learning show that the proposed method has a superior coverage performance in complicated environments.</abstract><type>Conference Paper/Proceeding/Abstract</type><journal>Towards Autonomous Robotic Systems: 25th Annual Conference, TAROS 2024, London, UK, August 21–23, 2024, Proceedings, Part II</journal><volume/><journalNumber/><paginationStart>320</paginationStart><paginationEnd>332</paginationEnd><publisher>Springer</publisher><placeOfPublication>Cham</placeOfPublication><isbnPrint>9783031720611</isbnPrint><isbnElectronic>9783031720628</isbnElectronic><issnPrint>0302-9743</issnPrint><issnElectronic>1611-3349</issnElectronic><keywords>Coverage Path Planning; Reinforcement Learning; Swarm Robotics</keywords><publishedDay>1</publishedDay><publishedMonth>1</publishedMonth><publishedYear>2025</publishedYear><publishedDate>2025-01-01</publishedDate><doi>10.1007/978-3-031-72062-8_28</doi><url/><notes>Lecture Notes in Computer Science (LNAI, volume 15052)</notes><college>COLLEGE NANME</college><department>Mathematics and Computer Science School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MACS</DepartmentCode><institution>Swansea University</institution><apcterm>Not Required</apcterm><funders>This work was supported by EU H2020-FET-OPEN RoboRoyale project [grant number 964492].</funders><projectreference/><lastEdited>2025-02-10T15:03:31.2720763</lastEdited><Created>2024-06-28T19:24:58.2731592</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Michael</firstname><surname>Watson</surname><order>1</order></author><author><firstname>Hanchi</firstname><surname>Ren</surname><order>2</order></author><author><firstname>Farshad</firstname><surname>Arvin</surname><order>3</order></author><author><firstname>Junyan</firstname><surname>Hu</surname><order>4</order></author></authors><documents><document><filename>66908__30779__f837f9af2d2545a0b33f45fa48fa9072.pdf</filename><originalFilename>Michael_TAROS.pdf</originalFilename><uploaded>2024-06-28T19:28:46.8381822</uploaded><type>Output</type><contentLength>642046</contentLength><contentType>application/pdf</contentType><version>Accepted Manuscript</version><cronfaStatus>true</cronfaStatus><embargoDate>2024-07-28T00:00:00.0000000</embargoDate><documentNotes>Author accepted manuscript document released under the terms of a Creative Commons CC-BY licence using the Swansea University Research Publications Policy (rights retention).</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/deed.en</licence></document></documents><OutputDurs/></rfc1807>
spelling	2025-02-10T15:03:31.2720763 v2 66908 2024-06-28 Predator-Prey Q-Learning Based Collaborative Coverage Path Planning for Swarm Robotics 9e043b899a2b786672a28ed4f864ffcc Hanchi Ren Hanchi Ren true false 2024-06-28 MACS Coverage Path Planning (CPP) is an effective approach to let intelligent robots cover an area by finding feasible paths through the environment. In this paper, we focus on using reinforcement learning to learn about a given environment and find the most efficient path that explores all target points. To overcome the limitations caused by standard Q-learning based CPP that often fall into a local optimum and may be in-efficient in large-scale environments, two methods of improvement are considered, i.e., the use of a robot swarm working towards the same goal and the augmenting of the Q-learning algorithm to include a predator-prey based reward system. Existing predator-prey based reward systems provide rewards the further away an agent is from its predator, the paper adapts this concept to work within a robot swarm by simulating each agent of the swarm as both predator and prey. Simulation case studies and comparisons with the standard Q-learning show that the proposed method has a superior coverage performance in complicated environments. Conference Paper/Proceeding/Abstract Towards Autonomous Robotic Systems: 25th Annual Conference, TAROS 2024, London, UK, August 21–23, 2024, Proceedings, Part II 320 332 Springer Cham 9783031720611 9783031720628 0302-9743 1611-3349 Coverage Path Planning; Reinforcement Learning; Swarm Robotics 1 1 2025 2025-01-01 10.1007/978-3-031-72062-8_28 Lecture Notes in Computer Science (LNAI, volume 15052) COLLEGE NANME Mathematics and Computer Science School COLLEGE CODE MACS Swansea University Not Required This work was supported by EU H2020-FET-OPEN RoboRoyale project [grant number 964492]. 2025-02-10T15:03:31.2720763 2024-06-28T19:24:58.2731592 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Michael Watson 1 Hanchi Ren 2 Farshad Arvin 3 Junyan Hu 4 66908__30779__f837f9af2d2545a0b33f45fa48fa9072.pdf Michael_TAROS.pdf 2024-06-28T19:28:46.8381822 Output 642046 application/pdf Accepted Manuscript true 2024-07-28T00:00:00.0000000 Author accepted manuscript document released under the terms of a Creative Commons CC-BY licence using the Swansea University Research Publications Policy (rights retention). true eng https://creativecommons.org/licenses/by/4.0/deed.en
title	Predator-Prey Q-Learning Based Collaborative Coverage Path Planning for Swarm Robotics
spellingShingle	Predator-Prey Q-Learning Based Collaborative Coverage Path Planning for Swarm Robotics Hanchi Ren
title_short	Predator-Prey Q-Learning Based Collaborative Coverage Path Planning for Swarm Robotics
title_full	Predator-Prey Q-Learning Based Collaborative Coverage Path Planning for Swarm Robotics
title_fullStr	Predator-Prey Q-Learning Based Collaborative Coverage Path Planning for Swarm Robotics
title_full_unstemmed	Predator-Prey Q-Learning Based Collaborative Coverage Path Planning for Swarm Robotics
title_sort	Predator-Prey Q-Learning Based Collaborative Coverage Path Planning for Swarm Robotics
author_id_str_mv	9e043b899a2b786672a28ed4f864ffcc
author_id_fullname_str_mv	9e043b899a2b786672a28ed4f864ffcc_***_Hanchi Ren
author	Hanchi Ren
author2	Michael Watson Hanchi Ren Farshad Arvin Junyan Hu
format	Conference Paper/Proceeding/Abstract
container_title	Towards Autonomous Robotic Systems: 25th Annual Conference, TAROS 2024, London, UK, August 21–23, 2024, Proceedings, Part II
container_start_page	320
publishDate	2025
institution	Swansea University
isbn	9783031720611 9783031720628
issn	0302-9743 1611-3349
doi_str_mv	10.1007/978-3-031-72062-8_28
publisher	Springer
college_str	Faculty of Science and Engineering
hierarchytype
hierarchy_top_id	facultyofscienceandengineering
hierarchy_top_title	Faculty of Science and Engineering
hierarchy_parent_id	facultyofscienceandengineering
hierarchy_parent_title	Faculty of Science and Engineering
department_str	School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str	1
active_str	0
description	Coverage Path Planning (CPP) is an effective approach to let intelligent robots cover an area by finding feasible paths through the environment. In this paper, we focus on using reinforcement learning to learn about a given environment and find the most efficient path that explores all target points. To overcome the limitations caused by standard Q-learning based CPP that often fall into a local optimum and may be in-efficient in large-scale environments, two methods of improvement are considered, i.e., the use of a robot swarm working towards the same goal and the augmenting of the Q-learning algorithm to include a predator-prey based reward system. Existing predator-prey based reward systems provide rewards the further away an agent is from its predator, the paper adapts this concept to work within a robot swarm by simulating each agent of the swarm as both predator and prey. Simulation case studies and comparisons with the standard Q-learning show that the proposed method has a superior coverage performance in complicated environments.
published_date	2025-01-01T05:21:31Z
_version_	1851097446313099264
score	11.089572

Predator-Prey Q-Learning Based Collaborative Coverage Path Planning for Swarm Robotics

Similar Items