No Cover Image

Journal article 1154 views

An Augmented Reality Based Human-Robot Interaction Interface Using Kalman Filter Sensor Fusion

Chunxu Li Orcid Logo, Ashraf Fahmy Abdo Orcid Logo, Johann Sienz Orcid Logo

Sensors, Volume: 19, Issue: 20, Start page: 4586

Swansea University Authors: Chunxu Li Orcid Logo, Ashraf Fahmy Abdo Orcid Logo, Johann Sienz Orcid Logo

Full text not available from this repository: check for access using links below.

Check full text

DOI (Published version): 10.3390/s19204586

Abstract

In this paper, the application of Augmented Reality (AR) for the control and adjustment of robots has been developed, with the aim of making interaction and adjustment of robots easier and more accurate from a remote location. A LeapMotion sensor based controller has been investigated to track the m...

Full description

Published in: Sensors
ISSN: 1424-8220
Published: MDPI AG 2019
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa52479
first_indexed 2019-10-17T14:22:41Z
last_indexed 2024-11-14T12:03:19Z
id cronfa52479
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2024-04-10T09:32:30.3307466</datestamp><bib-version>v2</bib-version><id>52479</id><entry>2019-10-17</entry><title>An Augmented Reality Based Human-Robot Interaction Interface Using Kalman Filter Sensor Fusion</title><swanseaauthors><author><sid>e6ed70d02c25b05ab52340312559d684</sid><ORCID>0000-0001-7851-0260</ORCID><firstname>Chunxu</firstname><surname>Li</surname><name>Chunxu Li</name><active>true</active><ethesisStudent>false</ethesisStudent></author><author><sid>b952b837f8a8447055210d209892b427</sid><ORCID>0000-0003-1624-1725</ORCID><firstname>Ashraf</firstname><surname>Fahmy Abdo</surname><name>Ashraf Fahmy Abdo</name><active>true</active><ethesisStudent>false</ethesisStudent></author><author><sid>17bf1dd287bff2cb01b53d98ceb28a31</sid><ORCID>0000-0003-3136-5718</ORCID><firstname>Johann</firstname><surname>Sienz</surname><name>Johann Sienz</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2019-10-17</date><deptcode>ACEM</deptcode><abstract>In this paper, the application of Augmented Reality (AR) for the control and adjustment of robots has been developed, with the aim of making interaction and adjustment of robots easier and more accurate from a remote location. A LeapMotion sensor based controller has been investigated to track the movement of the operator hands. The data from the controller allows gestures and the position of the hand palm&#x2019;s central point to be detected and tracked. A Kinect V2 camera is able to measure the corresponding motion velocities in x, y, z directions after our investigated post-processing algorithm is fulfilled. Unreal Engine 4 is used to create an AR environment for the user to monitor the control process immersively. Kalman filtering (KF) algorithm is employed to fuse the position signals from the LeapMotion sensor with the velocity signals from the Kinect camera sensor, respectively. The fused/optimal data are sent to teleoperate a Baxter robot in real-time by User Datagram Protocol (UDP). Several experiments have been conducted to test the validation of the proposed method.</abstract><type>Journal Article</type><journal>Sensors</journal><volume>19</volume><journalNumber>20</journalNumber><paginationStart>4586</paginationStart><paginationEnd/><publisher>MDPI AG</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic>1424-8220</issnElectronic><keywords>augmented reality; human-robot interaction; LeapMotion sensor; Kinect sensor; Kalman filter sensor fusion</keywords><publishedDay>22</publishedDay><publishedMonth>10</publishedMonth><publishedYear>2019</publishedYear><publishedDate>2019-10-22</publishedDate><doi>10.3390/s19204586</doi><url/><notes/><college>COLLEGE NANME</college><department>Aerospace, Civil, Electrical, and Mechanical Engineering</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>ACEM</DepartmentCode><institution>Swansea University</institution><apcterm/><funders/><projectreference/><lastEdited>2024-04-10T09:32:30.3307466</lastEdited><Created>2019-10-17T09:56:39.5775343</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Mechanical Engineering</level></path><authors><author><firstname>Chunxu</firstname><surname>Li</surname><orcid>0000-0001-7851-0260</orcid><order>1</order></author><author><firstname>Ashraf</firstname><surname>Fahmy Abdo</surname><orcid>0000-0003-1624-1725</orcid><order>2</order></author><author><firstname>Johann</firstname><surname>Sienz</surname><orcid>0000-0003-3136-5718</orcid><order>3</order></author></authors><documents/><OutputDurs/></rfc1807>
spelling 2024-04-10T09:32:30.3307466 v2 52479 2019-10-17 An Augmented Reality Based Human-Robot Interaction Interface Using Kalman Filter Sensor Fusion e6ed70d02c25b05ab52340312559d684 0000-0001-7851-0260 Chunxu Li Chunxu Li true false b952b837f8a8447055210d209892b427 0000-0003-1624-1725 Ashraf Fahmy Abdo Ashraf Fahmy Abdo true false 17bf1dd287bff2cb01b53d98ceb28a31 0000-0003-3136-5718 Johann Sienz Johann Sienz true false 2019-10-17 ACEM In this paper, the application of Augmented Reality (AR) for the control and adjustment of robots has been developed, with the aim of making interaction and adjustment of robots easier and more accurate from a remote location. A LeapMotion sensor based controller has been investigated to track the movement of the operator hands. The data from the controller allows gestures and the position of the hand palm’s central point to be detected and tracked. A Kinect V2 camera is able to measure the corresponding motion velocities in x, y, z directions after our investigated post-processing algorithm is fulfilled. Unreal Engine 4 is used to create an AR environment for the user to monitor the control process immersively. Kalman filtering (KF) algorithm is employed to fuse the position signals from the LeapMotion sensor with the velocity signals from the Kinect camera sensor, respectively. The fused/optimal data are sent to teleoperate a Baxter robot in real-time by User Datagram Protocol (UDP). Several experiments have been conducted to test the validation of the proposed method. Journal Article Sensors 19 20 4586 MDPI AG 1424-8220 augmented reality; human-robot interaction; LeapMotion sensor; Kinect sensor; Kalman filter sensor fusion 22 10 2019 2019-10-22 10.3390/s19204586 COLLEGE NANME Aerospace, Civil, Electrical, and Mechanical Engineering COLLEGE CODE ACEM Swansea University 2024-04-10T09:32:30.3307466 2019-10-17T09:56:39.5775343 Faculty of Science and Engineering School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Mechanical Engineering Chunxu Li 0000-0001-7851-0260 1 Ashraf Fahmy Abdo 0000-0003-1624-1725 2 Johann Sienz 0000-0003-3136-5718 3
title An Augmented Reality Based Human-Robot Interaction Interface Using Kalman Filter Sensor Fusion
spellingShingle An Augmented Reality Based Human-Robot Interaction Interface Using Kalman Filter Sensor Fusion
Chunxu Li
Ashraf Fahmy Abdo
Johann Sienz
title_short An Augmented Reality Based Human-Robot Interaction Interface Using Kalman Filter Sensor Fusion
title_full An Augmented Reality Based Human-Robot Interaction Interface Using Kalman Filter Sensor Fusion
title_fullStr An Augmented Reality Based Human-Robot Interaction Interface Using Kalman Filter Sensor Fusion
title_full_unstemmed An Augmented Reality Based Human-Robot Interaction Interface Using Kalman Filter Sensor Fusion
title_sort An Augmented Reality Based Human-Robot Interaction Interface Using Kalman Filter Sensor Fusion
author_id_str_mv e6ed70d02c25b05ab52340312559d684
b952b837f8a8447055210d209892b427
17bf1dd287bff2cb01b53d98ceb28a31
author_id_fullname_str_mv e6ed70d02c25b05ab52340312559d684_***_Chunxu Li
b952b837f8a8447055210d209892b427_***_Ashraf Fahmy Abdo
17bf1dd287bff2cb01b53d98ceb28a31_***_Johann Sienz
author Chunxu Li
Ashraf Fahmy Abdo
Johann Sienz
author2 Chunxu Li
Ashraf Fahmy Abdo
Johann Sienz
format Journal article
container_title Sensors
container_volume 19
container_issue 20
container_start_page 4586
publishDate 2019
institution Swansea University
issn 1424-8220
doi_str_mv 10.3390/s19204586
publisher MDPI AG
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Mechanical Engineering{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Mechanical Engineering
document_store_str 0
active_str 0
description In this paper, the application of Augmented Reality (AR) for the control and adjustment of robots has been developed, with the aim of making interaction and adjustment of robots easier and more accurate from a remote location. A LeapMotion sensor based controller has been investigated to track the movement of the operator hands. The data from the controller allows gestures and the position of the hand palm’s central point to be detected and tracked. A Kinect V2 camera is able to measure the corresponding motion velocities in x, y, z directions after our investigated post-processing algorithm is fulfilled. Unreal Engine 4 is used to create an AR environment for the user to monitor the control process immersively. Kalman filtering (KF) algorithm is employed to fuse the position signals from the LeapMotion sensor with the velocity signals from the Kinect camera sensor, respectively. The fused/optimal data are sent to teleoperate a Baxter robot in real-time by User Datagram Protocol (UDP). Several experiments have been conducted to test the validation of the proposed method.
published_date 2019-10-22T07:49:43Z
_version_ 1821390964646739968
score 11.364387

Similar Items