No Cover Image

Journal article 240 views 54 downloads

Skill Acquisition and Controller Design of Desktop Robot Manipulator Based on Audio–Visual Information Fusion

Chunxu Li Orcid Logo, Xiaoyu Chen, Xinglu Ma, Hao Sun, Bin Wang

Machines, Volume: 10, Issue: 9, Start page: 772

Swansea University Author: Chunxu Li Orcid Logo

  • 66026.VoR.pdf

    PDF | Version of Record

    © 2022 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.

    Download (5.92MB)

Check full text

DOI (Published version): 10.3390/machines10090772

Abstract

The development of AI and robotics has led to an explosion of research and the number of implementations in automated systems. However, whilst commonplace in manufacturing, these approaches have not impacted chemistry due to difficulty in developing robot systems that are dexterous enough for experi...

Full description

Published in: Machines
ISSN: 2075-1702
Published: MDPI AG 2022
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa66026
first_indexed 2024-04-10T08:29:46Z
last_indexed 2024-11-25T14:17:21Z
id cronfa66026
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2024-05-22T14:58:00.2861456</datestamp><bib-version>v2</bib-version><id>66026</id><entry>2024-04-09</entry><title>Skill Acquisition and Controller Design of Desktop Robot Manipulator Based on Audio&#x2013;Visual Information Fusion</title><swanseaauthors><author><sid>e6ed70d02c25b05ab52340312559d684</sid><ORCID>0000-0001-7851-0260</ORCID><firstname>Chunxu</firstname><surname>Li</surname><name>Chunxu Li</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2024-04-09</date><deptcode>ACEM</deptcode><abstract>The development of AI and robotics has led to an explosion of research and the number of implementations in automated systems. However, whilst commonplace in manufacturing, these approaches have not impacted chemistry due to difficulty in developing robot systems that are dexterous enough for experimental operation. In this paper, a control system for desktop experimental manipulators based on an audio-visual information fusion algorithm was designed. The robot could replace the operator to complete some tedious and dangerous experimental work by teaching it the arm movement skills. The system is divided into two parts: skill acquisition and movement control. For the former, the visual signal was obtained through two algorithms of motion detection, which were realized by an improved two-stream convolutional network; the audio signal was extracted by Voice AI with regular expressions. Then, we combined the audio and visual information to obtain high coincidence motor skills. The accuracy of skill acquisition can reach more than 81%. The latter employed motor control and grasping pose recognition, which achieved precise controlling and grasping. The system can be used for the teaching and control work of chemical experiments with specific processes. It can replace the operator to complete the chemical experiment work while greatly reducing the programming threshold and improving the efficiency.</abstract><type>Journal Article</type><journal>Machines</journal><volume>10</volume><journalNumber>9</journalNumber><paginationStart>772</paginationStart><paginationEnd/><publisher>MDPI AG</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic>2075-1702</issnElectronic><keywords>desktop experimental manipulators; skill acquisition; motion control; motion detection; speech recognition; information fusion; pose recognition</keywords><publishedDay>6</publishedDay><publishedMonth>9</publishedMonth><publishedYear>2022</publishedYear><publishedDate>2022-09-06</publishedDate><doi>10.3390/machines10090772</doi><url/><notes/><college>COLLEGE NANME</college><department>Aerospace, Civil, Electrical, and Mechanical Engineering</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>ACEM</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders>This research received no external funding.</funders><projectreference/><lastEdited>2024-05-22T14:58:00.2861456</lastEdited><Created>2024-04-09T20:19:08.0108395</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Mechanical Engineering</level></path><authors><author><firstname>Chunxu</firstname><surname>Li</surname><orcid>0000-0001-7851-0260</orcid><order>1</order></author><author><firstname>Xiaoyu</firstname><surname>Chen</surname><order>2</order></author><author><firstname>Xinglu</firstname><surname>Ma</surname><order>3</order></author><author><firstname>Hao</firstname><surname>Sun</surname><order>4</order></author><author><firstname>Bin</firstname><surname>Wang</surname><order>5</order></author></authors><documents><document><filename>66026__30436__30edca6bdb8441329abcb104d8579dd7.pdf</filename><originalFilename>66026.VoR.pdf</originalFilename><uploaded>2024-05-22T14:56:49.8636481</uploaded><type>Output</type><contentLength>6203473</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>&#xA9; 2022 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling 2024-05-22T14:58:00.2861456 v2 66026 2024-04-09 Skill Acquisition and Controller Design of Desktop Robot Manipulator Based on Audio–Visual Information Fusion e6ed70d02c25b05ab52340312559d684 0000-0001-7851-0260 Chunxu Li Chunxu Li true false 2024-04-09 ACEM The development of AI and robotics has led to an explosion of research and the number of implementations in automated systems. However, whilst commonplace in manufacturing, these approaches have not impacted chemistry due to difficulty in developing robot systems that are dexterous enough for experimental operation. In this paper, a control system for desktop experimental manipulators based on an audio-visual information fusion algorithm was designed. The robot could replace the operator to complete some tedious and dangerous experimental work by teaching it the arm movement skills. The system is divided into two parts: skill acquisition and movement control. For the former, the visual signal was obtained through two algorithms of motion detection, which were realized by an improved two-stream convolutional network; the audio signal was extracted by Voice AI with regular expressions. Then, we combined the audio and visual information to obtain high coincidence motor skills. The accuracy of skill acquisition can reach more than 81%. The latter employed motor control and grasping pose recognition, which achieved precise controlling and grasping. The system can be used for the teaching and control work of chemical experiments with specific processes. It can replace the operator to complete the chemical experiment work while greatly reducing the programming threshold and improving the efficiency. Journal Article Machines 10 9 772 MDPI AG 2075-1702 desktop experimental manipulators; skill acquisition; motion control; motion detection; speech recognition; information fusion; pose recognition 6 9 2022 2022-09-06 10.3390/machines10090772 COLLEGE NANME Aerospace, Civil, Electrical, and Mechanical Engineering COLLEGE CODE ACEM Swansea University Another institution paid the OA fee This research received no external funding. 2024-05-22T14:58:00.2861456 2024-04-09T20:19:08.0108395 Faculty of Science and Engineering School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Mechanical Engineering Chunxu Li 0000-0001-7851-0260 1 Xiaoyu Chen 2 Xinglu Ma 3 Hao Sun 4 Bin Wang 5 66026__30436__30edca6bdb8441329abcb104d8579dd7.pdf 66026.VoR.pdf 2024-05-22T14:56:49.8636481 Output 6203473 application/pdf Version of Record true © 2022 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. true eng https://creativecommons.org/licenses/by/4.0/
title Skill Acquisition and Controller Design of Desktop Robot Manipulator Based on Audio–Visual Information Fusion
spellingShingle Skill Acquisition and Controller Design of Desktop Robot Manipulator Based on Audio–Visual Information Fusion
Chunxu Li
title_short Skill Acquisition and Controller Design of Desktop Robot Manipulator Based on Audio–Visual Information Fusion
title_full Skill Acquisition and Controller Design of Desktop Robot Manipulator Based on Audio–Visual Information Fusion
title_fullStr Skill Acquisition and Controller Design of Desktop Robot Manipulator Based on Audio–Visual Information Fusion
title_full_unstemmed Skill Acquisition and Controller Design of Desktop Robot Manipulator Based on Audio–Visual Information Fusion
title_sort Skill Acquisition and Controller Design of Desktop Robot Manipulator Based on Audio–Visual Information Fusion
author_id_str_mv e6ed70d02c25b05ab52340312559d684
author_id_fullname_str_mv e6ed70d02c25b05ab52340312559d684_***_Chunxu Li
author Chunxu Li
author2 Chunxu Li
Xiaoyu Chen
Xinglu Ma
Hao Sun
Bin Wang
format Journal article
container_title Machines
container_volume 10
container_issue 9
container_start_page 772
publishDate 2022
institution Swansea University
issn 2075-1702
doi_str_mv 10.3390/machines10090772
publisher MDPI AG
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Mechanical Engineering{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Mechanical Engineering
document_store_str 1
active_str 0
description The development of AI and robotics has led to an explosion of research and the number of implementations in automated systems. However, whilst commonplace in manufacturing, these approaches have not impacted chemistry due to difficulty in developing robot systems that are dexterous enough for experimental operation. In this paper, a control system for desktop experimental manipulators based on an audio-visual information fusion algorithm was designed. The robot could replace the operator to complete some tedious and dangerous experimental work by teaching it the arm movement skills. The system is divided into two parts: skill acquisition and movement control. For the former, the visual signal was obtained through two algorithms of motion detection, which were realized by an improved two-stream convolutional network; the audio signal was extracted by Voice AI with regular expressions. Then, we combined the audio and visual information to obtain high coincidence motor skills. The accuracy of skill acquisition can reach more than 81%. The latter employed motor control and grasping pose recognition, which achieved precise controlling and grasping. The system can be used for the teaching and control work of chemical experiments with specific processes. It can replace the operator to complete the chemical experiment work while greatly reducing the programming threshold and improving the efficiency.
published_date 2022-09-06T08:29:23Z
_version_ 1821393460563804160
score 11.047999

Similar Items