Journal article 78 views 6 downloads
Skill Acquisition and Controller Design of Desktop Robot Manipulator Based on Audio–Visual Information Fusion
,
Xiaoyu Chen,
Xinglu Ma,
Hao Sun,
Bin Wang
Machines, Volume: 10, Issue: 9, Start page: 772
Swansea University Author:
Chunxu Li
-
PDF | Version of Record
© 2022 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Download (5.92MB)
DOI (Published version): 10.3390/machines10090772
Abstract
The development of AI and robotics has led to an explosion of research and the number of implementations in automated systems. However, whilst commonplace in manufacturing, these approaches have not impacted chemistry due to difficulty in developing robot systems that are dexterous enough for experi...
| Published in: | Machines |
|---|---|
| ISSN: | 2075-1702 |
| Published: |
MDPI AG
2022
|
| Online Access: |
Check full text
|
| URI: | https://cronfa.swan.ac.uk/Record/cronfa66026 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract: |
The development of AI and robotics has led to an explosion of research and the number of implementations in automated systems. However, whilst commonplace in manufacturing, these approaches have not impacted chemistry due to difficulty in developing robot systems that are dexterous enough for experimental operation. In this paper, a control system for desktop experimental manipulators based on an audio-visual information fusion algorithm was designed. The robot could replace the operator to complete some tedious and dangerous experimental work by teaching it the arm movement skills. The system is divided into two parts: skill acquisition and movement control. For the former, the visual signal was obtained through two algorithms of motion detection, which were realized by an improved two-stream convolutional network; the audio signal was extracted by Voice AI with regular expressions. Then, we combined the audio and visual information to obtain high coincidence motor skills. The accuracy of skill acquisition can reach more than 81%. The latter employed motor control and grasping pose recognition, which achieved precise controlling and grasping. The system can be used for the teaching and control work of chemical experiments with specific processes. It can replace the operator to complete the chemical experiment work while greatly reducing the programming threshold and improving the efficiency. |
|---|---|
| Keywords: |
desktop experimental manipulators; skill acquisition; motion control; motion detection; speech recognition; information fusion; pose recognition |
| College: |
Faculty of Science and Engineering |
| Funders: |
This research received no external funding. |
| Issue: |
9 |
| Start Page: |
772 |