No Cover Image

Conference Paper/Proceeding/Abstract 672 views 62 downloads

EmotiMask: Mapping Mouth Movements to an LED Matrix for Improving Recognition When Teaching With a Face Mask

Salim Hasshu, Stuart O’Connor, Simon Colreavy-Donnelly, Stefan Kuhn, Fabio Caraffini Orcid Logo

ECE Official Conference Proceedings

Swansea University Author: Fabio Caraffini Orcid Logo

Check full text

DOI (Published version): 10.22492/issn.2188-1162.2022.49

Abstract

The Covid-19 pandemic has led to the adoption of face masks in physical teaching spaces across the world. This has in-turn presented a number of challenges for practitioners in the face to face delivery of content and in effectively engaging learners in practical settings, where face coverings are a...

Full description

Published in: ECE Official Conference Proceedings
ISBN: 2188-1162
ISSN: 2188-1162
Published: The International Academic Forum(IAFOR) 2022
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa61352
Tags: Add Tag
No Tags, Be the first to tag this record!
first_indexed 2022-09-27T08:13:02Z
last_indexed 2023-01-13T19:22:04Z
id cronfa61352
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2022-10-11T14:56:44.7222340</datestamp><bib-version>v2</bib-version><id>61352</id><entry>2022-09-27</entry><title>EmotiMask: Mapping Mouth Movements to an LED Matrix for Improving Recognition When Teaching With a Face Mask</title><swanseaauthors><author><sid>d0b8d4e63d512d4d67a02a23dd20dfdb</sid><ORCID>0000-0001-9199-7368</ORCID><firstname>Fabio</firstname><surname>Caraffini</surname><name>Fabio Caraffini</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2022-09-27</date><deptcode>SCS</deptcode><abstract>The Covid-19 pandemic has led to the adoption of face masks in physical teaching spaces across the world. This has in-turn presented a number of challenges for practitioners in the face to face delivery of content and in effectively engaging learners in practical settings, where face coverings are an ongoing requirement. Being unable to identify the mouth movements of a speaker due to the lower portion of the face being obscured can lead to issues in clarity, attention, emotional recognition, and trust attribution, negatively affecting the learning experience. This is further exacerbated for those who require specialist support and those with impairments, particularly those centred around hearing. EmotiMask embeds an LED matrix within a face mask to replicate mouth movements and emotional state through speech detection and intelligent processing. By cycling through different LED configurations, the matrix can approximate speech in-progress, as well as various mouth patterns linked to distinct emotional states. An initial study placed EmotiMask within a HE practical session containing 10 students, with results suggesting a positive effect on clarity and emotional recognition over typical face masks. Further feedback noted that it was easier to identify the current speaker with EmotiMask, however speech amplification, additional led configurations, and improved portability are desired points of refinement. This study represents a step towards a ubiquitous solution for tackling some of the challenges presented when teaching in a pandemic or similar situations where face coverings are a requirement and has potential value in other sectors where such scenarios present themselves.</abstract><type>Conference Paper/Proceeding/Abstract</type><journal>ECE Official Conference Proceedings</journal><volume/><journalNumber/><paginationStart/><paginationEnd/><publisher>The International Academic Forum(IAFOR)</publisher><placeOfPublication/><isbnPrint/><isbnElectronic>2188-1162</isbnElectronic><issnPrint>2188-1162</issnPrint><issnElectronic/><keywords>Audio Processing, Covid-19, EmotiMask, Emotional Recognition, Face Masks, LED Matrix, Mouth Expressions, Speech Detection, Teaching &amp;amp; Learning</keywords><publishedDay>27</publishedDay><publishedMonth>9</publishedMonth><publishedYear>2022</publishedYear><publishedDate>2022-09-27</publishedDate><doi>10.22492/issn.2188-1162.2022.49</doi><url/><notes/><college>COLLEGE NANME</college><department>Computer Science</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>SCS</DepartmentCode><institution>Swansea University</institution><apcterm>Not Required</apcterm><funders/><projectreference/><lastEdited>2022-10-11T14:56:44.7222340</lastEdited><Created>2022-09-27T08:54:59.3773921</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Salim</firstname><surname>Hasshu</surname><order>1</order></author><author><firstname>Stuart</firstname><surname>O&#x2019;Connor</surname><order>2</order></author><author><firstname>Simon</firstname><surname>Colreavy-Donnelly</surname><order>3</order></author><author><firstname>Stefan</firstname><surname>Kuhn</surname><order>4</order></author><author><firstname>Fabio</firstname><surname>Caraffini</surname><orcid>0000-0001-9199-7368</orcid><order>5</order></author></authors><documents><document><filename>61352__25280__719315d32e104aa095e13f57feaa1bad.pdf</filename><originalFilename>61352.pdf</originalFilename><uploaded>2022-10-03T09:56:03.9504924</uploaded><type>Output</type><contentLength>4687172</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>Redistributed under a IAFOR User license.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language></document></documents><OutputDurs/></rfc1807>
spelling 2022-10-11T14:56:44.7222340 v2 61352 2022-09-27 EmotiMask: Mapping Mouth Movements to an LED Matrix for Improving Recognition When Teaching With a Face Mask d0b8d4e63d512d4d67a02a23dd20dfdb 0000-0001-9199-7368 Fabio Caraffini Fabio Caraffini true false 2022-09-27 SCS The Covid-19 pandemic has led to the adoption of face masks in physical teaching spaces across the world. This has in-turn presented a number of challenges for practitioners in the face to face delivery of content and in effectively engaging learners in practical settings, where face coverings are an ongoing requirement. Being unable to identify the mouth movements of a speaker due to the lower portion of the face being obscured can lead to issues in clarity, attention, emotional recognition, and trust attribution, negatively affecting the learning experience. This is further exacerbated for those who require specialist support and those with impairments, particularly those centred around hearing. EmotiMask embeds an LED matrix within a face mask to replicate mouth movements and emotional state through speech detection and intelligent processing. By cycling through different LED configurations, the matrix can approximate speech in-progress, as well as various mouth patterns linked to distinct emotional states. An initial study placed EmotiMask within a HE practical session containing 10 students, with results suggesting a positive effect on clarity and emotional recognition over typical face masks. Further feedback noted that it was easier to identify the current speaker with EmotiMask, however speech amplification, additional led configurations, and improved portability are desired points of refinement. This study represents a step towards a ubiquitous solution for tackling some of the challenges presented when teaching in a pandemic or similar situations where face coverings are a requirement and has potential value in other sectors where such scenarios present themselves. Conference Paper/Proceeding/Abstract ECE Official Conference Proceedings The International Academic Forum(IAFOR) 2188-1162 2188-1162 Audio Processing, Covid-19, EmotiMask, Emotional Recognition, Face Masks, LED Matrix, Mouth Expressions, Speech Detection, Teaching &amp; Learning 27 9 2022 2022-09-27 10.22492/issn.2188-1162.2022.49 COLLEGE NANME Computer Science COLLEGE CODE SCS Swansea University Not Required 2022-10-11T14:56:44.7222340 2022-09-27T08:54:59.3773921 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Salim Hasshu 1 Stuart O’Connor 2 Simon Colreavy-Donnelly 3 Stefan Kuhn 4 Fabio Caraffini 0000-0001-9199-7368 5 61352__25280__719315d32e104aa095e13f57feaa1bad.pdf 61352.pdf 2022-10-03T09:56:03.9504924 Output 4687172 application/pdf Version of Record true Redistributed under a IAFOR User license. true eng
title EmotiMask: Mapping Mouth Movements to an LED Matrix for Improving Recognition When Teaching With a Face Mask
spellingShingle EmotiMask: Mapping Mouth Movements to an LED Matrix for Improving Recognition When Teaching With a Face Mask
Fabio Caraffini
title_short EmotiMask: Mapping Mouth Movements to an LED Matrix for Improving Recognition When Teaching With a Face Mask
title_full EmotiMask: Mapping Mouth Movements to an LED Matrix for Improving Recognition When Teaching With a Face Mask
title_fullStr EmotiMask: Mapping Mouth Movements to an LED Matrix for Improving Recognition When Teaching With a Face Mask
title_full_unstemmed EmotiMask: Mapping Mouth Movements to an LED Matrix for Improving Recognition When Teaching With a Face Mask
title_sort EmotiMask: Mapping Mouth Movements to an LED Matrix for Improving Recognition When Teaching With a Face Mask
author_id_str_mv d0b8d4e63d512d4d67a02a23dd20dfdb
author_id_fullname_str_mv d0b8d4e63d512d4d67a02a23dd20dfdb_***_Fabio Caraffini
author Fabio Caraffini
author2 Salim Hasshu
Stuart O’Connor
Simon Colreavy-Donnelly
Stefan Kuhn
Fabio Caraffini
format Conference Paper/Proceeding/Abstract
container_title ECE Official Conference Proceedings
publishDate 2022
institution Swansea University
isbn 2188-1162
issn 2188-1162
doi_str_mv 10.22492/issn.2188-1162.2022.49
publisher The International Academic Forum(IAFOR)
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 1
active_str 0
description The Covid-19 pandemic has led to the adoption of face masks in physical teaching spaces across the world. This has in-turn presented a number of challenges for practitioners in the face to face delivery of content and in effectively engaging learners in practical settings, where face coverings are an ongoing requirement. Being unable to identify the mouth movements of a speaker due to the lower portion of the face being obscured can lead to issues in clarity, attention, emotional recognition, and trust attribution, negatively affecting the learning experience. This is further exacerbated for those who require specialist support and those with impairments, particularly those centred around hearing. EmotiMask embeds an LED matrix within a face mask to replicate mouth movements and emotional state through speech detection and intelligent processing. By cycling through different LED configurations, the matrix can approximate speech in-progress, as well as various mouth patterns linked to distinct emotional states. An initial study placed EmotiMask within a HE practical session containing 10 students, with results suggesting a positive effect on clarity and emotional recognition over typical face masks. Further feedback noted that it was easier to identify the current speaker with EmotiMask, however speech amplification, additional led configurations, and improved portability are desired points of refinement. This study represents a step towards a ubiquitous solution for tackling some of the challenges presented when teaching in a pandemic or similar situations where face coverings are a requirement and has potential value in other sectors where such scenarios present themselves.
published_date 2022-09-27T04:20:08Z
_version_ 1763754334127915008
score 11.017776

Similar Items