Conference Paper/Proceeding/Abstract 238 views
Speaker Motion Patterns during Self-repairs in Natural Dialogue
International Conference on Multimodal Interaction
Swansea University Author: Julian Hough
Full text not available from this repository: check for access using links below.
DOI (Published version): 10.1145/3536220.3563684
Abstract
An important milestone for any agent in interaction with humans on a regular basis is to achieve natural and efficient methods of communication. Such strategies should be derived on the hallmarks of human-human interaction. So far, the work in embodied conversational agents (ECAs) implementing such...
Published in: | International Conference on Multimodal Interaction |
---|---|
ISBN: | 978-1-4503-9389-8 |
Published: |
New York, NY, USA
ACM
2022
|
URI: | https://cronfa.swan.ac.uk/Record/cronfa64929 |
Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
first_indexed |
2023-11-07T21:33:05Z |
---|---|
last_indexed |
2023-11-07T21:33:05Z |
id |
cronfa64929 |
recordtype |
SURis |
fullrecord |
<?xml version="1.0" encoding="utf-8"?><rfc1807 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><bib-version>v2</bib-version><id>64929</id><entry>2023-11-07</entry><title>Speaker Motion Patterns during Self-repairs in Natural Dialogue</title><swanseaauthors><author><sid>082d773ae261d2bbf49434dd2608ab40</sid><firstname>Julian</firstname><surname>Hough</surname><name>Julian Hough</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2023-11-07</date><deptcode>SCS</deptcode><abstract>An important milestone for any agent in interaction with humans on a regular basis is to achieve natural and efficient methods of communication. Such strategies should be derived on the hallmarks of human-human interaction. So far, the work in embodied conversational agents (ECAs) implementing such signals has been predominantly through imitating human-like positive back-channels, such as nodding, rather than active interaction. The field of Conversation Analysis (CA) focusing on natural human dialogue suggests that people continuously collaborate on achieving mutual understanding by frequently repairing misunderstandings as they happen. Detecting repairs from speech in real-time is challenging, even with state-of-the-art Natural Language Processing (NLP) models. We present specific human motion patterns during key moments of interaction, namely self initiated self-repairs, which would help agents to recognise and collaboratively solve speaker trouble. The features we present in this paper are the pairwise joint distances of head and hands which are more discriminative than the positions themselves.</abstract><type>Conference Paper/Proceeding/Abstract</type><journal>International Conference on Multimodal Interaction</journal><volume/><journalNumber/><paginationStart/><paginationEnd/><publisher>ACM</publisher><placeOfPublication>New York, NY, USA</placeOfPublication><isbnPrint>978-1-4503-9389-8</isbnPrint><isbnElectronic/><issnPrint/><issnElectronic/><keywords>multimodal interaction, non-verbal communication, human motion analysis</keywords><publishedDay>7</publishedDay><publishedMonth>11</publishedMonth><publishedYear>2022</publishedYear><publishedDate>2022-11-07</publishedDate><doi>10.1145/3536220.3563684</doi><url/><notes/><college>COLLEGE NANME</college><department>Computer Science</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>SCS</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders/><projectreference/><lastEdited>2024-03-12T14:16:01.6918992</lastEdited><Created>2023-11-07T21:30:06.9466828</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Elif Ecem</firstname><surname>Ozkan</surname><orcid>0000-0002-9036-2149</orcid><order>1</order></author><author><firstname>Tom</firstname><surname>Gurion</surname><orcid>0000-0002-7245-4402</orcid><order>2</order></author><author><firstname>Julian</firstname><surname>Hough</surname><order>3</order></author><author><firstname>Patrick G.T.</firstname><surname>Healey</surname><orcid>0000-0003-3079-3374</orcid><order>4</order></author><author><firstname>Lorenzo</firstname><surname>Jamone</surname><orcid>0000-0002-1521-6168</orcid><order>5</order></author></authors><documents/><OutputDurs/></rfc1807> |
spelling |
v2 64929 2023-11-07 Speaker Motion Patterns during Self-repairs in Natural Dialogue 082d773ae261d2bbf49434dd2608ab40 Julian Hough Julian Hough true false 2023-11-07 SCS An important milestone for any agent in interaction with humans on a regular basis is to achieve natural and efficient methods of communication. Such strategies should be derived on the hallmarks of human-human interaction. So far, the work in embodied conversational agents (ECAs) implementing such signals has been predominantly through imitating human-like positive back-channels, such as nodding, rather than active interaction. The field of Conversation Analysis (CA) focusing on natural human dialogue suggests that people continuously collaborate on achieving mutual understanding by frequently repairing misunderstandings as they happen. Detecting repairs from speech in real-time is challenging, even with state-of-the-art Natural Language Processing (NLP) models. We present specific human motion patterns during key moments of interaction, namely self initiated self-repairs, which would help agents to recognise and collaboratively solve speaker trouble. The features we present in this paper are the pairwise joint distances of head and hands which are more discriminative than the positions themselves. Conference Paper/Proceeding/Abstract International Conference on Multimodal Interaction ACM New York, NY, USA 978-1-4503-9389-8 multimodal interaction, non-verbal communication, human motion analysis 7 11 2022 2022-11-07 10.1145/3536220.3563684 COLLEGE NANME Computer Science COLLEGE CODE SCS Swansea University Another institution paid the OA fee 2024-03-12T14:16:01.6918992 2023-11-07T21:30:06.9466828 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Elif Ecem Ozkan 0000-0002-9036-2149 1 Tom Gurion 0000-0002-7245-4402 2 Julian Hough 3 Patrick G.T. Healey 0000-0003-3079-3374 4 Lorenzo Jamone 0000-0002-1521-6168 5 |
title |
Speaker Motion Patterns during Self-repairs in Natural Dialogue |
spellingShingle |
Speaker Motion Patterns during Self-repairs in Natural Dialogue Julian Hough |
title_short |
Speaker Motion Patterns during Self-repairs in Natural Dialogue |
title_full |
Speaker Motion Patterns during Self-repairs in Natural Dialogue |
title_fullStr |
Speaker Motion Patterns during Self-repairs in Natural Dialogue |
title_full_unstemmed |
Speaker Motion Patterns during Self-repairs in Natural Dialogue |
title_sort |
Speaker Motion Patterns during Self-repairs in Natural Dialogue |
author_id_str_mv |
082d773ae261d2bbf49434dd2608ab40 |
author_id_fullname_str_mv |
082d773ae261d2bbf49434dd2608ab40_***_Julian Hough |
author |
Julian Hough |
author2 |
Elif Ecem Ozkan Tom Gurion Julian Hough Patrick G.T. Healey Lorenzo Jamone |
format |
Conference Paper/Proceeding/Abstract |
container_title |
International Conference on Multimodal Interaction |
publishDate |
2022 |
institution |
Swansea University |
isbn |
978-1-4503-9389-8 |
doi_str_mv |
10.1145/3536220.3563684 |
publisher |
ACM |
college_str |
Faculty of Science and Engineering |
hierarchytype |
|
hierarchy_top_id |
facultyofscienceandengineering |
hierarchy_top_title |
Faculty of Science and Engineering |
hierarchy_parent_id |
facultyofscienceandengineering |
hierarchy_parent_title |
Faculty of Science and Engineering |
department_str |
School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science |
document_store_str |
0 |
active_str |
0 |
description |
An important milestone for any agent in interaction with humans on a regular basis is to achieve natural and efficient methods of communication. Such strategies should be derived on the hallmarks of human-human interaction. So far, the work in embodied conversational agents (ECAs) implementing such signals has been predominantly through imitating human-like positive back-channels, such as nodding, rather than active interaction. The field of Conversation Analysis (CA) focusing on natural human dialogue suggests that people continuously collaborate on achieving mutual understanding by frequently repairing misunderstandings as they happen. Detecting repairs from speech in real-time is challenging, even with state-of-the-art Natural Language Processing (NLP) models. We present specific human motion patterns during key moments of interaction, namely self initiated self-repairs, which would help agents to recognise and collaboratively solve speaker trouble. The features we present in this paper are the pairwise joint distances of head and hands which are more discriminative than the positions themselves. |
published_date |
2022-11-07T14:15:58Z |
_version_ |
1793330206039080960 |
score |
11.031309 |