No Cover Image

Conference Paper/Proceeding/Abstract 243 views 65 downloads

Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022

Kavous Salehzadeh Niksirat Orcid Logo, Lahari Goswami Orcid Logo, Pooja S. B. Rao Orcid Logo, James Tyler Orcid Logo, Alessandro Silacci Orcid Logo, Sadiq Aliyu Orcid Logo, Annika Aebli Orcid Logo, Chat Wacharamanotham Orcid Logo, Mauro Cherubini Orcid Logo

Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems

Swansea University Author: Chat Wacharamanotham Orcid Logo

  • 63283.pdf

    PDF | Version of Record

    This work is licensed under a Creative Commons Attribution International 4.0 License.

    Download (1.38MB)

DOI (Published version): 10.1145/3544548.3580848

Abstract

In recent years, various initiatives from within and outside the HCI field have encouraged researchers to improve research ethics, openness, and transparency in their empirical research. We quantify how the CHI literature might have changed in these three aspects by analyzing samples of 118 CHI 2017...

Full description

Published in: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
ISBN: 978-1-4503-9421-5
Published: New York, NY, USA ACM 2023
Online Access: http://dx.doi.org/10.1145/3544548.3580848
URI: https://cronfa.swan.ac.uk/Record/cronfa63283
Tags: Add Tag
No Tags, Be the first to tag this record!
first_indexed 2023-04-29T06:11:37Z
last_indexed 2023-04-29T06:11:37Z
id cronfa63283
recordtype SURis
fullrecord <?xml version="1.0" encoding="utf-8"?><rfc1807 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><bib-version>v2</bib-version><id>63283</id><entry>2023-04-29</entry><title>Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022</title><swanseaauthors><author><sid>5310be7eb485ebc96c9671f5a45d6f62</sid><ORCID>0000-0003-4831-2516</ORCID><firstname>Chat</firstname><surname>Wacharamanotham</surname><name>Chat Wacharamanotham</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2023-04-29</date><deptcode>SCS</deptcode><abstract>In recent years, various initiatives from within and outside the HCI field have encouraged researchers to improve research ethics, openness, and transparency in their empirical research. We quantify how the CHI literature might have changed in these three aspects by analyzing samples of 118 CHI 2017 and 127 CHI 2022 papers—randomly drawn and stratified across conference sessions. We operationalized research ethics, openness, and transparency into 45 criteria and manually annotated the sampled papers. The results show that the CHI 2022 sample was better in 18 criteria, but in the rest of the criteria, it has no improvement. The most noticeable improvements were related to research transparency (10 out of 17 criteria). We also explored the possibility of assisting the verification process by developing a proof-of-concept screening system. We tested this tool with eight criteria. Six of them achieved high accuracy and F1 score. We discuss the implications for future research practices and education.</abstract><type>Conference Paper/Proceeding/Abstract</type><journal>Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems</journal><volume/><journalNumber/><paginationStart/><paginationEnd/><publisher>ACM</publisher><placeOfPublication>New York, NY, USA</placeOfPublication><isbnPrint>978-1-4503-9421-5</isbnPrint><isbnElectronic/><issnPrint/><issnElectronic/><keywords/><publishedDay>19</publishedDay><publishedMonth>4</publishedMonth><publishedYear>2023</publishedYear><publishedDate>2023-04-19</publishedDate><doi>10.1145/3544548.3580848</doi><url>http://dx.doi.org/10.1145/3544548.3580848</url><notes/><college>COLLEGE NANME</college><department>Computer Science</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>SCS</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders/><projectreference/><lastEdited>2023-11-20T14:15:24.7499697</lastEdited><Created>2023-04-29T07:01:41.9616893</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Kavous Salehzadeh</firstname><surname>Niksirat</surname><orcid>0000-0003-4438-3544</orcid><order>1</order></author><author><firstname>Lahari</firstname><surname>Goswami</surname><orcid>0000-0002-8975-5885</orcid><order>2</order></author><author><firstname>Pooja S. B.</firstname><surname>Rao</surname><orcid>0000-0003-3346-2749</orcid><order>3</order></author><author><firstname>James</firstname><surname>Tyler</surname><orcid>0000-0001-8102-962x</orcid><order>4</order></author><author><firstname>Alessandro</firstname><surname>Silacci</surname><orcid>0000-0001-8121-3013</orcid><order>5</order></author><author><firstname>Sadiq</firstname><surname>Aliyu</surname><orcid>0000-0002-5090-920x</orcid><order>6</order></author><author><firstname>Annika</firstname><surname>Aebli</surname><orcid>0000-0002-8209-9950</orcid><order>7</order></author><author><firstname>Chat</firstname><surname>Wacharamanotham</surname><orcid>0000-0003-4831-2516</orcid><order>8</order></author><author><firstname>Mauro</firstname><surname>Cherubini</surname><orcid>0000-0002-1860-6110</orcid><order>9</order></author></authors><documents><document><filename>63283__27579__62aada8bca9a4a94a9bd6eb2375781dc.pdf</filename><originalFilename>63283.pdf</originalFilename><uploaded>2023-05-23T14:34:11.9744949</uploaded><type>Output</type><contentLength>1445786</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>This work is licensed under a Creative Commons Attribution International 4.0 License.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling v2 63283 2023-04-29 Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022 5310be7eb485ebc96c9671f5a45d6f62 0000-0003-4831-2516 Chat Wacharamanotham Chat Wacharamanotham true false 2023-04-29 SCS In recent years, various initiatives from within and outside the HCI field have encouraged researchers to improve research ethics, openness, and transparency in their empirical research. We quantify how the CHI literature might have changed in these three aspects by analyzing samples of 118 CHI 2017 and 127 CHI 2022 papers—randomly drawn and stratified across conference sessions. We operationalized research ethics, openness, and transparency into 45 criteria and manually annotated the sampled papers. The results show that the CHI 2022 sample was better in 18 criteria, but in the rest of the criteria, it has no improvement. The most noticeable improvements were related to research transparency (10 out of 17 criteria). We also explored the possibility of assisting the verification process by developing a proof-of-concept screening system. We tested this tool with eight criteria. Six of them achieved high accuracy and F1 score. We discuss the implications for future research practices and education. Conference Paper/Proceeding/Abstract Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems ACM New York, NY, USA 978-1-4503-9421-5 19 4 2023 2023-04-19 10.1145/3544548.3580848 http://dx.doi.org/10.1145/3544548.3580848 COLLEGE NANME Computer Science COLLEGE CODE SCS Swansea University Another institution paid the OA fee 2023-11-20T14:15:24.7499697 2023-04-29T07:01:41.9616893 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Kavous Salehzadeh Niksirat 0000-0003-4438-3544 1 Lahari Goswami 0000-0002-8975-5885 2 Pooja S. B. Rao 0000-0003-3346-2749 3 James Tyler 0000-0001-8102-962x 4 Alessandro Silacci 0000-0001-8121-3013 5 Sadiq Aliyu 0000-0002-5090-920x 6 Annika Aebli 0000-0002-8209-9950 7 Chat Wacharamanotham 0000-0003-4831-2516 8 Mauro Cherubini 0000-0002-1860-6110 9 63283__27579__62aada8bca9a4a94a9bd6eb2375781dc.pdf 63283.pdf 2023-05-23T14:34:11.9744949 Output 1445786 application/pdf Version of Record true This work is licensed under a Creative Commons Attribution International 4.0 License. true eng https://creativecommons.org/licenses/by/4.0/
title Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022
spellingShingle Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022
Chat Wacharamanotham
title_short Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022
title_full Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022
title_fullStr Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022
title_full_unstemmed Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022
title_sort Changes in Research Ethics, Openness, and Transparency in Empirical Studies between CHI 2017 and CHI 2022
author_id_str_mv 5310be7eb485ebc96c9671f5a45d6f62
author_id_fullname_str_mv 5310be7eb485ebc96c9671f5a45d6f62_***_Chat Wacharamanotham
author Chat Wacharamanotham
author2 Kavous Salehzadeh Niksirat
Lahari Goswami
Pooja S. B. Rao
James Tyler
Alessandro Silacci
Sadiq Aliyu
Annika Aebli
Chat Wacharamanotham
Mauro Cherubini
format Conference Paper/Proceeding/Abstract
container_title Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems
publishDate 2023
institution Swansea University
isbn 978-1-4503-9421-5
doi_str_mv 10.1145/3544548.3580848
publisher ACM
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
url http://dx.doi.org/10.1145/3544548.3580848
document_store_str 1
active_str 0
description In recent years, various initiatives from within and outside the HCI field have encouraged researchers to improve research ethics, openness, and transparency in their empirical research. We quantify how the CHI literature might have changed in these three aspects by analyzing samples of 118 CHI 2017 and 127 CHI 2022 papers—randomly drawn and stratified across conference sessions. We operationalized research ethics, openness, and transparency into 45 criteria and manually annotated the sampled papers. The results show that the CHI 2022 sample was better in 18 criteria, but in the rest of the criteria, it has no improvement. The most noticeable improvements were related to research transparency (10 out of 17 criteria). We also explored the possibility of assisting the verification process by developing a proof-of-concept screening system. We tested this tool with eight criteria. Six of them achieved high accuracy and F1 score. We discuss the implications for future research practices and education.
published_date 2023-04-19T14:15:25Z
_version_ 1783092714747723776
score 11.013171

Similar Items