No Cover Image

Journal article 554 views 41 downloads

Nudge and bias in subjective ratings? The role of icon sets in determining ratings of icon characteristics

Siné McDougall Orcid Logo, Irene Reppa Orcid Logo, Jacqui Taylor

Behavior Research Methods, Volume: 55, Issue: 7, Pages: 3513 - 3530

Swansea University Author: Irene Reppa Orcid Logo

  • 61396_VoR.pdf

    PDF | Version of Record

    © The Author(s) 2022. This article is licensed under a Creative Commons Attribution 4.0 International License

    Download (1.27MB)

Check full text

DOI (Published version): 10.3758/s13428-022-01973-7

Abstract

Subjective ratings have been central to the evaluation of icon characteristics. The current study examined biases in ratings in relation to the context in which icons are presented. Context was manipulated between participants, with some groups rating icon sets with limited variability, and others r...

Full description

Published in: Behavior Research Methods
ISSN: 1554-3528
Published: Springer Science and Business Media LLC 2023
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa61396
Tags: Add Tag
No Tags, Be the first to tag this record!
first_indexed 2022-10-01T11:18:41Z
last_indexed 2023-01-13T19:22:08Z
id cronfa61396
recordtype SURis
fullrecord <?xml version="1.0" encoding="utf-8"?><rfc1807 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><bib-version>v2</bib-version><id>61396</id><entry>2022-10-01</entry><title>Nudge and bias in subjective ratings? The role of icon sets in determining ratings of icon characteristics</title><swanseaauthors><author><sid>7824f127c16603af4e08530245a62400</sid><ORCID>0000-0002-2853-2311</ORCID><firstname>Irene</firstname><surname>Reppa</surname><name>Irene Reppa</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2022-10-01</date><deptcode>PSYS</deptcode><abstract>Subjective ratings have been central to the evaluation of icon characteristics. The current study examined biases in ratings in relation to the context in which icons are presented. Context was manipulated between participants, with some groups rating icon sets with limited variability, and others rating icon sets with wide variability. It was predicted that the context created by the icon set would influence participants’ ratings; when the range of icons was limited, this would create bias given participants’ expectation that a full range of icon values was being presented. Six key icon characteristics were rated, which were visual (visual complexity, appeal), affective (valence, feelings), and semantic (concreteness, semantic distance). Some icon characteristics were susceptible to rating bias while others were not. Where subjective judgements were being made of visual icon characteristics (appeal/complexity) and highly concrete icons which were very pictorial, there was clear evidence of substantial bias in ratings. The same susceptibility to bias was not evident when ratings relied solely on learned semantic associations or were associated with the emotional attributions made to icons. The dynamic nature of the ratings bias was demonstrated when the rating context was changed without participants’ knowledge. When participants rated further blocks of icons providing a different range of the to-be-rated characteristic, this resulted in rapid and dramatic changes in rating behaviour. These findings demonstrate the need for representative sampling of icon characteristics to avoid ratings bias. Practically, this is important when determining the usability of newly designed icon sets in order to avoid over-valuing or under-valuing of key characteristics.</abstract><type>Journal Article</type><journal>Behavior Research Methods</journal><volume>55</volume><journalNumber>7</journalNumber><paginationStart>3513</paginationStart><paginationEnd>3530</paginationEnd><publisher>Springer Science and Business Media LLC</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic>1554-3528</issnElectronic><keywords>Ratings bias; Icon ratings; Icon design and usability; Concreteness; Complexity; Visual appeal; Afect</keywords><publishedDay>1</publishedDay><publishedMonth>10</publishedMonth><publishedYear>2023</publishedYear><publishedDate>2023-10-01</publishedDate><doi>10.3758/s13428-022-01973-7</doi><url/><notes/><college>COLLEGE NANME</college><department>Psychology School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>PSYS</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders/><projectreference/><lastEdited>2024-06-05T16:45:01.1369699</lastEdited><Created>2022-10-01T12:17:08.3238728</Created><path><level id="1">Faculty of Medicine, Health and Life Sciences</level><level id="2">School of Psychology</level></path><authors><author><firstname>Siné</firstname><surname>McDougall</surname><orcid>0000-0001-6595-1622</orcid><order>1</order></author><author><firstname>Irene</firstname><surname>Reppa</surname><orcid>0000-0002-2853-2311</orcid><order>2</order></author><author><firstname>Jacqui</firstname><surname>Taylor</surname><order>3</order></author></authors><documents><document><filename>61396__25612__2d5cb5bc4b234adb824bfeb51c7086c4.pdf</filename><originalFilename>61396_VoR.pdf</originalFilename><uploaded>2022-10-28T17:14:29.6871249</uploaded><type>Output</type><contentLength>1328360</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>© The Author(s) 2022. This article is licensed under a Creative Commons Attribution 4.0 International License</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>http://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling v2 61396 2022-10-01 Nudge and bias in subjective ratings? The role of icon sets in determining ratings of icon characteristics 7824f127c16603af4e08530245a62400 0000-0002-2853-2311 Irene Reppa Irene Reppa true false 2022-10-01 PSYS Subjective ratings have been central to the evaluation of icon characteristics. The current study examined biases in ratings in relation to the context in which icons are presented. Context was manipulated between participants, with some groups rating icon sets with limited variability, and others rating icon sets with wide variability. It was predicted that the context created by the icon set would influence participants’ ratings; when the range of icons was limited, this would create bias given participants’ expectation that a full range of icon values was being presented. Six key icon characteristics were rated, which were visual (visual complexity, appeal), affective (valence, feelings), and semantic (concreteness, semantic distance). Some icon characteristics were susceptible to rating bias while others were not. Where subjective judgements were being made of visual icon characteristics (appeal/complexity) and highly concrete icons which were very pictorial, there was clear evidence of substantial bias in ratings. The same susceptibility to bias was not evident when ratings relied solely on learned semantic associations or were associated with the emotional attributions made to icons. The dynamic nature of the ratings bias was demonstrated when the rating context was changed without participants’ knowledge. When participants rated further blocks of icons providing a different range of the to-be-rated characteristic, this resulted in rapid and dramatic changes in rating behaviour. These findings demonstrate the need for representative sampling of icon characteristics to avoid ratings bias. Practically, this is important when determining the usability of newly designed icon sets in order to avoid over-valuing or under-valuing of key characteristics. Journal Article Behavior Research Methods 55 7 3513 3530 Springer Science and Business Media LLC 1554-3528 Ratings bias; Icon ratings; Icon design and usability; Concreteness; Complexity; Visual appeal; Afect 1 10 2023 2023-10-01 10.3758/s13428-022-01973-7 COLLEGE NANME Psychology School COLLEGE CODE PSYS Swansea University Another institution paid the OA fee 2024-06-05T16:45:01.1369699 2022-10-01T12:17:08.3238728 Faculty of Medicine, Health and Life Sciences School of Psychology Siné McDougall 0000-0001-6595-1622 1 Irene Reppa 0000-0002-2853-2311 2 Jacqui Taylor 3 61396__25612__2d5cb5bc4b234adb824bfeb51c7086c4.pdf 61396_VoR.pdf 2022-10-28T17:14:29.6871249 Output 1328360 application/pdf Version of Record true © The Author(s) 2022. This article is licensed under a Creative Commons Attribution 4.0 International License true eng http://creativecommons.org/licenses/by/4.0/
title Nudge and bias in subjective ratings? The role of icon sets in determining ratings of icon characteristics
spellingShingle Nudge and bias in subjective ratings? The role of icon sets in determining ratings of icon characteristics
Irene Reppa
title_short Nudge and bias in subjective ratings? The role of icon sets in determining ratings of icon characteristics
title_full Nudge and bias in subjective ratings? The role of icon sets in determining ratings of icon characteristics
title_fullStr Nudge and bias in subjective ratings? The role of icon sets in determining ratings of icon characteristics
title_full_unstemmed Nudge and bias in subjective ratings? The role of icon sets in determining ratings of icon characteristics
title_sort Nudge and bias in subjective ratings? The role of icon sets in determining ratings of icon characteristics
author_id_str_mv 7824f127c16603af4e08530245a62400
author_id_fullname_str_mv 7824f127c16603af4e08530245a62400_***_Irene Reppa
author Irene Reppa
author2 Siné McDougall
Irene Reppa
Jacqui Taylor
format Journal article
container_title Behavior Research Methods
container_volume 55
container_issue 7
container_start_page 3513
publishDate 2023
institution Swansea University
issn 1554-3528
doi_str_mv 10.3758/s13428-022-01973-7
publisher Springer Science and Business Media LLC
college_str Faculty of Medicine, Health and Life Sciences
hierarchytype
hierarchy_top_id facultyofmedicinehealthandlifesciences
hierarchy_top_title Faculty of Medicine, Health and Life Sciences
hierarchy_parent_id facultyofmedicinehealthandlifesciences
hierarchy_parent_title Faculty of Medicine, Health and Life Sciences
department_str School of Psychology{{{_:::_}}}Faculty of Medicine, Health and Life Sciences{{{_:::_}}}School of Psychology
document_store_str 1
active_str 0
description Subjective ratings have been central to the evaluation of icon characteristics. The current study examined biases in ratings in relation to the context in which icons are presented. Context was manipulated between participants, with some groups rating icon sets with limited variability, and others rating icon sets with wide variability. It was predicted that the context created by the icon set would influence participants’ ratings; when the range of icons was limited, this would create bias given participants’ expectation that a full range of icon values was being presented. Six key icon characteristics were rated, which were visual (visual complexity, appeal), affective (valence, feelings), and semantic (concreteness, semantic distance). Some icon characteristics were susceptible to rating bias while others were not. Where subjective judgements were being made of visual icon characteristics (appeal/complexity) and highly concrete icons which were very pictorial, there was clear evidence of substantial bias in ratings. The same susceptibility to bias was not evident when ratings relied solely on learned semantic associations or were associated with the emotional attributions made to icons. The dynamic nature of the ratings bias was demonstrated when the rating context was changed without participants’ knowledge. When participants rated further blocks of icons providing a different range of the to-be-rated characteristic, this resulted in rapid and dramatic changes in rating behaviour. These findings demonstrate the need for representative sampling of icon characteristics to avoid ratings bias. Practically, this is important when determining the usability of newly designed icon sets in order to avoid over-valuing or under-valuing of key characteristics.
published_date 2023-10-01T16:45:00Z
_version_ 1801036550000082944
score 11.013731

Similar Items