No Cover Image

Journal article 248 views 37 downloads

Solving Not Answering. Validation of Guidance for Writing Higher-Order Multiple-Choice Questions in Medical Science Education

Maria Xiromeriti Orcid Logo, Phil Newton Orcid Logo

Medical Science Educator, Volume: 34, Issue: 6, Pages: 1469 - 1477

Swansea University Author: Phil Newton Orcid Logo

  • 67367.VOR.pdf

    PDF | Version of Record

    © The Author(s) 2024. Open Access. This article is licensed under a Creative Commons Attribution 4.0 International License (CC-BY 4.0).

    Download (1.15MB)

Check full text

DOI (Published version): 10.1007/s40670-024-02140-7

Abstract

Problem-solving and higher-order learning are goals of higher education. It has been repeatedly suggested that multiple-choice questions (MCQs) can be used to test higher-order learning, although objective empirical evidence is lacking and MCQs are often criticised for assessing only lower-order, fa...

Full description

Published in: Medical Science Educator
ISSN: 2156-8650
Published: Springer Science and Business Media LLC 2024
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa67367
first_indexed 2024-08-13T10:45:55Z
last_indexed 2025-02-03T20:21:43Z
id cronfa67367
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2025-02-03T14:41:39.7290150</datestamp><bib-version>v2</bib-version><id>67367</id><entry>2024-08-13</entry><title>Solving Not Answering. Validation of Guidance for Writing Higher-Order Multiple-Choice Questions in Medical Science Education</title><swanseaauthors><author><sid>6e0a363d04c407371184d82f7a5bddc8</sid><ORCID>0000-0002-5272-7979</ORCID><firstname>Phil</firstname><surname>Newton</surname><name>Phil Newton</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2024-08-13</date><deptcode>MEDS</deptcode><abstract>Problem-solving and higher-order learning are goals of higher education. It has been repeatedly suggested that multiple-choice questions (MCQs) can be used to test higher-order learning, although objective empirical evidence is lacking and MCQs are often criticised for assessing only lower-order, factual, or &#x2018;rote&#x2019; learning. These challenges are compounded by a lack of agreement on what constitutes higher order learning: it is normally defined subjectively using heavily criticised frameworks such as such as Bloom&#x2019;s taxonomy. There is also a lack of agreement on how to write MCQs which assess higher order learning. Here we tested guidance for the creation of MCQs to assess higher-order learning, by evaluating the performance of students who were subject matter novices, vs experts. We found that questions written using the guidance were much harder to answer when students had no prior subject knowledge, whereas lower-order questions could be answered by simply searching online. These findings suggest that questions written using the guidance do indeed test higher-order learning, and such MCQs may be a valid alternative to other written assessment formats designed to test higher-order learning, such as essays, where reliability and cheating are a major concern.</abstract><type>Journal Article</type><journal>Medical Science Educator</journal><volume>34</volume><journalNumber>6</journalNumber><paginationStart>1469</paginationStart><paginationEnd>1477</paginationEnd><publisher>Springer Science and Business Media LLC</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic>2156-8650</issnElectronic><keywords>Digital education; Bloom&#x2019;s taxonomy; Assessment; Higher-order learning</keywords><publishedDay>1</publishedDay><publishedMonth>12</publishedMonth><publishedYear>2024</publishedYear><publishedDate>2024-12-01</publishedDate><doi>10.1007/s40670-024-02140-7</doi><url/><notes/><college>COLLEGE NANME</college><department>Medical School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MEDS</DepartmentCode><institution>Swansea University</institution><apcterm>SU Library paid the OA fee (TA Institutional Deal)</apcterm><funders>Swansea University</funders><projectreference/><lastEdited>2025-02-03T14:41:39.7290150</lastEdited><Created>2024-08-13T11:44:02.8796661</Created><path><level id="1">Faculty of Medicine, Health and Life Sciences</level><level id="2">Swansea University Medical School - Medicine</level></path><authors><author><firstname>Maria</firstname><surname>Xiromeriti</surname><orcid>0000-0002-2975-184x</orcid><order>1</order></author><author><firstname>Phil</firstname><surname>Newton</surname><orcid>0000-0002-5272-7979</orcid><order>2</order></author></authors><documents><document><filename>67367__33478__084d865ae7684d5586f5f652e127ea22.pdf</filename><originalFilename>67367.VOR.pdf</originalFilename><uploaded>2025-02-03T14:39:33.4578975</uploaded><type>Output</type><contentLength>1204289</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>&#xA9; The Author(s) 2024. Open Access. This article is licensed under a Creative Commons Attribution 4.0 International License (CC-BY 4.0).</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>http://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling 2025-02-03T14:41:39.7290150 v2 67367 2024-08-13 Solving Not Answering. Validation of Guidance for Writing Higher-Order Multiple-Choice Questions in Medical Science Education 6e0a363d04c407371184d82f7a5bddc8 0000-0002-5272-7979 Phil Newton Phil Newton true false 2024-08-13 MEDS Problem-solving and higher-order learning are goals of higher education. It has been repeatedly suggested that multiple-choice questions (MCQs) can be used to test higher-order learning, although objective empirical evidence is lacking and MCQs are often criticised for assessing only lower-order, factual, or ‘rote’ learning. These challenges are compounded by a lack of agreement on what constitutes higher order learning: it is normally defined subjectively using heavily criticised frameworks such as such as Bloom’s taxonomy. There is also a lack of agreement on how to write MCQs which assess higher order learning. Here we tested guidance for the creation of MCQs to assess higher-order learning, by evaluating the performance of students who were subject matter novices, vs experts. We found that questions written using the guidance were much harder to answer when students had no prior subject knowledge, whereas lower-order questions could be answered by simply searching online. These findings suggest that questions written using the guidance do indeed test higher-order learning, and such MCQs may be a valid alternative to other written assessment formats designed to test higher-order learning, such as essays, where reliability and cheating are a major concern. Journal Article Medical Science Educator 34 6 1469 1477 Springer Science and Business Media LLC 2156-8650 Digital education; Bloom’s taxonomy; Assessment; Higher-order learning 1 12 2024 2024-12-01 10.1007/s40670-024-02140-7 COLLEGE NANME Medical School COLLEGE CODE MEDS Swansea University SU Library paid the OA fee (TA Institutional Deal) Swansea University 2025-02-03T14:41:39.7290150 2024-08-13T11:44:02.8796661 Faculty of Medicine, Health and Life Sciences Swansea University Medical School - Medicine Maria Xiromeriti 0000-0002-2975-184x 1 Phil Newton 0000-0002-5272-7979 2 67367__33478__084d865ae7684d5586f5f652e127ea22.pdf 67367.VOR.pdf 2025-02-03T14:39:33.4578975 Output 1204289 application/pdf Version of Record true © The Author(s) 2024. Open Access. This article is licensed under a Creative Commons Attribution 4.0 International License (CC-BY 4.0). true eng http://creativecommons.org/licenses/by/4.0/
title Solving Not Answering. Validation of Guidance for Writing Higher-Order Multiple-Choice Questions in Medical Science Education
spellingShingle Solving Not Answering. Validation of Guidance for Writing Higher-Order Multiple-Choice Questions in Medical Science Education
Phil Newton
title_short Solving Not Answering. Validation of Guidance for Writing Higher-Order Multiple-Choice Questions in Medical Science Education
title_full Solving Not Answering. Validation of Guidance for Writing Higher-Order Multiple-Choice Questions in Medical Science Education
title_fullStr Solving Not Answering. Validation of Guidance for Writing Higher-Order Multiple-Choice Questions in Medical Science Education
title_full_unstemmed Solving Not Answering. Validation of Guidance for Writing Higher-Order Multiple-Choice Questions in Medical Science Education
title_sort Solving Not Answering. Validation of Guidance for Writing Higher-Order Multiple-Choice Questions in Medical Science Education
author_id_str_mv 6e0a363d04c407371184d82f7a5bddc8
author_id_fullname_str_mv 6e0a363d04c407371184d82f7a5bddc8_***_Phil Newton
author Phil Newton
author2 Maria Xiromeriti
Phil Newton
format Journal article
container_title Medical Science Educator
container_volume 34
container_issue 6
container_start_page 1469
publishDate 2024
institution Swansea University
issn 2156-8650
doi_str_mv 10.1007/s40670-024-02140-7
publisher Springer Science and Business Media LLC
college_str Faculty of Medicine, Health and Life Sciences
hierarchytype
hierarchy_top_id facultyofmedicinehealthandlifesciences
hierarchy_top_title Faculty of Medicine, Health and Life Sciences
hierarchy_parent_id facultyofmedicinehealthandlifesciences
hierarchy_parent_title Faculty of Medicine, Health and Life Sciences
department_str Swansea University Medical School - Medicine{{{_:::_}}}Faculty of Medicine, Health and Life Sciences{{{_:::_}}}Swansea University Medical School - Medicine
document_store_str 1
active_str 0
description Problem-solving and higher-order learning are goals of higher education. It has been repeatedly suggested that multiple-choice questions (MCQs) can be used to test higher-order learning, although objective empirical evidence is lacking and MCQs are often criticised for assessing only lower-order, factual, or ‘rote’ learning. These challenges are compounded by a lack of agreement on what constitutes higher order learning: it is normally defined subjectively using heavily criticised frameworks such as such as Bloom’s taxonomy. There is also a lack of agreement on how to write MCQs which assess higher order learning. Here we tested guidance for the creation of MCQs to assess higher-order learning, by evaluating the performance of students who were subject matter novices, vs experts. We found that questions written using the guidance were much harder to answer when students had no prior subject knowledge, whereas lower-order questions could be answered by simply searching online. These findings suggest that questions written using the guidance do indeed test higher-order learning, and such MCQs may be a valid alternative to other written assessment formats designed to test higher-order learning, such as essays, where reliability and cheating are a major concern.
published_date 2024-12-01T08:15:09Z
_version_ 1826375398164791296
score 11.055436

Similar Items