No Cover Image

Journal article 15 views

Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors

Saeed Ur Rehman Orcid Logo, Anwar Ali Orcid Logo, Adil Mehmood Khan, Cynthia Okpala

Algorithms, Volume: 17, Issue: 12, Start page: 556

Swansea University Author: Anwar Ali Orcid Logo

Full text not available from this repository: check for access using links below.

Check full text

DOI (Published version): 10.3390/a17120556

Abstract

With the increasing availability of wearable devices for data collection, studies in human activity recognition have gained significant popularity. These studies report high accuracies on k-fold cross validation, which is not reflective of their generalization performance but is a result of the inap...

Full description

Published in: Algorithms
ISSN: 1999-4893
Published: MDPI AG 2024
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa68579
first_indexed 2025-01-09T20:33:52Z
last_indexed 2025-01-09T20:33:52Z
id cronfa68579
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2024-12-17T13:28:27.6241088</datestamp><bib-version>v2</bib-version><id>68579</id><entry>2024-12-17</entry><title>Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors</title><swanseaauthors><author><sid>f206105e1de57bebba0fd04fe9870779</sid><ORCID>0000-0001-7366-9002</ORCID><firstname>Anwar</firstname><surname>Ali</surname><name>Anwar Ali</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2024-12-17</date><deptcode>ACEM</deptcode><abstract>With the increasing availability of wearable devices for data collection, studies in human activity recognition have gained significant popularity. These studies report high accuracies on k-fold cross validation, which is not reflective of their generalization performance but is a result of the inappropriate split of testing and training datasets, causing these models to evaluate the same subjects that they were trained on, making them subject-dependent. This study comparatively discusses this validation approach with a universal approach, Leave-One-Subject-Out (LOSO) cross-validation which is not subject-dependent and ensures that an entirely new subject is used for evaluation in each fold, validated on four different machine learning models trained on windowed data and select hand-crafted features. The random forest model, with the highest accuracy of 76% when evaluated on LOSO, achieved an accuracy of 89% on k-fold cross-validation, demonstrating data leakage. Additionally, this experiment underscores the significance of hand-crafted features by contrasting their accuracy with that of raw sensor models. The feature models demonstrate a remarkable 30% higher accuracy, underscoring the importance of feature engineering in enhancing the robustness and precision of HAR systems.</abstract><type>Journal Article</type><journal>Algorithms</journal><volume>17</volume><journalNumber>12</journalNumber><paginationStart>556</paginationStart><paginationEnd/><publisher>MDPI AG</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic>1999-4893</issnElectronic><keywords>Machine learning; LOSO; human activity recognition</keywords><publishedDay>5</publishedDay><publishedMonth>12</publishedMonth><publishedYear>2024</publishedYear><publishedDate>2024-12-05</publishedDate><doi>10.3390/a17120556</doi><url/><notes/><college>COLLEGE NANME</college><department>Aerospace, Civil, Electrical, and Mechanical Engineering</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>ACEM</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders>This research received no external funding.</funders><projectreference/><lastEdited>2024-12-17T13:28:27.6241088</lastEdited><Created>2024-12-17T13:21:11.5144374</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Electronic and Electrical Engineering</level></path><authors><author><firstname>Saeed Ur</firstname><surname>Rehman</surname><orcid>0009-0009-4566-7144</orcid><order>1</order></author><author><firstname>Anwar</firstname><surname>Ali</surname><orcid>0000-0001-7366-9002</orcid><order>2</order></author><author><firstname>Adil Mehmood</firstname><surname>Khan</surname><order>3</order></author><author><firstname>Cynthia</firstname><surname>Okpala</surname><order>4</order></author></authors><documents/><OutputDurs/></rfc1807>
spelling 2024-12-17T13:28:27.6241088 v2 68579 2024-12-17 Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors f206105e1de57bebba0fd04fe9870779 0000-0001-7366-9002 Anwar Ali Anwar Ali true false 2024-12-17 ACEM With the increasing availability of wearable devices for data collection, studies in human activity recognition have gained significant popularity. These studies report high accuracies on k-fold cross validation, which is not reflective of their generalization performance but is a result of the inappropriate split of testing and training datasets, causing these models to evaluate the same subjects that they were trained on, making them subject-dependent. This study comparatively discusses this validation approach with a universal approach, Leave-One-Subject-Out (LOSO) cross-validation which is not subject-dependent and ensures that an entirely new subject is used for evaluation in each fold, validated on four different machine learning models trained on windowed data and select hand-crafted features. The random forest model, with the highest accuracy of 76% when evaluated on LOSO, achieved an accuracy of 89% on k-fold cross-validation, demonstrating data leakage. Additionally, this experiment underscores the significance of hand-crafted features by contrasting their accuracy with that of raw sensor models. The feature models demonstrate a remarkable 30% higher accuracy, underscoring the importance of feature engineering in enhancing the robustness and precision of HAR systems. Journal Article Algorithms 17 12 556 MDPI AG 1999-4893 Machine learning; LOSO; human activity recognition 5 12 2024 2024-12-05 10.3390/a17120556 COLLEGE NANME Aerospace, Civil, Electrical, and Mechanical Engineering COLLEGE CODE ACEM Swansea University Another institution paid the OA fee This research received no external funding. 2024-12-17T13:28:27.6241088 2024-12-17T13:21:11.5144374 Faculty of Science and Engineering School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Electronic and Electrical Engineering Saeed Ur Rehman 0009-0009-4566-7144 1 Anwar Ali 0000-0001-7366-9002 2 Adil Mehmood Khan 3 Cynthia Okpala 4
title Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
spellingShingle Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
Anwar Ali
title_short Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
title_full Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
title_fullStr Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
title_full_unstemmed Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
title_sort Human Activity Recognition: A Comparative Study of Validation Methods and Impact of Feature Extraction in Wearable Sensors
author_id_str_mv f206105e1de57bebba0fd04fe9870779
author_id_fullname_str_mv f206105e1de57bebba0fd04fe9870779_***_Anwar Ali
author Anwar Ali
author2 Saeed Ur Rehman
Anwar Ali
Adil Mehmood Khan
Cynthia Okpala
format Journal article
container_title Algorithms
container_volume 17
container_issue 12
container_start_page 556
publishDate 2024
institution Swansea University
issn 1999-4893
doi_str_mv 10.3390/a17120556
publisher MDPI AG
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Electronic and Electrical Engineering{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Aerospace, Civil, Electrical, General and Mechanical Engineering - Electronic and Electrical Engineering
document_store_str 0
active_str 0
description With the increasing availability of wearable devices for data collection, studies in human activity recognition have gained significant popularity. These studies report high accuracies on k-fold cross validation, which is not reflective of their generalization performance but is a result of the inappropriate split of testing and training datasets, causing these models to evaluate the same subjects that they were trained on, making them subject-dependent. This study comparatively discusses this validation approach with a universal approach, Leave-One-Subject-Out (LOSO) cross-validation which is not subject-dependent and ensures that an entirely new subject is used for evaluation in each fold, validated on four different machine learning models trained on windowed data and select hand-crafted features. The random forest model, with the highest accuracy of 76% when evaluated on LOSO, achieved an accuracy of 89% on k-fold cross-validation, demonstrating data leakage. Additionally, this experiment underscores the significance of hand-crafted features by contrasting their accuracy with that of raw sensor models. The feature models demonstrate a remarkable 30% higher accuracy, underscoring the importance of feature engineering in enhancing the robustness and precision of HAR systems.
published_date 2024-12-05T20:36:54Z
_version_ 1821348635542028288
score 11.04748

Similar Items