No Cover Image

Journal article 272 views 84 downloads

FedBoosting: Federated learning with gradient protected boosting for text recognition

Hans Ren, Jingjing Deng Orcid Logo, Xianghua Xie Orcid Logo, Xiaoke Ma, Yichuan Wang

Neurocomputing, Volume: 569

Swansea University Authors: Hans Ren, Xianghua Xie Orcid Logo

Check full text

DOI (Published version): 10.1016/j.neucom.2023.127126

Abstract

Conventional machine learning methodologies require the centralization of data for model training, which may be infeasible in situations where data sharing limitations are imposed due to concerns such as privacy and gradient protection. The Federated Learning (FL) framework enables the collaborative...

Full description

Published in: Neurocomputing
ISSN: 0925-2312
Published: Elsevier BV 2024
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa65267
first_indexed 2023-12-12T09:53:34Z
last_indexed 2024-11-25T14:15:43Z
id cronfa65267
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2024-07-11T15:00:56.3406771</datestamp><bib-version>v2</bib-version><id>65267</id><entry>2023-12-12</entry><title>FedBoosting: Federated learning with gradient protected boosting for text recognition</title><swanseaauthors><author><sid>9e043b899a2b786672a28ed4f864ffcc</sid><firstname>Hans</firstname><surname>Ren</surname><name>Hans Ren</name><active>true</active><ethesisStudent>false</ethesisStudent></author><author><sid>b334d40963c7a2f435f06d2c26c74e11</sid><ORCID>0000-0002-2701-8660</ORCID><firstname>Xianghua</firstname><surname>Xie</surname><name>Xianghua Xie</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2023-12-12</date><deptcode>MACS</deptcode><abstract>Conventional machine learning methodologies require the centralization of data for model training, which may be infeasible in situations where data sharing limitations are imposed due to concerns such as privacy and gradient protection. The Federated Learning (FL) framework enables the collaborative learning of a shared model without necessitating the centralization or sharing of data among the data proprietors. Nonetheless, in this paper, we demonstrate that the generalization capability of the joint model is suboptimal for Non-Independent and Non-Identically Distributed (Non-IID) data, particularly when employing the Federated Averaging (FedAvg) strategy as a result of the weight divergence phenomenon. Consequently, we present a novel boosting algorithm for FL to address both the generalization and gradient leakage challenges, as well as to facilitate accelerated convergence in gradient-based optimization. Furthermore, we introduce a secure gradient sharing protocol that incorporates Homomorphic Encryption (HE) and Differential Privacy (DP) to safeguard against gradient leakage attacks. Our empirical evaluation demonstrates that the proposed Federated Boosting (FedBoosting) technique yields significant enhancements in both prediction accuracy and computational efficiency in the visual text recognition task on publicly available benchmarks.</abstract><type>Journal Article</type><journal>Neurocomputing</journal><volume>569</volume><journalNumber/><paginationStart/><paginationEnd/><publisher>Elsevier BV</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint>0925-2312</issnPrint><issnElectronic/><keywords>Deep learning, Federated learning, Privacy preserving</keywords><publishedDay>7</publishedDay><publishedMonth>2</publishedMonth><publishedYear>2024</publishedYear><publishedDate>2024-02-07</publishedDate><doi>10.1016/j.neucom.2023.127126</doi><url/><notes/><college>COLLEGE NANME</college><department>Mathematics and Computer Science School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MACS</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders>EPSRC, EP/N028139/1</funders><projectreference/><lastEdited>2024-07-11T15:00:56.3406771</lastEdited><Created>2023-12-12T09:28:46.1839483</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Hans</firstname><surname>Ren</surname><order>1</order></author><author><firstname>Jingjing</firstname><surname>Deng</surname><orcid>0000-0001-9274-651x</orcid><order>2</order></author><author><firstname>Xianghua</firstname><surname>Xie</surname><orcid>0000-0002-2701-8660</orcid><order>3</order></author><author><firstname>Xiaoke</firstname><surname>Ma</surname><order>4</order></author><author><firstname>Yichuan</firstname><surname>Wang</surname><order>5</order></author></authors><documents><document><filename>65267__29934__ed8b8a46225d44c196ee9fd402ea8c17.pdf</filename><originalFilename>65267.VOR.pdf</originalFilename><uploaded>2024-04-05T14:43:33.1627705</uploaded><type>Output</type><contentLength>3144007</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling 2024-07-11T15:00:56.3406771 v2 65267 2023-12-12 FedBoosting: Federated learning with gradient protected boosting for text recognition 9e043b899a2b786672a28ed4f864ffcc Hans Ren Hans Ren true false b334d40963c7a2f435f06d2c26c74e11 0000-0002-2701-8660 Xianghua Xie Xianghua Xie true false 2023-12-12 MACS Conventional machine learning methodologies require the centralization of data for model training, which may be infeasible in situations where data sharing limitations are imposed due to concerns such as privacy and gradient protection. The Federated Learning (FL) framework enables the collaborative learning of a shared model without necessitating the centralization or sharing of data among the data proprietors. Nonetheless, in this paper, we demonstrate that the generalization capability of the joint model is suboptimal for Non-Independent and Non-Identically Distributed (Non-IID) data, particularly when employing the Federated Averaging (FedAvg) strategy as a result of the weight divergence phenomenon. Consequently, we present a novel boosting algorithm for FL to address both the generalization and gradient leakage challenges, as well as to facilitate accelerated convergence in gradient-based optimization. Furthermore, we introduce a secure gradient sharing protocol that incorporates Homomorphic Encryption (HE) and Differential Privacy (DP) to safeguard against gradient leakage attacks. Our empirical evaluation demonstrates that the proposed Federated Boosting (FedBoosting) technique yields significant enhancements in both prediction accuracy and computational efficiency in the visual text recognition task on publicly available benchmarks. Journal Article Neurocomputing 569 Elsevier BV 0925-2312 Deep learning, Federated learning, Privacy preserving 7 2 2024 2024-02-07 10.1016/j.neucom.2023.127126 COLLEGE NANME Mathematics and Computer Science School COLLEGE CODE MACS Swansea University Another institution paid the OA fee EPSRC, EP/N028139/1 2024-07-11T15:00:56.3406771 2023-12-12T09:28:46.1839483 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Hans Ren 1 Jingjing Deng 0000-0001-9274-651x 2 Xianghua Xie 0000-0002-2701-8660 3 Xiaoke Ma 4 Yichuan Wang 5 65267__29934__ed8b8a46225d44c196ee9fd402ea8c17.pdf 65267.VOR.pdf 2024-04-05T14:43:33.1627705 Output 3144007 application/pdf Version of Record true true eng https://creativecommons.org/licenses/by/4.0/
title FedBoosting: Federated learning with gradient protected boosting for text recognition
spellingShingle FedBoosting: Federated learning with gradient protected boosting for text recognition
Hans Ren
Xianghua Xie
title_short FedBoosting: Federated learning with gradient protected boosting for text recognition
title_full FedBoosting: Federated learning with gradient protected boosting for text recognition
title_fullStr FedBoosting: Federated learning with gradient protected boosting for text recognition
title_full_unstemmed FedBoosting: Federated learning with gradient protected boosting for text recognition
title_sort FedBoosting: Federated learning with gradient protected boosting for text recognition
author_id_str_mv 9e043b899a2b786672a28ed4f864ffcc
b334d40963c7a2f435f06d2c26c74e11
author_id_fullname_str_mv 9e043b899a2b786672a28ed4f864ffcc_***_Hans Ren
b334d40963c7a2f435f06d2c26c74e11_***_Xianghua Xie
author Hans Ren
Xianghua Xie
author2 Hans Ren
Jingjing Deng
Xianghua Xie
Xiaoke Ma
Yichuan Wang
format Journal article
container_title Neurocomputing
container_volume 569
publishDate 2024
institution Swansea University
issn 0925-2312
doi_str_mv 10.1016/j.neucom.2023.127126
publisher Elsevier BV
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 1
active_str 0
description Conventional machine learning methodologies require the centralization of data for model training, which may be infeasible in situations where data sharing limitations are imposed due to concerns such as privacy and gradient protection. The Federated Learning (FL) framework enables the collaborative learning of a shared model without necessitating the centralization or sharing of data among the data proprietors. Nonetheless, in this paper, we demonstrate that the generalization capability of the joint model is suboptimal for Non-Independent and Non-Identically Distributed (Non-IID) data, particularly when employing the Federated Averaging (FedAvg) strategy as a result of the weight divergence phenomenon. Consequently, we present a novel boosting algorithm for FL to address both the generalization and gradient leakage challenges, as well as to facilitate accelerated convergence in gradient-based optimization. Furthermore, we introduce a secure gradient sharing protocol that incorporates Homomorphic Encryption (HE) and Differential Privacy (DP) to safeguard against gradient leakage attacks. Our empirical evaluation demonstrates that the proposed Federated Boosting (FedBoosting) technique yields significant enhancements in both prediction accuracy and computational efficiency in the visual text recognition task on publicly available benchmarks.
published_date 2024-02-07T20:27:19Z
_version_ 1821348032136871936
score 11.04748

Similar Items