No Cover Image

Journal article 34 views 6 downloads

A Multimodal Ensemble-Based Framework for Detecting Fake News Using Visual and Textual Features

Muhammad Abdullah Orcid Logo, Hongying Zan, Arifa Javed Orcid Logo, Muhammad Sohail, Orken Mamyrbayev Orcid Logo, Zhanibek Turysbek Orcid Logo, Hassan Eshkiki Orcid Logo, Fabio Caraffini Orcid Logo

Mathematics, Volume: 14, Issue: 2, Start page: 360

Swansea University Authors: Hassan Eshkiki Orcid Logo, Fabio Caraffini Orcid Logo

  • 71351.VOR.pdf

    PDF | Version of Record

    © 2026 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.

    Download (15.79MB)

Check full text

DOI (Published version): 10.3390/math14020360

Abstract

Detecting fake news is essential in natural language processing to verify news authenticity and prevent misinformation-driven social, political, and economic disruptions targeting specific groups. A major challenge in multimodal fake news detection is effectively integrating textual and visual modal...

Full description

Published in: Mathematics
ISSN: 2227-7390
Published: MDPI AG 2026
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa71351
first_indexed 2026-01-30T13:40:18Z
last_indexed 2026-01-31T05:34:56Z
id cronfa71351
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2026-01-30T13:41:24.1073519</datestamp><bib-version>v2</bib-version><id>71351</id><entry>2026-01-30</entry><title>A Multimodal Ensemble-Based Framework for Detecting Fake News Using Visual and Textual Features</title><swanseaauthors><author><sid>c9972b26a83de11ffe211070f26fe16b</sid><ORCID>0000-0001-7795-453X</ORCID><firstname>Hassan</firstname><surname>Eshkiki</surname><name>Hassan Eshkiki</name><active>true</active><ethesisStudent>false</ethesisStudent></author><author><sid>d0b8d4e63d512d4d67a02a23dd20dfdb</sid><ORCID>0000-0001-9199-7368</ORCID><firstname>Fabio</firstname><surname>Caraffini</surname><name>Fabio Caraffini</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2026-01-30</date><deptcode>MACS</deptcode><abstract>Detecting fake news is essential in natural language processing to verify news authenticity and prevent misinformation-driven social, political, and economic disruptions targeting specific groups. A major challenge in multimodal fake news detection is effectively integrating textual and visual modalities, as semantic gaps and contextual variations between images and text complicate alignment, interpretation, and the detection of subtle or blatant inconsistencies. To enhance accuracy in fake news detection, this article introduces an ensemble-based framework that integrates textual and visual data using ViLBERT&#x2019;s two-stream architecture, incorporates VADER sentiment analysis to detect emotional language, and uses Image&#x2013;Text Contextual Similarity to identify mismatches between visual and textual elements. These features are processed through the Bi-GRU classifier, Transformer-XL, DistilBERT, and XLNet, combined via a stacked ensemble method with soft voting, culminating in a T5 metaclassifier that predicts the outcome for robustness. Results on the Fakeddit and Weibo benchmarking datasets show that our method outperforms state-of-the-art models, achieving up to 96% and 94% accuracy in fake news detection, respectively. This study highlights the necessity for advanced multimodal fake news detection systems to address the increasing complexity of misinformation and offers a promising solution.</abstract><type>Journal Article</type><journal>Mathematics</journal><volume>14</volume><journalNumber>2</journalNumber><paginationStart>360</paginationStart><paginationEnd/><publisher>MDPI AG</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic>2227-7390</issnElectronic><keywords>fake news detection; NLP; sentiment analysis; transformers; deep learning</keywords><publishedDay>21</publishedDay><publishedMonth>1</publishedMonth><publishedYear>2026</publishedYear><publishedDate>2026-01-21</publishedDate><doi>10.3390/math14020360</doi><url/><notes/><college>COLLEGE NANME</college><department>Mathematics and Computer Science School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MACS</DepartmentCode><institution>Swansea University</institution><apcterm>Other</apcterm><funders/><projectreference/><lastEdited>2026-01-30T13:41:24.1073519</lastEdited><Created>2026-01-30T13:35:57.7577885</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Muhammad</firstname><surname>Abdullah</surname><orcid>0009-0000-9434-7977</orcid><order>1</order></author><author><firstname>Hongying</firstname><surname>Zan</surname><order>2</order></author><author><firstname>Arifa</firstname><surname>Javed</surname><orcid>0009-0002-6112-6398</orcid><order>3</order></author><author><firstname>Muhammad</firstname><surname>Sohail</surname><order>4</order></author><author><firstname>Orken</firstname><surname>Mamyrbayev</surname><orcid>0000-0001-8318-3794</orcid><order>5</order></author><author><firstname>Zhanibek</firstname><surname>Turysbek</surname><orcid>0009-0004-2311-6249</orcid><order>6</order></author><author><firstname>Hassan</firstname><surname>Eshkiki</surname><orcid>0000-0001-7795-453X</orcid><order>7</order></author><author><firstname>Fabio</firstname><surname>Caraffini</surname><orcid>0000-0001-9199-7368</orcid><order>8</order></author></authors><documents><document><filename>71351__36148__5c55af71409942a2b3e73bf2b8ff2a6c.pdf</filename><originalFilename>71351.VOR.pdf</originalFilename><uploaded>2026-01-30T13:39:07.4505517</uploaded><type>Output</type><contentLength>16553566</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>&#xA9; 2026 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling 2026-01-30T13:41:24.1073519 v2 71351 2026-01-30 A Multimodal Ensemble-Based Framework for Detecting Fake News Using Visual and Textual Features c9972b26a83de11ffe211070f26fe16b 0000-0001-7795-453X Hassan Eshkiki Hassan Eshkiki true false d0b8d4e63d512d4d67a02a23dd20dfdb 0000-0001-9199-7368 Fabio Caraffini Fabio Caraffini true false 2026-01-30 MACS Detecting fake news is essential in natural language processing to verify news authenticity and prevent misinformation-driven social, political, and economic disruptions targeting specific groups. A major challenge in multimodal fake news detection is effectively integrating textual and visual modalities, as semantic gaps and contextual variations between images and text complicate alignment, interpretation, and the detection of subtle or blatant inconsistencies. To enhance accuracy in fake news detection, this article introduces an ensemble-based framework that integrates textual and visual data using ViLBERT’s two-stream architecture, incorporates VADER sentiment analysis to detect emotional language, and uses Image–Text Contextual Similarity to identify mismatches between visual and textual elements. These features are processed through the Bi-GRU classifier, Transformer-XL, DistilBERT, and XLNet, combined via a stacked ensemble method with soft voting, culminating in a T5 metaclassifier that predicts the outcome for robustness. Results on the Fakeddit and Weibo benchmarking datasets show that our method outperforms state-of-the-art models, achieving up to 96% and 94% accuracy in fake news detection, respectively. This study highlights the necessity for advanced multimodal fake news detection systems to address the increasing complexity of misinformation and offers a promising solution. Journal Article Mathematics 14 2 360 MDPI AG 2227-7390 fake news detection; NLP; sentiment analysis; transformers; deep learning 21 1 2026 2026-01-21 10.3390/math14020360 COLLEGE NANME Mathematics and Computer Science School COLLEGE CODE MACS Swansea University Other 2026-01-30T13:41:24.1073519 2026-01-30T13:35:57.7577885 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Muhammad Abdullah 0009-0000-9434-7977 1 Hongying Zan 2 Arifa Javed 0009-0002-6112-6398 3 Muhammad Sohail 4 Orken Mamyrbayev 0000-0001-8318-3794 5 Zhanibek Turysbek 0009-0004-2311-6249 6 Hassan Eshkiki 0000-0001-7795-453X 7 Fabio Caraffini 0000-0001-9199-7368 8 71351__36148__5c55af71409942a2b3e73bf2b8ff2a6c.pdf 71351.VOR.pdf 2026-01-30T13:39:07.4505517 Output 16553566 application/pdf Version of Record true © 2026 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. true eng https://creativecommons.org/licenses/by/4.0/
title A Multimodal Ensemble-Based Framework for Detecting Fake News Using Visual and Textual Features
spellingShingle A Multimodal Ensemble-Based Framework for Detecting Fake News Using Visual and Textual Features
Hassan Eshkiki
Fabio Caraffini
title_short A Multimodal Ensemble-Based Framework for Detecting Fake News Using Visual and Textual Features
title_full A Multimodal Ensemble-Based Framework for Detecting Fake News Using Visual and Textual Features
title_fullStr A Multimodal Ensemble-Based Framework for Detecting Fake News Using Visual and Textual Features
title_full_unstemmed A Multimodal Ensemble-Based Framework for Detecting Fake News Using Visual and Textual Features
title_sort A Multimodal Ensemble-Based Framework for Detecting Fake News Using Visual and Textual Features
author_id_str_mv c9972b26a83de11ffe211070f26fe16b
d0b8d4e63d512d4d67a02a23dd20dfdb
author_id_fullname_str_mv c9972b26a83de11ffe211070f26fe16b_***_Hassan Eshkiki
d0b8d4e63d512d4d67a02a23dd20dfdb_***_Fabio Caraffini
author Hassan Eshkiki
Fabio Caraffini
author2 Muhammad Abdullah
Hongying Zan
Arifa Javed
Muhammad Sohail
Orken Mamyrbayev
Zhanibek Turysbek
Hassan Eshkiki
Fabio Caraffini
format Journal article
container_title Mathematics
container_volume 14
container_issue 2
container_start_page 360
publishDate 2026
institution Swansea University
issn 2227-7390
doi_str_mv 10.3390/math14020360
publisher MDPI AG
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 1
active_str 0
description Detecting fake news is essential in natural language processing to verify news authenticity and prevent misinformation-driven social, political, and economic disruptions targeting specific groups. A major challenge in multimodal fake news detection is effectively integrating textual and visual modalities, as semantic gaps and contextual variations between images and text complicate alignment, interpretation, and the detection of subtle or blatant inconsistencies. To enhance accuracy in fake news detection, this article introduces an ensemble-based framework that integrates textual and visual data using ViLBERT’s two-stream architecture, incorporates VADER sentiment analysis to detect emotional language, and uses Image–Text Contextual Similarity to identify mismatches between visual and textual elements. These features are processed through the Bi-GRU classifier, Transformer-XL, DistilBERT, and XLNet, combined via a stacked ensemble method with soft voting, culminating in a T5 metaclassifier that predicts the outcome for robustness. Results on the Fakeddit and Weibo benchmarking datasets show that our method outperforms state-of-the-art models, achieving up to 96% and 94% accuracy in fake news detection, respectively. This study highlights the necessity for advanced multimodal fake news detection systems to address the increasing complexity of misinformation and offers a promising solution.
published_date 2026-01-21T05:35:15Z
_version_ 1856896516390649856
score 11.096068

Similar Items