No Cover Image

Journal article 380 views

BIAS: A Toolbox for Benchmarking Structural Bias in the Continuous Domain

Diederick Vermetten Orcid Logo, Bas van Stein, Fabio Caraffini Orcid Logo, Leandro L. Minku Orcid Logo, Anna V. Kononova

IEEE Transactions on Evolutionary Computation, Pages: 1 - 1

Swansea University Author: Fabio Caraffini Orcid Logo

Full text not available from this repository: check for access using links below.

Check full text

DOI (Published version): 10.1109/tevc.2022.3189848

Abstract

Benchmarking heuristic algorithms is vital to understand under which conditions and on what kind of problems certain algorithms perform well. Most benchmarks are performance-based, to test algorithm performance under a wide set of conditions. There are also resource-and behaviour-based benchmarks to...

Full description

Published in: IEEE Transactions on Evolutionary Computation
ISSN: 1089-778X 1941-0026
Published: Institute of Electrical and Electronics Engineers (IEEE) 2022
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa60904
Tags: Add Tag
No Tags, Be the first to tag this record!
first_indexed 2022-09-01T16:47:14Z
last_indexed 2023-01-13T19:21:22Z
id cronfa60904
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2022-09-23T11:53:52.9873222</datestamp><bib-version>v2</bib-version><id>60904</id><entry>2022-08-28</entry><title>BIAS: A Toolbox for Benchmarking Structural Bias in the Continuous Domain</title><swanseaauthors><author><sid>d0b8d4e63d512d4d67a02a23dd20dfdb</sid><ORCID>0000-0001-9199-7368</ORCID><firstname>Fabio</firstname><surname>Caraffini</surname><name>Fabio Caraffini</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2022-08-28</date><deptcode>SCS</deptcode><abstract>Benchmarking heuristic algorithms is vital to understand under which conditions and on what kind of problems certain algorithms perform well. Most benchmarks are performance-based, to test algorithm performance under a wide set of conditions. There are also resource-and behaviour-based benchmarks to test the resource consumption and the behaviour of algorithms. In this article, we propose a novel behaviour-based benchmark toolbox: BIAS (Bias in Algorithms, Structural). This toolbox can detect structural bias per dimension and across dimension based on 39 statistical tests. Moreover, it predicts the type of structural bias using a Random Forest model. BIAS can be used to better understand and improve existing algorithms (removing bias) as well as to test novel algorithms for structural bias in an early phase of development. Experiments with a large set of generated structural bias scenarios show that BIAS was successful in identifying bias. In addition we also provide the results of BIAS on 432 existing state-of-the-art optimisation algorithms showing that different kinds of structural bias are present in these algorithms, mostly towards the centre of the objective space or showing discretization behaviour. The proposed toolbox is made available open-source and recommendations are provided for the sample size and hyper-parameters to be used when applying the toolbox on other algorithms.</abstract><type>Journal Article</type><journal>IEEE Transactions on Evolutionary Computation</journal><volume/><journalNumber/><paginationStart>1</paginationStart><paginationEnd>1</paginationEnd><publisher>Institute of Electrical and Electronics Engineers (IEEE)</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint>1089-778X</issnPrint><issnElectronic>1941-0026</issnElectronic><keywords/><publishedDay>13</publishedDay><publishedMonth>7</publishedMonth><publishedYear>2022</publishedYear><publishedDate>2022-07-13</publishedDate><doi>10.1109/tevc.2022.3189848</doi><url/><notes/><college>COLLEGE NANME</college><department>Computer Science</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>SCS</DepartmentCode><institution>Swansea University</institution><apcterm>Other</apcterm><funders/><projectreference/><lastEdited>2022-09-23T11:53:52.9873222</lastEdited><Created>2022-08-28T18:58:41.2864299</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Diederick</firstname><surname>Vermetten</surname><orcid>0000-0003-3040-7162</orcid><order>1</order></author><author><firstname>Bas van</firstname><surname>Stein</surname><order>2</order></author><author><firstname>Fabio</firstname><surname>Caraffini</surname><orcid>0000-0001-9199-7368</orcid><order>3</order></author><author><firstname>Leandro L.</firstname><surname>Minku</surname><orcid>0000-0002-2639-0671</orcid><order>4</order></author><author><firstname>Anna V.</firstname><surname>Kononova</surname><order>5</order></author></authors><documents/><OutputDurs/></rfc1807>
spelling 2022-09-23T11:53:52.9873222 v2 60904 2022-08-28 BIAS: A Toolbox for Benchmarking Structural Bias in the Continuous Domain d0b8d4e63d512d4d67a02a23dd20dfdb 0000-0001-9199-7368 Fabio Caraffini Fabio Caraffini true false 2022-08-28 SCS Benchmarking heuristic algorithms is vital to understand under which conditions and on what kind of problems certain algorithms perform well. Most benchmarks are performance-based, to test algorithm performance under a wide set of conditions. There are also resource-and behaviour-based benchmarks to test the resource consumption and the behaviour of algorithms. In this article, we propose a novel behaviour-based benchmark toolbox: BIAS (Bias in Algorithms, Structural). This toolbox can detect structural bias per dimension and across dimension based on 39 statistical tests. Moreover, it predicts the type of structural bias using a Random Forest model. BIAS can be used to better understand and improve existing algorithms (removing bias) as well as to test novel algorithms for structural bias in an early phase of development. Experiments with a large set of generated structural bias scenarios show that BIAS was successful in identifying bias. In addition we also provide the results of BIAS on 432 existing state-of-the-art optimisation algorithms showing that different kinds of structural bias are present in these algorithms, mostly towards the centre of the objective space or showing discretization behaviour. The proposed toolbox is made available open-source and recommendations are provided for the sample size and hyper-parameters to be used when applying the toolbox on other algorithms. Journal Article IEEE Transactions on Evolutionary Computation 1 1 Institute of Electrical and Electronics Engineers (IEEE) 1089-778X 1941-0026 13 7 2022 2022-07-13 10.1109/tevc.2022.3189848 COLLEGE NANME Computer Science COLLEGE CODE SCS Swansea University Other 2022-09-23T11:53:52.9873222 2022-08-28T18:58:41.2864299 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Diederick Vermetten 0000-0003-3040-7162 1 Bas van Stein 2 Fabio Caraffini 0000-0001-9199-7368 3 Leandro L. Minku 0000-0002-2639-0671 4 Anna V. Kononova 5
title BIAS: A Toolbox for Benchmarking Structural Bias in the Continuous Domain
spellingShingle BIAS: A Toolbox for Benchmarking Structural Bias in the Continuous Domain
Fabio Caraffini
title_short BIAS: A Toolbox for Benchmarking Structural Bias in the Continuous Domain
title_full BIAS: A Toolbox for Benchmarking Structural Bias in the Continuous Domain
title_fullStr BIAS: A Toolbox for Benchmarking Structural Bias in the Continuous Domain
title_full_unstemmed BIAS: A Toolbox for Benchmarking Structural Bias in the Continuous Domain
title_sort BIAS: A Toolbox for Benchmarking Structural Bias in the Continuous Domain
author_id_str_mv d0b8d4e63d512d4d67a02a23dd20dfdb
author_id_fullname_str_mv d0b8d4e63d512d4d67a02a23dd20dfdb_***_Fabio Caraffini
author Fabio Caraffini
author2 Diederick Vermetten
Bas van Stein
Fabio Caraffini
Leandro L. Minku
Anna V. Kononova
format Journal article
container_title IEEE Transactions on Evolutionary Computation
container_start_page 1
publishDate 2022
institution Swansea University
issn 1089-778X
1941-0026
doi_str_mv 10.1109/tevc.2022.3189848
publisher Institute of Electrical and Electronics Engineers (IEEE)
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 0
active_str 0
description Benchmarking heuristic algorithms is vital to understand under which conditions and on what kind of problems certain algorithms perform well. Most benchmarks are performance-based, to test algorithm performance under a wide set of conditions. There are also resource-and behaviour-based benchmarks to test the resource consumption and the behaviour of algorithms. In this article, we propose a novel behaviour-based benchmark toolbox: BIAS (Bias in Algorithms, Structural). This toolbox can detect structural bias per dimension and across dimension based on 39 statistical tests. Moreover, it predicts the type of structural bias using a Random Forest model. BIAS can be used to better understand and improve existing algorithms (removing bias) as well as to test novel algorithms for structural bias in an early phase of development. Experiments with a large set of generated structural bias scenarios show that BIAS was successful in identifying bias. In addition we also provide the results of BIAS on 432 existing state-of-the-art optimisation algorithms showing that different kinds of structural bias are present in these algorithms, mostly towards the centre of the objective space or showing discretization behaviour. The proposed toolbox is made available open-source and recommendations are provided for the sample size and hyper-parameters to be used when applying the toolbox on other algorithms.
published_date 2022-07-13T04:19:24Z
_version_ 1763754287440068608
score 10.99807

Similar Items