No Cover Image

Conference Paper/Proceeding/Abstract 108 views 18 downloads

An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms

Leonardo Lucio Custode Orcid Logo, Fabio Caraffini Orcid Logo, Anil Yaman Orcid Logo, Giovanni Iacca Orcid Logo

Proceedings of the Genetic and Evolutionary Computation Conference Companion, Volume: 12, Pages: 1838 - 1845

Swansea University Author: Fabio Caraffini Orcid Logo

  • 3638530.3664163.pdf

    PDF | Version of Record

    © 2024 Copyright held by the owner/author(s). Released under the terms of a CC-BY-NC-SA license.

    Download (695.49KB)

DOI (Published version): 10.1145/3638530.3664163

Abstract

Hyperparameter optimization is a crucial problem in Evolutionary Computation. In fact, the values of the hyperparameters directly impact the trajectory taken by the optimization process, and their choice requires extensive reasoning by human operators. Although a variety of self-adaptive Evolutionar...

Full description

Published in: Proceedings of the Genetic and Evolutionary Computation Conference Companion
ISBN: 979-8-4007-0495-6 979-8-4007-0495-6
Published: New York, NY, USA ACM 2024
URI: https://cronfa.swan.ac.uk/Record/cronfa67311
Tags: Add Tag
No Tags, Be the first to tag this record!
first_indexed 2024-08-02T21:44:58Z
last_indexed 2024-08-02T21:44:58Z
id cronfa67311
recordtype SURis
fullrecord <?xml version="1.0" encoding="utf-8"?><rfc1807 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"><bib-version>v2</bib-version><id>67311</id><entry>2024-08-02</entry><title>An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms</title><swanseaauthors><author><sid>d0b8d4e63d512d4d67a02a23dd20dfdb</sid><ORCID>0000-0001-9199-7368</ORCID><firstname>Fabio</firstname><surname>Caraffini</surname><name>Fabio Caraffini</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2024-08-02</date><deptcode>MACS</deptcode><abstract>Hyperparameter optimization is a crucial problem in Evolutionary Computation. In fact, the values of the hyperparameters directly impact the trajectory taken by the optimization process, and their choice requires extensive reasoning by human operators. Although a variety of self-adaptive Evolutionary Algorithms have been proposed in the literature, no definitive solution has been found. In this work, we perform a preliminary investigation to automate the reasoning process that leads to the choice of hyperparameter values. We employ two open-source Large Language Models (LLMs), namely Llama2-70b and Mixtral, to analyze the optimization logs online and provide novel real-time hyperparameter recommendations. We study our approach in the context of step-size adaptation for (1 + 1)-ES. The results suggest that LLMs can be an effective method for optimizing hyperparameters in Evolution Strategies, encouraging further research in this direction.</abstract><type>Conference Paper/Proceeding/Abstract</type><journal>Proceedings of the Genetic and Evolutionary Computation Conference Companion</journal><volume>12</volume><journalNumber/><paginationStart>1838</paginationStart><paginationEnd>1845</paginationEnd><publisher>ACM</publisher><placeOfPublication>New York, NY, USA</placeOfPublication><isbnPrint>979-8-4007-0495-6</isbnPrint><isbnElectronic>979-8-4007-0495-6</isbnElectronic><issnPrint/><issnElectronic/><keywords>Evolutionary Algorithms, Large Language Models, Landscape Analysis, Parameter Tuning</keywords><publishedDay>1</publishedDay><publishedMonth>8</publishedMonth><publishedYear>2024</publishedYear><publishedDate>2024-08-01</publishedDate><doi>10.1145/3638530.3664163</doi><url/><notes/><college>COLLEGE NANME</college><department>Mathematics and Computer Science School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MACS</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders/><projectreference/><lastEdited>2024-09-30T13:52:00.2990839</lastEdited><Created>2024-08-02T22:42:19.3768341</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Leonardo Lucio</firstname><surname>Custode</surname><orcid>0000-0002-1652-1690</orcid><order>1</order></author><author><firstname>Fabio</firstname><surname>Caraffini</surname><orcid>0000-0001-9199-7368</orcid><order>2</order></author><author><firstname>Anil</firstname><surname>Yaman</surname><orcid>0000-0003-1379-3778</orcid><order>3</order></author><author><firstname>Giovanni</firstname><surname>Iacca</surname><orcid>0000-0001-9723-1830</orcid><order>4</order></author></authors><documents><document><filename>67311__31063__c34c2a02ee4e4461a698325cbd8b59d1.pdf</filename><originalFilename>3638530.3664163.pdf</originalFilename><uploaded>2024-08-07T13:12:56.3559064</uploaded><type>Output</type><contentLength>712179</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>© 2024 Copyright held by the owner/author(s). Released under the terms of a CC-BY-NC-SA license.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en</licence></document></documents><OutputDurs/></rfc1807>
spelling v2 67311 2024-08-02 An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms d0b8d4e63d512d4d67a02a23dd20dfdb 0000-0001-9199-7368 Fabio Caraffini Fabio Caraffini true false 2024-08-02 MACS Hyperparameter optimization is a crucial problem in Evolutionary Computation. In fact, the values of the hyperparameters directly impact the trajectory taken by the optimization process, and their choice requires extensive reasoning by human operators. Although a variety of self-adaptive Evolutionary Algorithms have been proposed in the literature, no definitive solution has been found. In this work, we perform a preliminary investigation to automate the reasoning process that leads to the choice of hyperparameter values. We employ two open-source Large Language Models (LLMs), namely Llama2-70b and Mixtral, to analyze the optimization logs online and provide novel real-time hyperparameter recommendations. We study our approach in the context of step-size adaptation for (1 + 1)-ES. The results suggest that LLMs can be an effective method for optimizing hyperparameters in Evolution Strategies, encouraging further research in this direction. Conference Paper/Proceeding/Abstract Proceedings of the Genetic and Evolutionary Computation Conference Companion 12 1838 1845 ACM New York, NY, USA 979-8-4007-0495-6 979-8-4007-0495-6 Evolutionary Algorithms, Large Language Models, Landscape Analysis, Parameter Tuning 1 8 2024 2024-08-01 10.1145/3638530.3664163 COLLEGE NANME Mathematics and Computer Science School COLLEGE CODE MACS Swansea University Another institution paid the OA fee 2024-09-30T13:52:00.2990839 2024-08-02T22:42:19.3768341 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Leonardo Lucio Custode 0000-0002-1652-1690 1 Fabio Caraffini 0000-0001-9199-7368 2 Anil Yaman 0000-0003-1379-3778 3 Giovanni Iacca 0000-0001-9723-1830 4 67311__31063__c34c2a02ee4e4461a698325cbd8b59d1.pdf 3638530.3664163.pdf 2024-08-07T13:12:56.3559064 Output 712179 application/pdf Version of Record true © 2024 Copyright held by the owner/author(s). Released under the terms of a CC-BY-NC-SA license. true eng https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en
title An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms
spellingShingle An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms
Fabio Caraffini
title_short An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms
title_full An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms
title_fullStr An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms
title_full_unstemmed An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms
title_sort An investigation on the use of Large Language Models for hyperparameter tuning in Evolutionary Algorithms
author_id_str_mv d0b8d4e63d512d4d67a02a23dd20dfdb
author_id_fullname_str_mv d0b8d4e63d512d4d67a02a23dd20dfdb_***_Fabio Caraffini
author Fabio Caraffini
author2 Leonardo Lucio Custode
Fabio Caraffini
Anil Yaman
Giovanni Iacca
format Conference Paper/Proceeding/Abstract
container_title Proceedings of the Genetic and Evolutionary Computation Conference Companion
container_volume 12
container_start_page 1838
publishDate 2024
institution Swansea University
isbn 979-8-4007-0495-6
979-8-4007-0495-6
doi_str_mv 10.1145/3638530.3664163
publisher ACM
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 1
active_str 0
description Hyperparameter optimization is a crucial problem in Evolutionary Computation. In fact, the values of the hyperparameters directly impact the trajectory taken by the optimization process, and their choice requires extensive reasoning by human operators. Although a variety of self-adaptive Evolutionary Algorithms have been proposed in the literature, no definitive solution has been found. In this work, we perform a preliminary investigation to automate the reasoning process that leads to the choice of hyperparameter values. We employ two open-source Large Language Models (LLMs), namely Llama2-70b and Mixtral, to analyze the optimization logs online and provide novel real-time hyperparameter recommendations. We study our approach in the context of step-size adaptation for (1 + 1)-ES. The results suggest that LLMs can be an effective method for optimizing hyperparameters in Evolution Strategies, encouraging further research in this direction.
published_date 2024-08-01T13:51:58Z
_version_ 1811625509137678336
score 11.037144

Similar Items