No Cover Image

Journal article 93 views 13 downloads

On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus

Jan A. Bergstra Orcid Logo, John Tucker Orcid Logo

Entropy, Volume: 27, Issue: 1, Start page: 31

Swansea University Author: John Tucker Orcid Logo

  • 68639.VOR.pdf

    PDF | Version of Record

    © 2025 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.

    Download (258.37KB)

Check full text

DOI (Published version): 10.3390/e27010031

Abstract

Classic formulae for entropy and cross-entropy contain operations 0 and log2 that are not defined on all inputs. This can lead to calculations with problematic subexpressions such as 0 log2 0 and uncertainties in large scale calculations; partiality also introduces complications in logical analysis....

Full description

Published in: Entropy
ISSN: 1099-4300
Published: MDPI AG 2025
Online Access: Check full text

URI: https://cronfa.swan.ac.uk/Record/cronfa68639
first_indexed 2025-01-13T20:35:03Z
last_indexed 2025-02-08T05:44:34Z
id cronfa68639
recordtype SURis
fullrecord <?xml version="1.0"?><rfc1807><datestamp>2025-02-07T14:48:04.1928910</datestamp><bib-version>v2</bib-version><id>68639</id><entry>2025-01-02</entry><title>On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus</title><swanseaauthors><author><sid>431b3060563ed44cc68c7056ece2f85e</sid><ORCID>0000-0003-4689-8760</ORCID><firstname>John</firstname><surname>Tucker</surname><name>John Tucker</name><active>true</active><ethesisStudent>false</ethesisStudent></author></swanseaauthors><date>2025-01-02</date><deptcode>MACS</deptcode><abstract>Classic formulae for entropy and cross-entropy contain operations 0 and log2 that are not defined on all inputs. This can lead to calculations with problematic subexpressions such as 0 log2 0 and uncertainties in large scale calculations; partiality also introduces complications in logical analysis. Instead of adding conventions or splitting formulae into cases, we create a new algebra of real numbers with two symbols &#xB1;&#x221E; for signed infinite values and a symbol named &#x22A5; for the undefined. In this resulting arithmetic, entropy, cross-entropy, Kullback&#x2013;Leibler divergence, and Shannon divergence can be expressed without concerning any further conventions. The algebra may form a basis for probability theory more generally.</abstract><type>Journal Article</type><journal>Entropy</journal><volume>27</volume><journalNumber>1</journalNumber><paginationStart>31</paginationStart><paginationEnd/><publisher>MDPI AG</publisher><placeOfPublication/><isbnPrint/><isbnElectronic/><issnPrint/><issnElectronic>1099-4300</issnElectronic><keywords>Partial formulae; fracterm calculus; transreals; entropic transreals; peripheral numbers; entropy; cross-entropy</keywords><publishedDay>2</publishedDay><publishedMonth>1</publishedMonth><publishedYear>2025</publishedYear><publishedDate>2025-01-02</publishedDate><doi>10.3390/e27010031</doi><url/><notes/><college>COLLEGE NANME</college><department>Mathematics and Computer Science School</department><CollegeCode>COLLEGE CODE</CollegeCode><DepartmentCode>MACS</DepartmentCode><institution>Swansea University</institution><apcterm>Another institution paid the OA fee</apcterm><funders>This research received no external funding.</funders><projectreference/><lastEdited>2025-02-07T14:48:04.1928910</lastEdited><Created>2025-01-02T15:25:16.0368950</Created><path><level id="1">Faculty of Science and Engineering</level><level id="2">School of Mathematics and Computer Science - Computer Science</level></path><authors><author><firstname>Jan A.</firstname><surname>Bergstra</surname><orcid>0000-0003-2492-506x</orcid><order>1</order></author><author><firstname>John</firstname><surname>Tucker</surname><orcid>0000-0003-4689-8760</orcid><order>2</order></author></authors><documents><document><filename>68639__33302__aacc8d1138f74147b5f410243e66ae58.pdf</filename><originalFilename>68639.VOR.pdf</originalFilename><uploaded>2025-01-13T14:04:48.2562177</uploaded><type>Output</type><contentLength>264571</contentLength><contentType>application/pdf</contentType><version>Version of Record</version><cronfaStatus>true</cronfaStatus><documentNotes>&#xA9; 2025 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.</documentNotes><copyrightCorrect>true</copyrightCorrect><language>eng</language><licence>https://creativecommons.org/licenses/by/4.0/</licence></document></documents><OutputDurs/></rfc1807>
spelling 2025-02-07T14:48:04.1928910 v2 68639 2025-01-02 On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus 431b3060563ed44cc68c7056ece2f85e 0000-0003-4689-8760 John Tucker John Tucker true false 2025-01-02 MACS Classic formulae for entropy and cross-entropy contain operations 0 and log2 that are not defined on all inputs. This can lead to calculations with problematic subexpressions such as 0 log2 0 and uncertainties in large scale calculations; partiality also introduces complications in logical analysis. Instead of adding conventions or splitting formulae into cases, we create a new algebra of real numbers with two symbols ±∞ for signed infinite values and a symbol named ⊥ for the undefined. In this resulting arithmetic, entropy, cross-entropy, Kullback–Leibler divergence, and Shannon divergence can be expressed without concerning any further conventions. The algebra may form a basis for probability theory more generally. Journal Article Entropy 27 1 31 MDPI AG 1099-4300 Partial formulae; fracterm calculus; transreals; entropic transreals; peripheral numbers; entropy; cross-entropy 2 1 2025 2025-01-02 10.3390/e27010031 COLLEGE NANME Mathematics and Computer Science School COLLEGE CODE MACS Swansea University Another institution paid the OA fee This research received no external funding. 2025-02-07T14:48:04.1928910 2025-01-02T15:25:16.0368950 Faculty of Science and Engineering School of Mathematics and Computer Science - Computer Science Jan A. Bergstra 0000-0003-2492-506x 1 John Tucker 0000-0003-4689-8760 2 68639__33302__aacc8d1138f74147b5f410243e66ae58.pdf 68639.VOR.pdf 2025-01-13T14:04:48.2562177 Output 264571 application/pdf Version of Record true © 2025 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license. true eng https://creativecommons.org/licenses/by/4.0/
title On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus
spellingShingle On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus
John Tucker
title_short On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus
title_full On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus
title_fullStr On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus
title_full_unstemmed On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus
title_sort On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus
author_id_str_mv 431b3060563ed44cc68c7056ece2f85e
author_id_fullname_str_mv 431b3060563ed44cc68c7056ece2f85e_***_John Tucker
author John Tucker
author2 Jan A. Bergstra
John Tucker
format Journal article
container_title Entropy
container_volume 27
container_issue 1
container_start_page 31
publishDate 2025
institution Swansea University
issn 1099-4300
doi_str_mv 10.3390/e27010031
publisher MDPI AG
college_str Faculty of Science and Engineering
hierarchytype
hierarchy_top_id facultyofscienceandengineering
hierarchy_top_title Faculty of Science and Engineering
hierarchy_parent_id facultyofscienceandengineering
hierarchy_parent_title Faculty of Science and Engineering
department_str School of Mathematics and Computer Science - Computer Science{{{_:::_}}}Faculty of Science and Engineering{{{_:::_}}}School of Mathematics and Computer Science - Computer Science
document_store_str 1
active_str 0
description Classic formulae for entropy and cross-entropy contain operations 0 and log2 that are not defined on all inputs. This can lead to calculations with problematic subexpressions such as 0 log2 0 and uncertainties in large scale calculations; partiality also introduces complications in logical analysis. Instead of adding conventions or splitting formulae into cases, we create a new algebra of real numbers with two symbols ±∞ for signed infinite values and a symbol named ⊥ for the undefined. In this resulting arithmetic, entropy, cross-entropy, Kullback–Leibler divergence, and Shannon divergence can be expressed without concerning any further conventions. The algebra may form a basis for probability theory more generally.
published_date 2025-01-02T08:26:21Z
_version_ 1825832521542139904
score 11.053243

Similar Items