Journal article 125 views 35 downloads
On Defining Expressions for Entropy and Cross-Entropy: The Entropic Transreals and Their Fracterm Calculus
,
John Tucker
Entropy, Volume: 27, Issue: 1, Start page: 31
Swansea University Author:
John Tucker
-
PDF | Version of Record
© 2025 by the authors. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.
Download (258.37KB)
DOI (Published version): 10.3390/e27010031
Abstract
Classic formulae for entropy and cross-entropy contain operations 0 and log2 that are not defined on all inputs. This can lead to calculations with problematic subexpressions such as 0 log2 0 and uncertainties in large scale calculations; partiality also introduces complications in logical analysis....
| Published in: | Entropy |
|---|---|
| ISSN: | 1099-4300 |
| Published: |
MDPI AG
2025
|
| Online Access: |
Check full text
|
| URI: | https://cronfa.swan.ac.uk/Record/cronfa68639 |
| Abstract: |
Classic formulae for entropy and cross-entropy contain operations 0 and log2 that are not defined on all inputs. This can lead to calculations with problematic subexpressions such as 0 log2 0 and uncertainties in large scale calculations; partiality also introduces complications in logical analysis. Instead of adding conventions or splitting formulae into cases, we create a new algebra of real numbers with two symbols ±∞ for signed infinite values and a symbol named ⊥ for the undefined. In this resulting arithmetic, entropy, cross-entropy, Kullback–Leibler divergence, and Shannon divergence can be expressed without concerning any further conventions. The algebra may form a basis for probability theory more generally. |
|---|---|
| Keywords: |
Partial formulae; fracterm calculus; transreals; entropic transreals; peripheral numbers; entropy; cross-entropy |
| College: |
Faculty of Science and Engineering |
| Funders: |
This research received no external funding. |
| Issue: |
1 |
| Start Page: |
31 |