Journal article 58 views 1 download
On learning higher-order cumulants in diffusion models
,
Diaa Eddin Habibi,
Lingxiao Wang ,
Kai Zhou
Machine Learning: Science and Technology, Volume: 6, Issue: 2, Start page: 025004
Swansea University Authors:
Gert Aarts , Diaa Eddin Habibi
-
PDF | Version of Record
©2025TheAuthor(s). Released under the terms of the Creative Commons Attribution 4.0 licence.
Download (1.67MB)
DOI (Published version): 10.1088/2632-2153/adc53a
Abstract
To analyse how diffusion models learn correlations beyond Gaussian ones, we study the behaviour of higher-order cumulants, or connected n-point functions, under both the forward and backward process. We derive explicit expressions for the moment- and cumulant-generating functionals, in terms of the...
| Published in: | Machine Learning: Science and Technology |
|---|---|
| ISSN: | 2632-2153 |
| Published: |
IOP Publishing
2025
|
| Online Access: |
Check full text
|
| URI: | https://cronfa.swan.ac.uk/Record/cronfa69229 |
| Abstract: |
To analyse how diffusion models learn correlations beyond Gaussian ones, we study the behaviour of higher-order cumulants, or connected n-point functions, under both the forward and backward process. We derive explicit expressions for the moment- and cumulant-generating functionals, in terms of the distribution of the initial data and properties of forward process. It is shown analytically that during the forward process higher-order cumulants are conserved in models without a drift, such as the variance-expanding scheme, and that therefore the endpoint of the forward process maintains nontrivial correlations. We demonstrate that since these correlations are encoded in the score function, higher-order cumulants are learnt in the backward process, also when starting from a normal prior. We confirm our analytical results in an exactly solvable toy model with nonzero cumulants and in scalar lattice field theory. |
|---|---|
| Keywords: |
learning, cumulants, lattice field theory, diffusion models |
| College: |
Faculty of Science and Engineering |
| Funders: |
GAis supported by STFC Consolidated Grant ST/X000648/1. DEH is supported by the UKRI AIMLAC CDT EP/S023992/1. We thanks the DEEP-IN working group at RIKEN-iTHEMS for its support in the preparation of this paper. LW is also supported by the RIKEN TRIP initiative (RIKEN Quantum) and JST-BOOST Grant (No. 24036419). KZ is supported by the CUHK-Shenzhen university development fund under Grant Nos. UDF01003041 and UDF03003041, and Shenzhen Peacock fund under No. 2023TC0179. Weacknowledge the support of the Supercomputing Wales project, which is part-funded by the European Regional Development Fund (ERDF) via Welsh Government. |
| Issue: |
2 |
| Start Page: |
025004 |