Mathematical analysis of singularities in the diffusion model under the submanifold assumption

Author name not available (Why is that?)

Publication date: 19 January 2023

Abstract: This paper provide several mathematical analyses of the diffusion model in machine learning. The drift term of the backwards sampling process is represented as a conditional expectation involving the data distribution and the forward diffusion. The training process aims to find such a drift function by minimizing the mean-squared residue related to the conditional expectation. Using small-time approximations of the Green's function of the forward diffusion, we show that the analytical mean drift function in DDPM and the score function in SGM asymptotically blow up in the final stages of the sampling process for singular data distributions such as those concentrated on lower-dimensional manifolds, and is therefore difficult to approximate by a network. To overcome this difficulty, we derive a new target function and associated loss, which remains bounded even for singular data distributions. We illustrate the theoretical findings with several numerical examples.

Has companion code repository: https://github.com/yubin-lu/cem_diffusion_model

This page was built for publication: Mathematical analysis of singularities in the diffusion model under the submanifold assumption

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6423793)