Infinite Divisibility of Information

DOI10.1109/TIT.2022.3156432zbMATH Open1505.94027arXiv2008.06092OpenAlexW4214872586MaRDI QIDQ5088550

Cheuk Ting Li

Publication date: 13 July 2022

Published in: IEEE Transactions on Information Theory (Search for Journal in Brave)

Abstract: We study an information analogue of infinitely divisible probability distributions, where the i.i.d. sum is replaced by the joint distribution of an i.i.d. sequence. A random variable $X$ is called informationally infinitely divisible if, for any $nge1$, there exists an i.i.d. sequence of random variables $Z_1,ldots,Z_n$ that contains the same information as $X$, i.e., there exists an injective function $f$ such that $X=f(Z_1,ldots,Z_n)$. While there does not exist informationally infinitely divisible discrete random variable, we show that any discrete random variable $X$ has a bounded multiplicative gap to infinite divisibility, that is, if we remove the injectivity requirement on $f$, then there exists i.i.d. $Z_1,ldots,Z_n$ and $f$ satisfying $X=f(Z_1,ldots,Z_n)$, and the entropy satisfies $H(X)/nleH(Z_1)le1.59H(X)/n+2.43$. We also study a new class of discrete probability distributions, called spectral infinitely divisible distributions, where we can remove the multiplicative gap $1.59$. Furthermore, we study the case where $X=(Y_1,ldots,Y_m)$ is itself an i.i.d. sequence, $mge2$, for which the multiplicative gap $1.59$ can be replaced by $1+5sqrt(logm)/m$. This means that as $m$ increases, $(Y_1,ldots,Y_m)$ becomes closer to being spectral infinitely divisible in a uniform manner. This can be regarded as an information analogue of Kolmogorov's uniform theorem. Applications of our result include independent component analysis, distributed storage with a secrecy constraint, and distributed random number generation.

Full work available at URL: https://arxiv.org/abs/2008.06092

zbMATH Keywords

information analogue of infinitely divisible probability distributions

Mathematics Subject Classification ID

Probability distributions: general theory (60E05) Measures of information, entropy (94A17)

Related Items (1)

Unnamed Item

Recommendations

• Unnamed Item 👍 👎
• The limits of information 👍 👎
• Information-theoretic incompleteness 👍 👎
• Infinite-dimensional divergence information analysis 👍 👎
• Infinite Shannon entropy 👍 👎
• Finite Self-Information 👍 👎
• Information-Theoretic Incompleteness 👍 👎
• Statistical estimation of the structure of a finite population 👍 👎

This page was built for publication: Infinite Divisibility of Information