Asymptotics of the Shannon and Renyi entropies for sums of independent random variables
type: Item
instance of: scholarly article
title: Asymptotics of the Shannon and Renyi entropies for sums of independent random variables
zbMATH Open document ID: 0921.62005
published in: Problems of Information Transmission
publication date: 1999-10-04T00:00:00Z
review text: Let \(X_1,X_2,\dots\) be i.i.d. integer valued random variables with finite variance. Let \break \(| E\exp(itX_1) |<1\), \(t\neq 0\). The authors study the Shannon and Renyi entropies \[ H_n=-\sum_n P_n(k)\log P_n(k)\quad\text{and}\quad(1-\alpha)^{-1}\log\sum_k(P_n (k))^\alpha,\;\alpha>1, \] of \(X_1+\cdots +X_n\) as \(n\to\infty\), where \(P_n(k)=P(X_1+ \cdots+X_n=k)\). It is shown from the local limit theorem for \(P_n(k)\), that \(H_n=2^{-1}\log n+O(1)\) and \(H_n (\alpha)= 2^{-1}\log n+O(1)\). When \(E| X_1|^N <\infty\) with \(N\geq 3\), asymptotic expansions of the form \(2^{-1}\log n+\sum_k C_kn^{-k}\) with \(2k\leq N-2\) are proved by hard estimates, using the Edgeworth expansion of \(P_n(k)\). The \(c_k\) depend on the semi-invariants of \(X_1\), with \(c_0=2^{-1} \log(2\pi e \sigma^2)\) for Shannon and \(c_0=2^{-1}\log (2\pi\sigma^2) +2^{-1} (\alpha-1)^{-1} \log\alpha\) for Renyi, \(c_1\) and \(c_2\) are computed generally and some \(c_k\) with \(k\geq 3\) for special distributions. The corresponding theorems for absolutely continuous distributions are stated.
Mathematics Subject Classification ID: 62B10
Mathematics Subject Classification ID: 60G50
Mathematics Subject Classification ID: 62E20
Mathematics Subject Classification ID: 60F99
Mathematics Subject Classification ID: 94A17
zbMATH DE Number: 1274202
zbMATH Keywords: entropy
zbMATH Keywords: local limit theorem
author: Arkadii G. D'yachkov
author: Pavel A. Vilenkin
MaRDI profile type: Publication
description: scientific article; zbMATH DE number 1274202