A limit theorem for the entropy density of nonhomogeneous Markov information source
type: Item
instance of: scholarly article
title: A limit theorem for the entropy density of nonhomogeneous Markov information source
zbMATH Open document ID: 0833.60034
DOI: 10.1016/0167-7152(94)00080-R
published in: Statistics \& Probability Letters
publication date: 1995-04-23T00:00:00Z
review text: A variant of strong law of large numbers for relative entropy density which holds for an arbitrary nonhomogeneous Markov information source with the alphabet \(S = \{1,2, \dots, m\}\) is proved. This is an important question in the information theory. If \(\{X_n, n \geq 0\}\) is a sequence of successive letters produced by mentioned source with initial distribution \((p(1), p(2), \dots, p(m))\) and transition matrix \(P_n = (p_n (i,j))\), \(i, j \in S\), \(n \geq 1\), where \(p_n(i,j) = P(X_n = j\mid X_{n-1} = i)\), then relative density is defined as \[ f_n(\omega) = -(1/n) \biggl[\log p (X_0) + \sum^n_{k=1} \log p_k(X_{k-1}, X_k) \biggr] \] and entropy of initial distribution is defined as \[ H(p_1, p_2,\dots, p_n) = -\sum^n_{k=1} p_k \log p_k, \] where log is natural logarithm. The main result is that almost sure \[ \lim_{n \to \infty} \{f_n(\omega) - (1/n) H [p_k (X_{k-1},1), \dots, p_k(X_{k-1},m)]\} = 0. \]
Mathematics Subject Classification ID: 60F15
Mathematics Subject Classification ID: 60J99
Mathematics Subject Classification ID: 94A17
zbMATH DE Number: 739593
zbMATH Keywords: strong law of large numbers
zbMATH Keywords: relative entropy density
zbMATH Keywords: entropy of initial distribution
author: Wen Liu
author: Weiguo Yang
reviewed by: Andrei Volodin
MaRDI profile type: Publication
cites work: The strong ergodic theorem for densities: Generalized Shannon-McMillan- Breiman theorem
cites work: Q3794956
cites work: The Individual Ergodic Theorem of Information Theory
cites work: A Note on the Ergodic Theorem of Information Theory
cites work: A counterexample to Perez's generalization of the Shannon-McMillan theorem
cites work: Relative entropy densities and a class of limit theorems of the sequence of m-valued random variables
cites work: The Basic Theorems of Information Theory
cites work: A Mathematical Theory of Communication
full work available at URL: 0167-7152(94)00080-r
OpenAlex ID: W1996682477
Recommended article: Q4343054
Recommended article: Q4516627
Recommended article: The Asymptotic Equipartition Property for a Nonhomogeneous Markov Information Source
Recommended article: Q4516718
Recommended article: The central limit theorem for \(m\)th-order nonhomogeneous Markov information source
Recommended article: Q5436212
Recommended article: The Asymptotic Equipartition Property for&lt;tex&gt;$M$&lt;/tex&gt;th-Order Nonhomogeneous Markov Information Sources
Recommended article: Q4801945
description: scientific article; zbMATH DE number 739593