A sandwich proof of the Shannon-McMillan-Breiman theorem
type: Item
instance of: scholarly article
title: A sandwich proof of the Shannon-McMillan-Breiman theorem
zbMATH Open document ID: 0653.28013
author: Paul H. Algoet
author: Thomas M. Cover
published in: The Annals of Probability
publication date: 1988-01-01T00:00:00Z
review text: Let \(X_ 0,X_ 1,..\). be a stationary ergodic process with Polish state space S. Let M be a reference measure on the space of all one-sided infinite sequences from S. Suppose that M is finite order Markov with stationary transition kernel. Further, suppose that for each positive integer n, the distribution \(P_ n\) of \((X_ 0,X_ 1,...,X_{n-1})\) is absolutely continuous with respect to the marginal distribution \(M_ n\) that one obtains from M by projecting on the first n coordinates. Let \(\{f_ n\}\) be the sequence of random variables \(f_ n=(dP_ n/dM_ n)(X_ 0,x_ 1,...,X_{n-1}),\quad n=1,2,...\quad.\) It is known that the sequence \(\{n^{-1} \log f_ n\}\) converges almost surely. (This result is a generalization of the Shannon-McMillan-Breiman theorem.) The authors provide a new proof of this result which involves an ingenious ``sandwiching'' of the limit superior and limit inferior of \(\{n^{-1} \log f_ n\}\) between two random variables that have the same expected value.
Mathematics Subject Classification ID: 28D05
Mathematics Subject Classification ID: 94A17
Mathematics Subject Classification ID: 60F15
zbMATH DE Number: 4065355
zbMATH Keywords: ergodic theorem of information theory
zbMATH Keywords: relative entropy rate
zbMATH Keywords: stationary ergodic process
zbMATH Keywords: Polish state space
zbMATH Keywords: reference measure
zbMATH Keywords: marginal distribution
zbMATH Keywords: generalization of the Shannon-McMillan-Breiman theorem
zbMATH Keywords: sandwiching
MaRDI profile type: Publication
full work available at URL: https://doi.org/10.1214/aop/1176991794
OpenAlex ID: W2033672007
DOI: 10.1214/AOP/1176991794
description: scientific article; zbMATH DE number 4065355