Density-free convergence properties of various estimators of entropy
type: Item
instance of: scholarly article
title: Density-free convergence properties of various estimators of entropy
zbMATH Open document ID: 0632.62031
DOI: 10.1016/0167-9473(87)90065-X
author: László Györfi
author: Edward C. van der Meulen
published in: Computational Statistics and Data Analysis
publication date: 1987-01-01T00:00:00Z
review text: Let \(f(x)\) be a probability density function, \(x\in\mathbb R^d\). The Shannon (or differential) entropy is defined as \[ H(f)=-\int f(x)\log f(x)\,dx. \] In this paper we propose, based on a random sample \(X_1,\dots,X_n\) generated from \(f\), two new nonparametric estimators for \(H(f)\). Both entropy estimators are histogram-based in the sense that they involve a histogram-based density estimator \(\hat f_n\). We prove their a.s. consistency with the only condition on \(f\) that \(H(f)\) is finite.
Mathematics Subject Classification ID: 62G05
Mathematics Subject Classification ID: 62B10
Mathematics Subject Classification ID: 94A17
Mathematics Subject Classification ID: 60F15
Mathematics Subject Classification ID: 60F25
zbMATH DE Number: 4028627
zbMATH Keywords: density-free convergence properties
zbMATH Keywords: differential entropy
zbMATH Keywords: almost sure convergence
zbMATH Keywords: \(L_1\)-convergence
zbMATH Keywords: Shannon entropy
zbMATH Keywords: entropy estimators
zbMATH Keywords: histogram-based
zbMATH Keywords: histogram-based density estimator
zbMATH Keywords: consistency
MaRDI profile type: Publication
cites work: A nonparametric estimation of the entropy for absolutely continuous distributions (Corresp.)
cites work: Q5649740
cites work: Q5539510
cites work: Q5591992
cites work: Q4773598
cites work: Q3686615
cites work: Q3336497
cites work: On the Estimation of Functionals of the Probability Density and Its Derivatives
cites work: On a Class of Non-Parametric Estimates of Non-Linear Functionals of Density
cites work: Entropy-Based Tests of Uniformity
cites work: Q3768185
cites work: Q5595624
cites work: Q4191418
cites work: On entropy-based goodness-of-fit tests
cites work: Properties of the statistical estimate of the entropy of a random vector with a probability density
cites work: Information Theory and Statistical Mechanics
cites work: On the Optimum Rate of Transmitting Information
cites work: Q5529067
cites work: Q3330303
cites work: Q4113257
cites work: On the dimension and entropy of probability distributions
cites work: Large sample properties of maximum entropy histograms
cites work: Q3878557
cites work: Q4096256
cites work: On the entropy of continuous probability distributions (Corresp.)
full work available at URL: 0167-9473(87)90065-x
OpenAlex ID: W1963714278
description: scientific article; zbMATH DE number 4028627