The relation between information theory and the differential geometry approach to statistics
type: Item
instance of: scholarly article
title: The relation between information theory and the differential geometry approach to statistics
zbMATH Open document ID: 0609.62006
DOI: 10.1016/0020-0255(85)90050-7
published in: Information Sciences
publication date: 1985-01-01T00:00:00Z
review text: \textit{N. N. Chentsov} [Statistical decision rules and optimal inference. Transl. Math. Monogr. 53 (1982; Zbl 0484.62008)] has shown that the Riemannian metric on the probability simplex \(\sum x_ i=1\) defined by \((ds)^ 2=\sum (dx_ i)^ 2/x_ i\) has an invariance property under certain probabilistically natural mappings. No other Riemannian metric has the same property. The geometry associated with this metric is shown to lead almost automatically to measures of divergence between probability distributions which are associated with Kullback, Bhattacharyya, and Matusita. Certain vector fields are associated in a natural way with random variables. The integral curves of these vector fields yield the maximum entropy or minimum divergence estimates of probabilities. Some other consequences of this geometric view are also explored.
Mathematics Subject Classification ID: 62B10
Mathematics Subject Classification ID: 62A01
Mathematics Subject Classification ID: 53A99
Mathematics Subject Classification ID: 94A17
zbMATH DE Number: 3986359
zbMATH Keywords: information theory
zbMATH Keywords: information measures
zbMATH Keywords: Fisher information matrix
zbMATH Keywords: Bhattacharyya distance
zbMATH Keywords: Kullback distance
zbMATH Keywords: Matusita distance
zbMATH Keywords: Riemannian metric
zbMATH Keywords: probability simplex
zbMATH Keywords: measures of divergence
zbMATH Keywords: vector fields
zbMATH Keywords: maximum entropy
zbMATH Keywords: minimum divergence estimates
Wikidata QID: Q115368296
author: L. Lorne Campbell
MaRDI profile type: Publication
full work available at URL: 0020-0255(85)90050-7
OpenAlex ID: W2052608203
cites work: On measures of information and their characterizations
cites work: Differential geometry of curved exponential families. Curvatures and information loss
cites work: Q5841953
cites work: The Renyi redundancy of generalized Huffman codes
cites work: Entropy differential metric, distance and divergence measures in probability spaces: A unified approach
cites work: An Extended Cencov Characterization of the Information Metric
cites work: A coding theorem and Rényi's entropy
cites work: Q3943797
cites work: Defining the curvature of a statistical problem (with applications to second order efficiency)
cites work: Q5595624
cites work: Accounting for intrinsic nonlinearity in nonlinear regression parameter inference regions
cites work: Q5341405
cites work: Information geometry in functional spaces of classical and quantum finite statistical systems
cites work: Q5529067
cites work: Decision Rules, Based on the Distance, for Problems of Fit, Two Samples, and Estimation
cites work: Decision rule, based on the distance, for the classification problem
cites work: Q5845474
cites work: A new mathematical framework for the study of linkage and selection
Recommended article: Information geometry and sufficient statistics
Recommended article: Information geometry and its applications
Recommended article: Computational Information Geometry in Statistics: Foundations
Recommended article: Information geometry of Bayesian statistics
Recommended article: Q4341419
Recommended article: Information geometry and statistical manifold.
Recommended article: Information Geometry and Its Applications: Survey
Recommended article: Information geometry and Plefka's mean-field theory
Recommended article: Q3316376
Recommended article: Information geometry and asymptotic geodesics on the space of normal distributions
description: scientific article; zbMATH DE number 3986359