Maximum entropy fundamentals
type: Item
instance of: scholarly article
title: Maximum entropy fundamentals
zbMATH Open document ID: 1006.94007
author: P. Harremoeës
author: Flemming Topsøe
published in: Entropy
publication date: 2002-09-10T00:00:00Z
full work available at URL: list01.htm
review text: Summary: In its modern formulation, the maximum entropy principle was promoted by E. T. Jaynes, starting in the mid-fifties. The principle dictates that one should look for a distribution, consistent with available information, which maximizes the entropy. However, this principle focuses only on distributions and it appears advantageous to bring information theoretical thinking more prominently into play by also focusing on the ``observer'' and on coding. This view was brought forward by the second named author in the late seventies and is the view we follow here. It leads to the consideration of a certain game, the code length game, and, via standard game theoretical thinking, to a principle of game theoretical equilibrium. This principle is more basic than the maximum entropy principle in the sense that the search for one type of optimal strategies in the code length game translates directly into the search for distributions with maximum entropy. In the present paper we offer a self-contained and comprehensive treatment of fundamentals of both principles mentioned based on a study of the code length game. Though new concepts and results are presented, the reading should be instructional and accessible to a rather wide audience, at least if certain mathematical details are left aside at a first reading. The most frequently studied instance of entropy maximization pertains to the mean energy model, which involves a moment constraint related to a given function, here taken to represent ``energy''. This type of application is very well known from the literature with hundreds of applications pertaining to several different fields and will also serve here as an important illustration of the theory. But our approach reaches further, especially regarding the study of continuity properties of the entropy function, and this leads to new results which allow for a discussion of models with so-called entropy loss. These results have tempted us to speculate over the development…
Mathematics Subject Classification ID: 94A17
Mathematics Subject Classification ID: 91F20
zbMATH DE Number: 1796727
zbMATH Keywords: maximum entropy
zbMATH Keywords: minimum risk
zbMATH Keywords: game theoretical equilibrium
zbMATH Keywords: information topology
zbMATH Keywords: Nash equilibrium code
zbMATH Keywords: entropy loss
zbMATH Keywords: partition function
zbMATH Keywords: exponential family
zbMATH Keywords: continuity of entropy
zbMATH Keywords: hyperbolic distributions
zbMATH Keywords: Zipf's law
zbMATH Keywords: development of natural languages
MaRDI profile type: Publication
OpenAlex ID: W2050249319
Wikidata QID: Q56157779
DOI: 10.3390/E3030191
description: scientific article; zbMATH DE number 1796727