Some observations on the concepts of information-theoretic entropy and randomness
type: Item
instance of: scholarly article
title: Some observations on the concepts of information-theoretic entropy and randomness
zbMATH Open document ID: 1004.94009
author: J. D. H. Smith
published in: Entropy
publication date: 2002-09-10T00:00:00Z
full work available at URL: list01.htm
review text: Summary: Certain aspects of the history, derivation, and physical application of the information-theoretic entropy concept are discussed. Pre-dating Shannon, the concept is traced back to Pauli. A derivation from first principles is given, without use of approximations. The concept depends on the underlying degree of randomness. In physical applications, this translates to dependence on the experimental apparatus available. An example illustrates how this dependence affects Prigogine's proposal for the use of the Second Law of Thermodynamics as a selection principle for the breaking of time symmetry. The dependence also serves to yield a resolution of the so-called ``Gibbs paradox''. Extension of the concept from the discrete to the continuous case is discussed. The usual extension is shown to be dimensionally incorrect. Correction introduces a reference density, leading to the concept of Kullback entropy. Practical relativistic considerations suggest a possible proper reference density.
Mathematics Subject Classification ID: 94A17
zbMATH DE Number: 1796688
zbMATH Keywords: information-theoretic entropy
zbMATH Keywords: randomness
zbMATH Keywords: Kullback entropy
MaRDI profile type: Publication
OpenAlex ID: W2171838068
DOI: 10.3390/E3010001
description: scientific article; zbMATH DE number 1796688