The entropy of a discrete real variable (Q406139)

Summary: The discrete Shannon entropy \(H\) was formulated only to measure indeterminacy effected through a set of probabilities, but the indeterminacy in a real-valued discrete variable depends on both the allowed outcomes \(\mathbf x\) and the corresponding probabilities \(\mathbf p\). A fundamental measure that is sensitive to both \(\mathbf x\) and \(\mathbf p\) is derived here from the total differential entropy of a continuous real variable and its conjugate in the discrete limit, where the conjugate is universally eliminated. The asymptotic differential entropy recovers \(H\) plus the new measure, named \(\Xi\), which provides a novel probe of intrinsic organization in sequences of real numbers.