Free energy, value, and attractors
type: Item
instance of: scholarly article
title: Free energy, value, and attractors
zbMATH Open document ID: 1422.91632
DOI: 10.1155/2012/937860
author: Ping Ao
author: Karl J. Friston
published in: Computational \& Mathematical Methods in Medicine
publication date: 2012-03-13T00:00:00Z
review text: Summary: It has been suggested recently that action and perception can be understood as minimising the free energy of sensory samples. This ensures that agents sample the environment to maximise the evidence for their model of the world, such that exchanges with the environment are predictable and adaptive. However, the free energy account does not invoke reward or cost-functions from reinforcement-learning and optimal control theory. We therefore ask whether reward is necessary to explain adaptive behaviour. The free energy formulation uses ideas from statistical physics to explain action in terms of minimising sensory surprise. Conversely, reinforcement-learning has its roots in behaviourism and engineering and assumes that agents optimise a policy to maximise future reward. This paper tries to connect the two formulations and concludes that optimal policies correspond to empirical priors on the trajectories of hidden environmental states, which compel agents to seek out the (valuable) states they expect to encounter.
Mathematics Subject Classification ID: 91E40
Mathematics Subject Classification ID: 91E30
Mathematics Subject Classification ID: 92D50
Mathematics Subject Classification ID: 37D45
zbMATH DE Number: 6014158
zbMATH Keywords: action
zbMATH Keywords: perception
zbMATH Keywords: free energy minimization
zbMATH Keywords: adaptive behaviour
zbMATH Keywords: reinforcement-learning
zbMATH Keywords: attractors
Wikidata QID: Q39666481
MaRDI profile type: Publication
full work available at URL: https://doi.org/10.1155/2012/937860
OpenAlex ID: W1971715144
cites work: Equilibrium points in n -person games
cites work: On the Theory of Dynamic Programming
cites work: Quantal response equilibria for normal form games
cites work: Attractors for random dynamical systems
cites work: Proof of the Ergodic Theorem
cites work: Synergetics. An introduction. Nonequilibrium phase transitions and self-organization in physics, chemistry, and biology. 3rd rev. and enl. ed
cites work: Applications of centre manifold theory
cites work: The existence of generalized synchronization of chaotic systems in complex networks
cites work: Q4138569
cites work: Ergodicity of Flows on Homogeneous Spaces
cites work: Global random attractors are uniquely determined by attracting deterministic compact sets
cites work: On Information and Sufficiency
cites work: Generalised filtering
cites work: Potential in stochastic differential equations: novel construction
cites work: Emerging of Stochastic Dynamical Equalities and Steady State Thermodynamics from Darwinian Dynamics
cites work: Discrete Choice Methods with Simulation
cites work: Q4084463
cites work: A Stochastic Approximation Method
cites work: Q3997575
cites work: \({\mathcal Q}\)-learning
cites work: A theoretical model of phase transitions in the human brain
cites work: Parameter Estimation of Sigmoid Superpositions: Dynamical System Approach
cites work: Invariant template matching in systems with spatiotemporal coding: a matter of instability
cites work: Q3901427
cites work: Dynamic noncooperative game theory
cites work: Q3093215
description: scientific article; zbMATH DE number 6014158