Average cost temporal-difference learning
type: Item
instance of: scholarly article
title: Average cost temporal-difference learning
zbMATH Open document ID: 0932.93085
DOI: 10.1016/S0005-1098(99)00099-0
author: John N. Tsitsiklis
author: Benjamin Van Roy
published in: Automatica
publication date: 2000-02-28T00:00:00Z
review text: The authors propose a variant of temporal-difference learning that approximates average and differential costs of an irreducible aperiodic Markov chain. Approximations are performed through linear combinations of fixed basis functions whose weights are incrementally updated during a single endless trajectory of the Markov chain. In addition, a proof of convergence and a characterization of the limit of convergence are presented. A bound on the resulting approximation error that exhibits an interesting dependence on ``mixing time'' of the Markov chain is provided.
reviewed by: Chengshu Wang
Mathematics Subject Classification ID: 93E20
Mathematics Subject Classification ID: 90C39
Mathematics Subject Classification ID: 49L20
Mathematics Subject Classification ID: 93E35
zbMATH DE Number: 1355384
zbMATH Keywords: dynamic programming
zbMATH Keywords: learning
zbMATH Keywords: average cost
zbMATH Keywords: aperiodic Markov chain
zbMATH Keywords: convergence
zbMATH Keywords: mixing time
MaRDI profile type: Publication
Recommended article: On average versus discounted reward temporal-difference learning
Recommended article: Learning algorithms for Markov decision processes with average cost
Recommended article: Reinforcement learning based algorithms for average cost Markov decision processes
Recommended article: Reinforcement learning for long-run average cost.
Recommended article: Model-based average reward reinforcement learning
Recommended article: Kernel-based reinforcement learning in average-cost problems
Recommended article: Differential Temporal Difference Learning
Recommended article: Q5477863
Recommended article: Q5477859
Recommended article: Linear least-squares algorithms for temporal difference learning
description: scientific article; zbMATH DE number 1355384