Analysis of an identification algorithm arising in the adaptive estimation of Markov chains
type: Item
instance of: scholarly article
title: Analysis of an identification algorithm arising in the adaptive estimation of Markov chains
zbMATH Open document ID: 0685.93063
DOI: 10.1007/BF02551353
author: Ari Arapostathis
author: Steven I. Marcus
published in: MCSS. Mathematics of Control, Signals, and Systems
publication date: 1990-01-01T00:00:00Z
review text: The paper is devoted to the adaptive estimation problem for partially observable finite-state Markov chains. The authors describe an algorithm which utilizes the recursive equation characterizing the conditional distribution of the state of the Markov chain, given the past observations. Several important analytical properties of this algorithm are studied. In particular, an interesting connection is established with the ordinary differential equation method for stochastic approximations. This allows to give a deep analysis of the algorithms suggested here. We find in the paper clearly formulated statements as well as their detailed proofs. Some useful related topics are also discussed.
Mathematics Subject Classification ID: 93E10
Mathematics Subject Classification ID: 93C40
Mathematics Subject Classification ID: 60J10
zbMATH DE Number: 4123647
zbMATH Keywords: adaptive estimation
zbMATH Keywords: finite-state Markov chains
zbMATH Keywords: stochastic approximations
Wikidata QID: Q60167569
reviewed by: Jordan Stoyanov
MaRDI profile type: Publication
cites work: Optimal control of Markov processes with incomplete state information
cites work: Stochastic control of two partially observed competing queues
cites work: Gradient approach for recursive estimation and control in finite Markov chains
cites work: Products of Random Matrices
cites work: Discrete-time multivariable adaptive control
cites work: Adaptive control of service in queueing systems
cites work: Optimal adaptive control of priority assignment in queueing systems
cites work: A general Martingale approach to discrete-time stochastic control and estimation
cites work: Q5590550
cites work: A limit theorem for partially observed Markov chains
cites work: A limit theorem for Markov chains in compact metric spaces with applications to products of random matrices
cites work: Identification of a transition matrix of a Markov chain from noisy measurements of state
cites work: The average-optimal adaptive control of a Markov renewal model in presence of an unknown parameter
cites work: A Survey of Some Results in Stochastic Adaptive Control
cites work: Stochastic approximation with discontinuous dynamics and state dependent noise: w.p. 1 and weak convergence
cites work: Q3344923
cites work: Stochastic approximation methods for constrained and unconstrained systems
cites work: An Invariant Measure Approach to the Convergence of Stochastic Approximations with State Dependent Noise
cites work: Théorèmes de convergence presque sure pour une classe d'algorithmes stochastiques à pas decroissant
cites work: Estimation and control in discounted stochastic dynamic programming
cites work: Convergence results for continuous-time adaptive stochastic filtering algorithms
cites work: Non-negative matrices and Markov chains. 2nd ed
full work available at URL: https://doi.org/10.1007/bf02551353
OpenAlex ID: W2324831421
Recommended article: Q3754533
Recommended article: On the identification of continuous-time Markov chains with a given invariant measure
Recommended article: Q3827909
Recommended article: Adaptive estimation for bifurcating Markov chains
Recommended article: Adaptive estimation of the transition density of a Markov chain
Recommended article: Q5369208
Recommended article: Adaptive estimation of the transition density of a regular Markov chain
Recommended article: Markov approximation jointly with identification of a stochastic object
description: scientific article; zbMATH DE number 4123647