Approximate iterative algorithms (Q2873813)

This is an excellent book on dynamic programming and Markov decision processes. Dynamic programming, invented by the late Richard Bellman, has created a new field of optimality and approximation theory.NEWLINENEWLINEThe author has divided his book into three parts: I: Mathematical background with 8 chapters, II: General theory of approximate iterative algorithms with 3 chapters, and III: Application to Markov decision processes with 6 chapters.NEWLINENEWLINEPart I covers real analysis and linear algebra, measure theory, probability theory, stochastic processes and functional analysis including fixed-point equations and the distribution of a maximum. Part II deals with linear convergence, the general theory of approximate iterative systems, and selection of approximate schedules. Part III contains chapters on Markov decision processes, value iteration, approximation in dynamic programming, sampling, iteration by truncation, grid approximation and adaptive control.NEWLINENEWLINEThe author has elaborated the theory in the application to online parameter estimation and exploration schedule.