Reinforcement learning for optimal feedback control. A Lyapunov-based approach
type: Item
instance of: scholarly article
title: Reinforcement learning for optimal feedback control. A Lyapunov-based approach
zbMATH Open document ID: 1403.49001
DOI: 10.1007/978-3-319-78384-0
author: Rushikesh Kamalapurkar
author: Patrick Walters
author: Joel A. Rosenfeld
author: Warren E. Dixon
published in: Communications and Control Engineering
publication date: 2018-07-04T00:00:00Z
review text: The book is devoted to design problems related to development of approximate optimal controllers based on reinforcement learning-based solutions and related decision making problems. All considerations are based on rigorous mathematical treatment including optimal control, stability theory and convergence of design procedures. The authors discuss both model-free and model-based methods. All considerations related to the optimal control and optimal decision making are based on the principle of optimality and dynamic programming techniques. Since these techniques lead to the Hamiltonian-Jacobi-Bellman equations whose solutions could be usually found only numerically, the authors concentrate on approximate dynamic programming algorithms which use a parametric approximation of the optimal policy and/or the optimal value function. One way to implement the approximate iteration algorithms is based on adaptive-critic-based reward system [\textit{R. S. Sutton} and \textit{A. G. Barto}, Reinforcement learning: an introduction. Cambridge: MIT Press (1998); \url{https://mitpress.mit.edu/books/reinforcement-learning}] used for reinforcement learning. The book starts with brief introduction to optimal control followed by review of techniques related to exact and approximate dynamic programming. Their relations with Lyapunov-based stability analysis and differential games are also presented. Then, the authors discuss problem of adaptive online approximate optimal control of uncertain nonlinear systems. A new adaptive -- critic -- identifier architecture is proposed and it is based on a persistence of excitation-based online learning schemes. Some similarities with the Werbos approach [\textit{P. J. Werbos}, Lect. Notes Comput. Sci. Eng. 50, 15--34 (2006; Zbl 1270.65015)] to the heuristic dynamic programming could be found. Then, the authors introduce a concurrent learning to solve approximate optimal control problems under a finite excitation condition. The method is extended to …
Mathematics Subject Classification ID: 49-02
Mathematics Subject Classification ID: 49L20
Mathematics Subject Classification ID: 93B52
Mathematics Subject Classification ID: 93B40
Mathematics Subject Classification ID: 93B51
Mathematics Subject Classification ID: 93D05
Mathematics Subject Classification ID: 93D15
Mathematics Subject Classification ID: 91A23
Mathematics Subject Classification ID: 49N90
Mathematics Subject Classification ID: 92B20
zbMATH DE Number: 6898569
zbMATH Keywords: optimal control
zbMATH Keywords: reinforcement learning
zbMATH Keywords: Lyapunov-based approach
zbMATH Keywords: approximate optimal controllers
zbMATH Keywords: approximate dynamic programming
zbMATH Keywords: HJB equations
zbMATH Keywords: differential games
zbMATH Keywords: autonomous vehicles
zbMATH Keywords: universal reproducing kernels
reviewed by: Andrzej Świerniak
MaRDI profile type: Publication
full work available at URL: https://doi.org/10.1007/978-3-319-78384-0
OpenAlex ID: W2800330509
description: scientific article; zbMATH DE number 6898569