Two-phase iteration for value function approximation and hyperparameter optimization in Gaussian-kernel-based adaptive critic design
type: Item
instance of: scholarly article
title: Two-phase iteration for value function approximation and hyperparameter optimization in Gaussian-kernel-based adaptive critic design
zbMATH Open document ID: 1395.62087
DOI: 10.1155/2015/760459
author: Y. Aharonov
published in: Mathematical Problems in Engineering
publication date: 2018-08-27T00:00:00Z
review text: Summary: Adaptive Dynamic Programming (ADP) with critic-actor architecture is an effective way to perform online learning control. To avoid the subjectivity in the design of a neural network that serves as a critic network, kernel-based adaptive critic design (ACD) was developed recently. There are two essential issues for a static kernel-based model: how to determine proper hyperparameters in advance and how to select right samples to describe the value function. They all rely on the assessment of sample values. Based on the theoretical analysis, this paper presents a two-phase simultaneous learning method for a Gaussian-kernel-based critic network. It is able to estimate the values of samples without infinitively revisiting them. And the hyperparameters of the kernel model are optimized simultaneously. Based on the estimated sample values, the sample set can be refined by adding alternatives or deleting redundances. Combining this critic design with actor network, we present a Gaussian-kernel-based Adaptive Dynamic Programming (GK-ADP) approach. Simulations are used to verify its feasibility, particularly the necessity of two-phase learning, the convergence characteristics, and the improvement of the system performance by using a varying sample set.
Mathematics Subject Classification ID: 62G08
Mathematics Subject Classification ID: 68T05
Mathematics Subject Classification ID: 90C39
zbMATH DE Number: 6927173
MaRDI profile type: Publication
full work available at URL: https://doi.org/10.1155/2015/760459
OpenAlex ID: W1638131409
Wikidata QID: Q59119827
cites work: Reinforcement Learning and Feedback Control: Using Natural Decision Methods to Design Optimal Adaptive Controllers
cites work: Reinforcement learning algorithms with function approximation: recent advances and applications
cites work: Optimal control of unknown nonaffine nonlinear discrete-time systems based on adaptive dynamic programming
cites work: Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems
cites work: An iterative adaptive dynamic programming algorithm for optimal control of unknown discrete-time nonlinear systems with constrained inputs
cites work: Q4261789
cites work: Q3394879
cites work: The Kernel Recursive Least-Squares Algorithm
cites work: Kernel methods in machine learning
cites work: Q4421713
description: scientific article; zbMATH DE number 6927173