Optimal control theory. Applications to management science and economics (Q2064032)

The author of this book is an expert in operations management, finance and economics, marketing, optimization, optimal control, etc. The first two editions of the book [Optimal control theory. Applications to management science. Boston/The Hague/London: Martinus Nijhoff Publishing (1981; Zbl 0495.49001); Optimal control theory. Applications to management science and economics. 2nd ed. Dordrecht: Kluwer Academic Publishers (2000; Zbl 0998.49002); Optimal control theory. Applications to management science and economics. 3rd edition. Cham: Springer (2019; Zbl 1412.49001)] were published by the same author in cooperation with \textit{G. L. Thompson}, and naturally the present fourth edition includes the contributions of both experts. We also point out that the material of this book has been discussed with students, since its level allows to do so. Obviously, the book is primarily addressed to students and researchers in management science, operations research, and economics. The book comprises 13 chapters, 5 appendices, a long reference list, and an index (plus lists of figures and tables). Chapter 1 (entitled ``What is Optimal Control Theory?'') comprises 5 sections: 1.1 Basic Concepts and Definitions; 1.2 Formulation of Simple Control Models; 1.3 History of Optimal Control Theory; 1.4 Notation and Concepts Used; 1.5 Plan of the Book. In addition, at the end of Chapter 1 there are appropriate exercises and a specific reference list. Chapter 2 (``The Maximum Principle: Continuous Time'') introduces the maximum principle as a necessary condition that must be satisfied by any optimal control for the basic problem formulated in Section 2.1 of this chapter. Specifically, the state equation considered here is an ODE in the Euclidean space \(E^n\), \[ \dot{x}(t)=f(x(t),u(t),t), \ t\in [0,T], \ \ x(0)=x_0, \] where \(u(t) \in E^m\) is the control variable. The admissible controls are assumed to be piecewise continuous functions \(u\) satisfying a constraint of the form \(u(t)\in \Omega (t), \ t\in [0,T]\). The optimal control is defined to be an admissible control which maximizes a so-called objective function \[ J=\int_0^TF(x(t),u(t),t)\, dt + S(x(T),T), \] where \(F\) and \(S\) are here assumed to be continuously differentiable. So the \emph{optimal control problem} (OCP) is to find an admissible control \(u^*\) which maximizes \(J\) over all admissible controls \(u\) and all \(x\) satisfying the state equation above and the initial condition \(x(0)=x_0\). This OCP is said to be in \emph{Bolza form}. If \(S=0\) then the OCP is in \emph{Lagrange form}, while if \(F=0\) it is said to be in \emph{Mayer form}. An OCP in Mayer form with \(S= cx(T)\), where \(c\) is a given row vector, \(c=(c_1,c_2,\dots,c_n)\), is said to be in \emph{linear Mayer form}. In fact, any Bolza problem can be reduced to a linear Mayer problem, on the expense of introducing an additional scalar state equation. Then, in Section 2.2, the maximum principle is derived by using the dynamic programming approach, and an economic interpretation is provided. Some examples (with solutions) are discussed in Section 2.3. In Section 2.4 a result on sufficiency conditions are stated, and the fixed-end point problems are addressed. Then, Section 2.5 is devoted to solving a two-point boundary value problem by using Excel. And, at the end of the chapter, the author proposes a lot of exercises and provides an adequate reference list. In Chapter 3 (``The Maximum Principle: Mixed Inequality Constraints'') the author presents the case of inequality constraints involving control and possibly state variables. In Section 3.1 a Lagrangian form of the maximum principle is discussed for models in which some constraints involve only control variables, and others involve both state and control variables. In Section 3.2 the author states conditions under which the Lagrangian maximum principle is also sufficient for optimality. In Section 3.3 the author considers the special case \(F(x,u,t)=\phi (x,u)e^{-pt}\), \(S(x,T)=\psi (x)e^{-pT}\), which occurs in most management science and economic problems. The maximum principle is stated and a specific example is analyzed. The next sections of Chapter 3 are the following: Section 3.4 (``Transversality Conditions: Special Cases''); Section 3.5 (``Free Terminal Time Problems''); Section 3.6 (``Infinite Horizon and Stationarity''); Section 3.7 (``Model Types''). As usual, the chapter ends with many specific exercises and references. Chapter 4 (``The Maximum Principle: Pure State and Mixed Inequality Constraints''). As the author says, intuitively, pure state constraints are difficult to deal with because only the control variables are under the direct influence of the decision maker. The case of pure state constraints can be handled by a direct method (by associating a multiplier with each constraint for appending it to the Hamiltonian to form the Lagrangian, and then proceeding in a similar way as in the previous chapter in the case of mixed constraints), or by an indirect method (when a pure constraint is active, one can constrain the value of its time derivative, which will involve time derivatives of the state variables; so, the restrictions on the time derivatives of the pure state constraints become mixed constraints and these are appended to the Hamiltonian to form the Lagrangian, and so on. The direct and indirect maximum principle is derived, and finally the author adds specific exercises and references. Chapters 5--7 are devoted to applications (in finance, production and inventory, marketing), which are very important for students and researchers oriented to management and economics. Chapter 8 (``The Maximum Principle: Discrete Time'') is focused on the case where time is represented by a discrete variable \(k=0, 1, \cdots, T\). The maximum principle is reduced to a nonlinear programming problem and the necessary conditions for its solution are stated by using the Kuhn-Tucker theorem. This procedure requires some simplifications and so only a restricted form of the discrete maximum principle is obtained. A more general discrete maximum principle is stated (without proof) in Section 8.3. Again, several illustrative examples, many exercises and references are included. In Chapter 9 (``Maintenance and Replacement'') the author presents some maintenance and replacement models, with solutions obtained by the maximum principle, and specific numerical examples. In addition, exercises and references are included. Chapters 10 and 11 are devoted to applications of optimal control theory to natural resources and to economics. Certainly, formulation of the corresponding models and their solutions are of interest for a large audience. Again, specific exercises are proposed and reference lists are added at the end of the two chapters. In Chapter 12 (``Stochastic Optimal Control'') the author deals with the case where the state equation is perturbed by a Wiener process (Brownian motion), which gives rise to the state as a Markov diffusion process. In Section 12.1, the author formulates an OCP governed by stochastic differential equations involving a Wiener process, also known as Ito equations. The goal is ``to synthesize optimal feedback controls for systems subject to Ito equations in a way that maximizes the expected value of a given objective function.'' Some practical stochastic models are analyzed, including an advertising model named after the author (see page 354). As usual some exercises and a reference list are added at the end of the chapter. Chapter 13 (``Differential Games'') is focused on the situation when there are more decision makers, each having one's own objective function to be maximized, subject to a set of differential equations. The extension of optimal control theory to such situations is called the theory of differential games, which is far more complex than the classic optimal control theory. We confine ourselves to mentioning the titles of the sections of this chapter: Two-person zero-sum differential games; Nash differential games; A feedback Nash stochastic differential game in advertising; A feedback Stackelberg stochastic differential game of cooperative advertising. Again, exercises and references are added. Some appendices are included to complete the exposition: A. Solutions of Linear Differential Equations; B. Calculus of Variations and Optimal Control Theory; C. An Alternative Derivation of the Maximum Principle; D. Special Topics in Optimal Control; E. Answers to Selected Exercises. Certainly, the book contains enough mathematical tools to solve plenty of practical problems in management science and economics.