MM optimization algorithms (Q2826638)

Solving optimization problems iteratively by sequentially replacing the original problem by simpler auxiliary optimization problems, obtained e.g. by replacing the objective and constraint functions by simpler functions, is a well known standard method in optimization.NEWLINENEWLINEHere, optimization procedures, called ``MM algorithms'', are considered for solving optimization problems iteratively by approximating first the objective function at each iteration point \(x_n\) from above, below, resp., and minimizing, maximizing then the upper, lower, resp., approximate objective function. Hence, ``MM'' means ``majorization-minimization'' for a minimization problem and ``minorization-maximization'' for a maximization problem. Related approximation and optimization methods have been suggested already earlier in the literature on operations research and stochastic optimization, as e.g. the construction of convex, concave, resp., tangent functions to the objective function at each iteration point \(x_n\). Because of the joint gradient of the objective function and its approximation, and the coinciding function values at \(x_n\), descent, ascent directions, resp., may be obtained and corresponding algorithms can be constructed. Further known A\&O methods are based on second order approximation of functions, inner linearization of composed objective functions and on the construction of efficient/stationary points.NEWLINENEWLINEAfter presenting first examples for optimization by means of majorization, minorization of the objective function in Chapter 1, in the following chapters tools from calculus and optimization theory are examined for finding appropriate approximations of functions: Convex sets and convex functions, basic inequalities, applications of different types of differentials in optimization, etc., are reviewed in Chapters 2 and 3. A selection of available techniques for majorization and minorization of functions are given in Chapter 4. Applications of the projection technique to the treatment of constrained optimization problems as well as applications of proximal maps to the approximation method under consideration can be found in Chapter 5. Applications of the MM approximation and optimization procedure to problems in regression and multivariate analysis are discussed in Chapter 6. Convergence properties of the proposed method are studied in the last Chapter 7. Each chapter contains many problems for solving by the reader, furthermore, many references are given. Readers of this interesting book need good knowledge in calculus, linear algebra and optimization theory and its applications.