Collinear gradients method for minimizing smooth functions. Optimality conditions and algorithms (Q6155635)

This study introduces an innovative approach to minimizing a continuous multivariate function, denoted as \(J(\mathbf{x})\), exclusively relying on gradient information. The core concept revolves around the observation that in the case of a quadratic function, specifically \(J(\mathbf{x}) = (\mathbf{x} - \overline{\mathbf{x}}) \cdot \mathbf{A}(\mathbf{x} - \overline{\mathbf{x}})\), where \(\overline{\mathbf{x}}\) represents the point of minimum, all gradients \(\nabla J(\mathbf{x})\) along the trajectory leading to the minimum point \(\overline{\mathbf{x}}\) are parallel. Therefore, the primary emphasis of this study is on developing an efficient method for approximating a direction from an initial point where all gradients align. The computed direction corresponds to the Newton direction, enabling the achievement of a solution in just one step for the specific scenario of a quadratic function with an exact search direction. The computation of this direction involves solving a more intricate problem compared to approximating the Hessian matrix using finite differences or the BFGS method. However, the use of the Fletcher-Reeves conjugate gradient method proves adequate for this purpose. A limited number of numerical tests conducted on standard numerical problems demonstrate that, in certain circumstances, the proposed method can rival the commonly employed variants of the Newton method.