Estimation by least squares (Q2757793)

The present lecture book considers estimations basing on the least square method. First works in this field were already done independently by C. F. Gauß (1795) and A. M.\ Legendre (1806) about two centuries ago. Nevertheless the method is also recently very often used to solve an overdetermined system of \(N\) equations to find \(n\) unknown parameters \(x_1\), \(x_2\), \dots, \(x_N\) knowing \(N\) measurements \(f_i(x_1\), \(x_2\), \dots, \(x_n)=y_i\), \(i=1\), 2, \dots, \(N\), \(n<N\). NEWLINENEWLINENEWLINEThe overdetermined system of equations cannot be solved exactly. Thus, one tries to find a solution \(x_1\), \(x_2\), \dots, \(x_N\) so that the sum of the squares of the differences \(d_i=l_i-f_i(x_1\), \(x_2\), \dots, \(x_n)\) of all \(N\) equations equals its minimum value. NEWLINENEWLINENEWLINESuch a mathematical problem occurs in all metrological sciences, e.g. in astrophysics and geodesy, but the least square method is also applied in evolution models of economy. NEWLINENEWLINENEWLINEIn the first chapter of the present book, basic mathematical and statistical knowledge is explained. Definitions and results of first and second order differential calculus are presented. Besides, properties of projection operators, especially of orthogonal projection in Euclidean coordinate systems (including its matrix presentation) are discussed. Further, the decomposition of positive definite matrices (Cholesky factorisation, matrix inversion) and some distribution functions of statistics are presented. NEWLINENEWLINENEWLINESection 2 considers optimizations by least squares, that means the principle of least square solutions and their geometrical interpretation. For conditional solutions of least squares normal equations are obtained. Elementary transformations of normal equations are discussed. NEWLINENEWLINENEWLINEChapter 3 shows the connection between the least square method and the maximum likelihood method of statistics. Conditional solutions and the case of complex parameters are considered. NEWLINENEWLINENEWLINEPractical problems occuring during the application of the linear least square method are studied in detail in Chapter 4. Detection of errors, tests of significance and validity, and multiple regressions are discussed. As examples the dependence of the revolution time of planets on their mean distance from the Sun and the calculation of linear correlation coefficients are explained. Mathematical formulations of problems satisfying conditions or constraints are compared and solved for the case of the calculation of the angles of an triangle. Besides, configurational problems of models are considered (degrees of freedom, prediction matrices). Further, the acceptibility of the dispersion of observations is studied. The method of the degree of freedom to estimate the components of the dispersion of series of incoherent and independent observations (method of F. R. Helmert) is explained. NEWLINENEWLINENEWLINEAs the least square estimator is extremely sensitive to errors of the observations, three other methods of compensational computation are introduced in Chapter 5. The estimation problem is represented using the simplex algorithm. Then the simplex algorithm is applied for the norm method of order 1. After introduction of some theoretical elements of robust estimation, the related algorithms of Huber and of Wilcox, Mallows and Schweppe are explained and applied. Besides, the filter method is shown, where the solution of a problem is parameterized with respect to the time, and where the state of the system is given backwards by the simultaneous observation and the state of the preceding time step. The book concentrates on the covariance filter of Kalman and the information filter of Kalman. NEWLINENEWLINENEWLINEChapter 6 shows some classes of different problems of geodesy, topography and photogrammetry where the least square method is used traditionally. The testing of the similarity of two grids of survey marks, the compensation computation of geodetic grids basing on observations of distances and angles, or on base lines, orbitographics and aerotriangulation are discussed. Here especially the physical methods are explained. Besides, the stationing of a point of unknown position by a theodolite and sighting of an ensemble of known points is shown. NEWLINENEWLINENEWLINETo illustrate the application of the least square method, in Chapter 7 a set of more than 8000 data registered by the mareograph of Marseille is analyzed. NEWLINENEWLINENEWLINEIn the last chapter, well-chosen exercises - followed by the answers - are given. The problems considered are Lagrange multiplicators, simple linear regressions, changes of survey marks, modelling of radars, compensation calculations of grids of points and of base lines, treatment of the cumulation of equations of normales, optimum configurations, polynomial approximation, detection of exo-planets, survey marks fixed to a deformable solid body, the link between the mean quadratic error and the least square method, mechanics of least squares, the connection between least squares and the Cholesky algorithm, as well as other special evolutionary and calculational methods. NEWLINENEWLINENEWLINEThe textbook is accessible to undergraduate students with basic knowledge in mathematics and physics.