Change-point analysis in nonstationary stochastic models (Q2833452)

Applications of stochastic models or processes are enormous from agriculture, astronomy, dynamics, economy, environment, geotechnics, hydrology, mechanics, physics, chemistry, engineerings, material science, data science, social science, medicine, and transportation, etc. The most famous one is that Scottish botanist Robert Brown investigated the fertilization process in \textit{Clarkia pulchella} to discover species of flowering plant, when he noticed a rapid oscillatory motion of the microscopic particles within the pollen grains suspended in water under the microscope. By the 1860s theoretical physicists had special interests in using Brownian motion to consistently explain various characteristics: a given particle appeared equally likely to move in any direction; further motion seemed totally unrelated to past motion; and the motion never stopped. \par The idea that molecules of a liquid or gas are in motion never stopped, colliding with each other and bouncing back and forth, is a prominent part of the kinetic theory of gases developed in the third quarter of the 19th century by the physicists James Clerk Maxwell, Ludwig Boltzmann, and Rudolf Clausius in explanation of heat phenomena. \par The temperature of a substance is proportional to the average kinetic energy with which the molecules of the substance are moving or vibrating. It was natural to guess that somehow this motion might be imparted to larger particles that could be observed under the microscope; if true, this would be the first directly observable effect that would corroborate the kinetic theory. This line of reasoning led the German physicist Albert Einstein in 1905 to produce his quantitative theory of Brownian motion. The French physicist Jean-Baptiste Perrin was successful in verifying Einstein's analysis to be awarded the Nobel Prize for Physics in 1926. His work established the physical theory of Brownian motion and ended the skepticism about the existence of atoms and molecules as actual physical entities. \par The 20th-century American mathematician Norbert Wiener made a remarkable result: within the set of all continuous functions on an interval, the set of differentiable functions has measure zero. In other words, the chance that picking a function randomly is differentiable is almost surely impossible. In physical terms, a particle moving under Brownian motion almost surely is moving on a non-differentiable path. This discovery clarified Albert Einstein's fundamental ideas about Brownian motion (displayed by the continual motion of specks of dust in a fluid under the constant bombardment of surrounding molecules). \par The Brownian motion is not a stationary process because its marginal distribution changes in time. The stationary stochastic process, translation process are time invariant. Even transformations of stationary process and oscillatory process are non-stationary. Change-point analysis is a powerful new analysis to test if a change has been made, and is capable of detecting subtle changes missed by control charts, and better characterizes the changes spotted by providing confidence levels and confidence intervals. A change-point analysis can be applied to data analysis to provide further information, to control charts when dealing with large historical data sets. A change-point analysis is more powerful and flexible and simpler to use, better characterizes the changes, controls the overall error rate and is robust to outliers. \par The book under review addresses the retrospective problem and sequential problem of the existence of processes, and proposes theoretical methods to optimally detect changes in retrospective and sequential formulation, as well as the characterization of the methods in both theory and experiment. Both change-point (abrupt changes in statistical characteristics), trends or unit roots (gradual changes in statistical characteristics), outliers and switches in coefficients of stochastic models (purely random and disappear changes in statistical characteristics) are considered in this book for the retrospective and sequential problems of non-stationary detection. First part of the book devotes to the retrospective change-point problem with special sections to the asymptotically optimal choice of parameters of decision statistics and a priori lower bounds for performance efficiency in different change-point problems, and the second part of the book contributes sequential problems of change-point detection to test hypothesis and to make decisions about the presence of a change-point at every step of data collection in a sequential way. \par \textit{E. S. Page} [Biometrika 41, 100--115 (1954; Zbl 0056.38002); Biometrika 42, 523--527 (1955; Zbl 0067.11602)] first formulated the retrospective change-point problem, and proposed the CUSUM rule for detection the change-point which is closely related to the maximum likelihood method. The retrospective problem of detection of changes in characteristics of random processes occur in econometrics, financial economics, financial mathematics, biomedicine and many other applications. A family $X=\{X^N\}_{N=N_0, N_0+1, \dots}, X^N=\{x^N(n)\}_{n=1}^N$ with $x^N (n) = x^{(i)}(n)$ if $[\theta_{i-1}N] \le n < [\theta_i N]$ for $\theta = \{\theta_1, \dots, \theta_k\}$ is glued random sequence generated by the collection $X$ and the collection $\{\theta = \{\theta_1, \dots, \theta_k\}\}$ are the points of gluing or change-point. Hence, the offline change-point problem for random sequences is to find an estimate of the unknown parameter $\theta$ in the described scheme by the given sample $X^N$. Brodsky and Darkhovsky (1990--2008) show that detection of changes in any d.f. can be reduced to detection of changes in the mean value of some new sequence constructed from an initial one, and proposed a three step procedure of change-point estimation which is first obtaining a preliminary list of change-points, second rejection of points from the preliminary list, and refining the change-points and calculation of confidence intervals. The lower limit determines the order of the asymptotically minimax estimate in one change-point problem in Theorem 2.6.1 and in multiple change-point problem in Theorem 2.6.2. \par \textit{D. A. Dickey} and \textit{W. A. Fuller} [J. Am. Stat. Assoc. 74, 427--431 (1979; Zbl 0413.62075); Econometrica 49, 1057--1072 (1981; Zbl 0471.62090)] proposed the most well-known test for unit roots by using a simple AR(1) model, \textit{S. E. Said} and \textit{D. A. Dickey} [Biometrika 71, 599--607 (1984; Zbl 0564.62075)] proved that the ADF (augmented Dickey-Fuller) test can be used in specifications of times series models with MA(\(q\)) terms, \textit{S. Ng} and \textit{P. Perron} [Econometrica 69, No. 6, 1519--1554 (2001; Zbl 1056.62529)] constructed test statistics that are based upon the GLS detrended data constructed by ERS. These tests are modified forms of \textit{P. C. B. Phillips} and \textit{P. Perron} [Biometrika 75, No. 2, 335--346 (1988; Zbl 0644.62094)] statistics and ERS point optimal statistic. The unit root and the structural change hypotheses are closely interrelated, and the problem of their distinguishing from each other is actual. Chapter 3 considers a new approach to retrospective detection of a stochastic trend and the problem of discrimination of a stochastic trend and a structural change hypotheses. \par Chapter 4 proposes asymptotically optimal methods for detection and estimation of possible switches. \textit{S. M. Goldfeld} and \textit{R. E. Quandt} [J. Econom. 1, 3--16 (1973; Zbl 0294.62087)] first proposed regression models with Markov switches. The author of the book proposes a nonparametric method for the retrospective detection of the number of d.f.'s and classification of observations with different d.f's. For type 1 (Theorem 4.3.1) and type 2 (Theorem 4.3.2) of binary mixtures, the probabilities of errors of the proposed method converges to zero exponentially as the sample size increase to infinity. Both the informational lower bound for performance efficiency of classification methods and the asymptotic optimality of the proposed method follows from Theorem 4.6.1. \par The general change-point problem for multivariate linear systems including AR (autoregression model), ARMA (autoregression moving average), SES (simultaneous equation systems) and reduced form is formulated in Subsection 5.2, Theorem 5.3.1 proves that the a priori theoretical lower bounds for the error probability in change-point estimation in multivariate models is given in terms of special functions for a unique change-point, Theorem 5.3.2 is for multiple change-points. For a unique change-point, Theorem 5.4.1 (for deterministic predictors) and Theorem 5.4.2 (for stochastic predictors) show that there is a new method for the retrospective change-point detection and estimation in multivariate linear models and this new method is asymptotically optimal by the order of convergence of change-point estimates to their true values as the sample size becomes larger. The new method for multiple change-points is related to a recurrent reduction to the case of one change-point. The author develops a new asymptotically optimal method that gives consistent estimates of an unknown number of change-points and their coordinates. A simulation study of characteristics of the proposed method for finite sample sizes is performed with comparison of other methods (Chow test, CUSUM, OLS CUSUM, fluctuation test, Wald test, LM test). Both deterministic, stochastic and multiple structural regressions are given, and a application of the proposed method to retrospective detection of structural changes in the monthly model of the CPI inflation in Russia from January 1995 to September 2015. \par Chapter 6 introduces a new nonparametric method for retrospective detection of change-points in state-space models, Theorem 6.3.1 and Theorem 6.3.2 present the exponential rate of convergence to zero of type-1 and type-2 error probability under some general assumptions. \par Chapter 7 devotes to the retrospective methods for revealing structural changes in different models of financial time series. The models in financial economics is the most difficult application of stochastic models in real practice. Applying previous result to the detection of structural changes in GARCH models, the author is able to correctly estimate the number of change-points and coordinates of these change-points for GARCH(1, 1) models. For real data of NASDAQ and NYSE, the method reveals several tentative change-points. Detection of changes in SV models and copula models is also given to properly identify the structural shifts and to estimate parameters. Theorem 7.5.1 and Theorem 7.5.2 show the statistical characteristics of the proposed method which is strongly resembles Hurst's R/S approach and Kolmogorov's statistic for detection of discrepancies between two d.f.'s. \par The second part, sequential problems of change-point detection, starts Chapter 8 with sequential hypothesis testing. \textit{A. Wald}'s seminal idea on sequential testing [Sequential analysis. New York: Wiley \& Sons (1947; Zbl 0029.15805)] has been extended to generalize the SPRT (sequential probability ratio test) to different cases of composite hypotheses testing. \textit{J. Kiefer} and \textit{L. Weiss} [Ann. Math. Stat. 28, 57--74 (1957; Zbl 0079.35406)] formulated a sequential hypothesis testing problem with the average sample volume in the indifference zone much larger than corresponding sample volumes, and \textit{T. L. Lai} [Ann. Probab. 1, 825--837 (1973; Zbl 0294.60028); Ann. Stat. 16, No. 2, 856--886 (1988; Zbl 0657.62088)] and \textit{V. P. Dragalin} and \textit{A. A. Novikov} [Theory Probab. Appl. 32, No. 4, 617--627 (1987; Zbl 0716.62076); translation from Teor. Veroyatn. Primen. 32, No. 4, 679--690 (1987)] proposed asymptotical solutions. \textit{V. P. Dragalin} and \textit{A. A. Novikov} [Obozr. Prikl. Prom. Mat. 6, No. 2, 387--399 (1999; Zbl 1007.62065)] studied sequential testing of multiple composite hypotheses in the presence of an indifference zone for an unknown parameter, \textit{T. L. Lai} [J. R. Stat. Soc., Ser. B 57, No. 4, 613--658 (1995; Zbl 0832.62072); IEEE Trans. Inf. Theory 46, No. 2, 595--608 (2000; Zbl 0994.62078)] proposed asymptotic lower bounds for the average sample size in sequential multihypotheses testing problems for dependent sequences that coincide with asymptotic lower bounds found earlier by \textit{G. Simons} [Ann. Math. Stat. 38, 1343--1364 (1967; Zbl 0178.22103)]. Chapter 8 presents a new performance measures for sequential tests (both one-sided and multisided sequential tests) for composite hypotheses and the nonasymptotical a priori lower bounds for these measures, and results remain true without the exponential family assumption for the distributed function of observations. Theorem 8.3.1 and Theorem 8.3.2 give the estimate and asymptotical optimal of performance measures for the one-sided tests, and Theorem 8.4.1 and Theorem 8.4.2 give the results for the multisided tests. \par Any sequential method of detection is the process of decision making about the presence of nonhomogeneity, hence it is natural to characterize the quality of a method of sequential detection by the probability of the 1st-type error (false alarm), the probability of the 2nd type error (false tranquility) and the probability of the error of estimation of the change-point. The sequential diagnosis of sequential observations consists of the moment the distribution law of observations changes and the detection of the change-point as soon as possible on false alarms. Page [loc. cit. (1954); loc. cit. (1955)] proposed a stopping rule with alarm threshold, and \textit{A. N. Shiryaev} [Sov. Math., Dokl. 2, 740--743 (1961; Zbl 0109.12802); translation from Dokl. Akad. Nauk SSSR 138, 799--801 (1961); Theory Probab. Appl. 8, 22--46 (1963; Zbl 0213.43804); translation from Teor. Veroyatn. Primen. 8, 26--51 (1963); Theory Probab. Appl. 8, 247--265 (1963; Zbl 0279.90011); translation from Teor. Veroyatn. Primen. 8, 264--281 (1963); Teor. Veroyatn. Primen. 10, 380--385 (1965; Zbl 0139.36003); Statistical sequential analysis. Optimal stopping rules. 2nd ed., rev. (Russian). Moskva: Nauka (1976; Zbl 0463.62068)] studied the problem of sequential change-point detection for random processes with discrete and continuous time. Chapter 9 gives the brief history on the sequential change-point detection and provides performance characteristics of sequential tests for the nonparametric version of the CUSUM in Theorem 9.2.1 and Theorem 9.2.2, for the nonparametric version of the Girshick-Rubin-Shyriav (GRSh) method in Theorem 9.2.3 and Theorem 9.2.4, for the exponential smoothing method in Theorem 9.2.5 and Theorem 9.2.6, for the moving sample methods in Theorem 9.2.7 and Theorem 9.2.8. A priori information estimate for the delay time in change-point detection is presented in Theorem 9.3.1 with the Kullback-Leibler information measure, and the normalized delay time in change-point detection converges almost surely to some constant as sample size increases by the Rao-Cramer inequality. The comparative analysis of sequential methods differs from the classical methodology based upon the nonlinear renewal theory and Wald's sequential procedures. The analysis of the false alarm probability for CUSUM and GRSh methods is given in Theorem 9.4.1 and Theorem 9.4.2. The analysis of the asymptotic optimality is relied on comparison of the limit characteristics of the normalized delay time in detection and the rate of convergence of the normalized delay time with a priori informational boundaries, and simulations for both methods are presented in subsection 9.5. \par Kolmogorov and Shiryaev (1959) proposed the formal statement of the quickest detection of spontaneous effects problem (later refer to the disorder problem), and Shiryaev [loc. cit. (1963)] found the optimal solution of this problem for the full a priori information on the distribution function of observations and a change-point. Lai [loc. cit. (1995); loc. cit. (1988)] generalized the problem of sequential change-point detection in dynamical systems to the non-i.i.d. case and studied the window-limited generalized likelihood ratio (GLR) to prove the asymptotical optimality in different problems of sequential change-point detection in dynamic systems. \textit{C.-S. J. Chu} et al. [Econometrica 64, No. 5, 1045--1065 (1996; Zbl 0856.90027)] analyzed the problem of monitoring structural changes in coefficients of a linear regression, and used the fluctuation test for sequential detection and diagnosis of abrupt changes in coefficients. There is lack of research on optimality and asymptotic optimality of these methods. Chapter 10 devotes to the theoretical study of finite sample volume on a new method of sequential detection of change-points in linear models. Based on moving windows statistics, the detection method and detection of an unknown change-point in the mathematical expectation of observations are constructed. The probability of type-1 error (false decision) with upper estimate in Theorem 10.4.1 and the probability of type-2 error (missed goal) with upper estimate in Theorem 10.4.2 both converge to zero exponentially. The author and \textit{B. S. Darkhovsky} [Non-parametric statistical diagnosis. Problems and methods. Dordrecht: Kluwer Academic Publishers (2000; Zbl 0995.62031)] show that these properties characterize an asymptotically optimal method of detection in general. Comparison with other well-known methods for sequential detection of structural changes in linear models is carried out by Monte-Carlo tests, practical applications for the analysis of stability of the German quarterly model of demand for money (1961--1995) and the Russian monthly model of inflation (1994--2005) are discussed in the later sections of this chapter. \par Chapter 11 gives the theoretical analysis of the early change-point detection problems with respect to the detection a change-point as soon as possible and understanding the small false alarm rate (FAR). For both univariate models with independent observations and multivariate models with dependent observations, the performance characteristics of a change-point detection method has the lower bound in Theorem 11.2.1 and Theorem 11.2.2. Both the CUSUM and GRSh methods are not asymptotically optimal for any model of changes except abrupt ones (the classical change-point model). The Monte Carlo tests are performed for the proposed methods of early change-point detection in univariate and multivariate models. \par Chapter 12 aims at sequential detection of switches in models with changing structures. The problem is formulated first, and the supremal probability of a false decision for the proposed method is given in Theorem 12.3.1. The assumption of the exponential rate of convergence to zero for error probabilities is quite mild and requires the proper choice of the large parameter, any reasonable methods of change-point detection must have such a large parameter, Theorem 12.4.1 proves that a priori information inequalities for the problem of sequential change-point detection, and these inequalities can be used for the asymptotical optimality of the proposed method. \par The last chapter discusses the nonparametric approach to sequential change-point detection and estimation, the supremal probability of a false decision in the sequential change-point detection is controlled by the exponential rate with the large parameter. Change-in-mean and change-in-dispersion are presented in simulations Section 13.4. The asymptotical optimality of the CUSUM method for sequential detection of a change-point is obtained in Theorem 13.5.1 a Theorem 13.5.2 for the 2nd order optimality in contrast to the classical 1st order optimality. \par This is a very readable and attractive book for people who hope to learn the retrospective and sequential change-point problems of detection and estimates of nonstationary stochastic processes, packed with historical information and idea development and with numerical, stimulative and empirical analysis and it is certainly written with authority. The references adequately sample the relevant literature on the detection of change-point and estimations as well as its applications in financial economics. But the reader especially interested in statistic methods and models of those problems studied in this book, would also do well to investigate the wealth of literature in asymptotic theory, optimal test of the sequential change-point detection related to sequential data collection in the recent development of data science.