Singularity, misspecification and the convergence rate of EM

DOI10.1214/19-AOS1924zbMath1462.62382arXiv1810.00828OpenAlexW3113313164MaRDI QIDQ1996764

Martin J. Wainwright, Raaz Dwivedi, Michael I. Jordan, Bin Yu, Koulik Khamaru, Nhat Ho

Publication date: 26 February 2021

Published in: The Annals of Statistics (Search for Journal in Brave)

Full work available at URL: https://arxiv.org/abs/1810.00828

zbMATH Keywords

empirical processFisher information matrixmixture modelslocalization argumentexpectation-maximization (EM)nonasymptotic convergence guarantees

Mathematics Subject Classification ID

Asymptotic properties of nonparametric inference (62G20) Classification and discrimination; cluster analysis (statistical aspects) (62H30) Nonparametric estimation (62G05) Statistical aspects of information-theoretic topics (62B10)

Related Items

Supermix: sparse regularization for mixtures, Iterative algorithm for discrete structure recovery, Sharp global convergence guarantees for iterative nonconvex optimization with random data, Network Gradient Descent Algorithm for Decentralized Federated Learning, A Doubly Enhanced EM Algorithm for Model-Based Tensor Clustering, Randomly initialized EM algorithm for two-component Gaussian mixture achieves near optimality in \(O(\sqrt{n})\) iterations

Uses Software

• CHIME

Cites Work

• Unnamed Item
• Unnamed Item
• Unnamed Item
• Asymptotic Behaviour of the Posterior Distribution in Overfitted Mixture Models
• Statistical guarantees for the EM algorithm: from population to sample-based analysis
• Hypothesis test for normal mixture models: the EM approach
• On the convergence properties of the EM algorithm
• Entropies and rates of convergence for maximum likelihood and Bayes estimation for mixtures of normal densities.
• Optimal rate of convergence for finite mixture models
• Convergence of latent mixing measures in finite and infinite mixture models
• Strong identifiability and optimal minimax rates for finite mixture estimation
• Local Rademacher complexities and oracle inequalities in risk minimization. (2004 IMS Medallion Lecture). (With discussions and rejoinder)
• CHIME: clustering of high-dimensional Gaussian mixtures with EM algorithm and its optimality
• Local Rademacher complexities
• Mixture Densities, Maximum Likelihood and the EM Algorithm
• Non-finite Fisher information and homogeneity: an EM approach
• Bayesian Model Selection in Finite Mixtures by Marginal Density Decompositions
• Dealing With Label Switching in Mixture Models
• Simultaneous Clustering and Estimation of Heterogeneous Graphical Models
• High-Dimensional Statistics
• Asymptotic Convergence Properties of the EM Algorithm for Mixture of Experts
• Estimating the Coefficients of a Mixture of Two Linear Regressions by Expectation Maximization