Spurious Valleys in Two-layer Neural Network Optimization Landscapes

zbMath1434.68464arXiv1802.06384MaRDI QIDQ5214225

Joan Bruna, Luca Venturi, Afonso S. Bandeira

Publication date: 7 February 2020

Full work available at URL: https://arxiv.org/abs/1802.06384

zbMATH Keywords

neural networkslossnonconvex optimizationgradient-descent methods

Mathematics Subject Classification ID

Artificial neural networks and deep learning (68T07) Nonconvex programming, global optimization (90C26) Learning and adaptive systems in artificial intelligence (68T05) Neural nets and related approaches to inference from stochastic processes (62M45)

Related Items (13)

On PDE Characterization of Smooth Hierarchical Functions Computed by Neural Networks ⋮ On the landscape of one-hidden-layer sparse networks and beyond ⋮ Landscape analysis for shallow neural networks: complete classification of critical points for affine target functions ⋮ Nonlinear Weighted Directed Acyclic Graph and A Priori Estimates for Neural Networks ⋮ Learning the mapping \(\mathbf{x}\mapsto \sum\limits_{i=1}^d x_i^2\): the cost of finding the needle in a haystack ⋮ On the Benefit of Width for Neural Networks: Disappearance of Basins ⋮ A machine learning approach to portfolio pricing and risk management for high‐dimensional problems ⋮ Spurious Valleys, NP-Hardness, and Tractability of Sparse Matrix Factorization with Fixed Support ⋮ Certifying the Absence of Spurious Local Minima at Infinity ⋮ Optimization for deep learning: an overview ⋮ Unnamed Item ⋮ Unnamed Item ⋮ Multilinear Compressive Sensing and an Application to Convolutional Linear Networks

Cites Work

• Unnamed Item
• Unnamed Item
• Social welfare and profit maximization from revealed preferences
• Learning a deep convolutional neural network via tensor decomposition
• Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks
• Optimization Methods for Large-Scale Machine Learning
• Deterministic Guarantees for Burer‐Monteiro Factorizations of Smooth Semidefinite Programs
• Symmetric Tensors and Symmetric Tensor Rank
• Breaking the Curse of Dimensionality with Convex Neural Networks
• On the Equivalence between Kernel Quadrature Rules and Random Feature Expansions
• Geometrical Insights for Implicit Generative Modeling

This page was built for publication: Spurious Valleys in Two-layer Neural Network Optimization Landscapes