Complete Dictionary Recovery over the Sphere

Author name not available (Why is that?)

Publication date: 26 April 2015

Abstract: We consider the problem of recovering a complete (i.e., square and invertible) matrix

m a t h b f A_{0}

, from

m a t h b f Y i n m a t h b b R^{n i m e s p}

with

m a t h b f Y = m a t h b f A_{0} m a t h b f X_{0}

, provided

m a t h b f X_{0}

is sufficiently sparse. This recovery problem is central to the theoretical understanding of dictionary learning, which seeks a sparse representation for a collection of input signals, and finds numerous applications in modern signal processing and machine learning. We give the first efficient algorithm that provably recovers

m a t h b f A_{0}

when

m a t h b f X_{0}

has

O (n)

nonzeros per column, under suitable probability model for

m a t h b f X_{0}

. In contrast, prior results based on efficient algorithms provide recovery guarantees when

m a t h b f X_{0}

has only

O (n^{1 - d e l t a})

nonzeros per column for any constant

d e l t a i n (0, 1)

. Our algorithmic pipeline centers around solving a certain nonconvex optimization problem with a spherical constraint, and hence is naturally phrased in the language of manifold optimization. To show this apparently hard problem is tractable, we first provide a geometric characterization of the high-dimensional objective landscape, which shows that with high probability there are no "spurious" local minima. This particular geometric structure allows us to design a Riemannian trust region algorithm over the sphere that provably converges to one local minimizer with an arbitrary initialization, despite the presence of saddle points. The geometric approach we develop here may also shed light on other problems arising from nonconvex recovery of structured signals.

Has companion code repository: https://github.com/sunju/dl_focm

Mathematics Subject Classification ID

No records found.

This page was built for publication: Complete Dictionary Recovery over the Sphere

Report a bug (only for logged in users!)Click here to report a bug for this page (MaRDI item Q6261239)