A general spectral gap property for measures. (Q1411679)

Let \(G\) be a locally compact group and \({\mathcal P}(G)\) be the space of all regular Borel probability measures on \(G\). Given a unitary representation \(\rho\) of \(G\) and \(\mu \in {\mathcal P}(G)\) one defines \(\rho (\mu ) = \int \rho (g) d\mu (g)\). One would like to know under what conditions on \(\mu\) or on \(G\) the spectral radius of \(\rho (\mu )\) or the norm of \(\rho (\mu )\) is less than one. A positive answer has applications in ergodic theory and dynamical systems. The author obtains that the spectral radius of \(\rho (\mu )\) is less than one for a class of unitary representations and for measures on the linear group \(GL(n, R)\) provided the support of \(\mu\) satisfies certain non-degeneracy conditions. As an application of the main result the author shows that for odd integers \(n\) and for any unitary representation \(\rho\) of \(SL(n, R)\) without invariant vectors, \(\rho (\mu)\) has spectral radius less than one (respectively, \(\| \rho (\mu ) \| <1\)) if the support of \(\mu\) is not contained in a proper Zariski-closed subgroup of \(G\) (respectively, if the support of \(\mu\) is not contained in a coset of a proper Zariski-closed subgroup of \(G\)). One may also look at the papers of \textit{Y. Derriennic} and \textit{Y. Guivarch} [C. R. Acad. Sci., Paris, Sér. A 277, 613--615 (1973; Zbl 0272.60005)], \textit{R. Jones}, \textit{J. Rosenblatt} and \textit{A. Tempelman} [Ill. J. Math. 38, No. 4, 521--553 (1994; Zbl 0831.28008)] and \textit{M. Lin} and \textit{R. Wittmann} [Stud. Math. 114, No. 2, 127--145 (1995; Zbl 0839.22004)].