Smooth lattice orbits of nilpotent groups and strict comparison of projections (Q2149505)

The main result of this paper concerns the existence of a smooth vector \(g \in \mathcal{H}^\infty_\pi\) in a projective unitary representation \((\pi, \mathcal{H}_\pi)\) of a 1-connected nilpotent Lie group \(G\) to generate either a frame or a Riesz sequence under the orbit of a given lattice \(\Gamma \leq G\). Here, the orbit \(\pi(\Gamma)g\) is said to be a frame if for suitable constants \(0< A\leq B\) we have \[ A\|f\|_{\mathcal{H}_ \pi}^2 \leq \sum_{\gamma \in \Gamma}|\langle f, \pi(\gamma)g\rangle|^2 \leq B\|f\|_{\mathcal{H}_\pi}^2 \qquad \forall f \in \mathcal{H}_\pi. \] If instead we have \[ A \|c\|_{\ell^2}^2 \leq \bigg\| \sum_{\gamma \in \Gamma}c_\gamma \pi(\gamma)g\bigg\|_{\mathcal{H}_\pi}^2 \leq B \|c\|_{\ell^2}^2, \qquad \forall c \in \ell^2(\Gamma) \] for some \(0< A\leq B\), then \(\pi(\Gamma)g\) is said to be a Riesz sequence. The main result states that if the pair \((\Gamma, \sigma)\) satisfies the so-called Kleppner's condition, where \(\sigma : G \times G \to \mathbb{T}\) is the \(2\)-cocycle associated to the projective unitary representation \(\pi\), and if \(\pi\) is a relative discrete series representation of formal dimension \(d_\pi\), then the following assertions are valid: \begin{itemize} \item[1.] If \(\mathrm{vol}(G/\Gamma)d_\pi < 1\), then there exists \(g \in \mathcal{H}_\pi^\infty\) such that \(\pi(\Gamma)g\) is a frame \item[2.] If \(\mathrm{vol}(G/\Gamma)_\pi > 1\), then there exists \(g \in \mathcal{H}_\pi^\infty\) such that \(\pi(\Gamma)g\) is a Riesz sequence. \end{itemize}