Restrictions of discrete series of certain solvable groups (Q2927942)

The paper concerns the restrictions of unitary irreducible representations of a Lie group \(G\) in the general case. In particular, the author studies the restriction of square integrable representations modulo the center of a solvable connected group, semi-direct product of a torus by a Heisenberg group to its algebraic connected subgroups. Let \(\Omega\subset\mathfrak{g}^\ast\) be a strongly regular coadjoint orbit and \(B\) the associated alternating bilinear form. Let also \(H\) be a subgroup of \(G\) with Lie algebra \(\mathfrak{h}\). The author proves that a property of \(H\) called ``property \(\mathcal{P}(\Omega)\)'' (Definition 2.1) is equivalent to the projection \(p:\Omega\longrightarrow \mathfrak{h}^\ast\) being proper on its image. It is also proved that if \(\pi\) is a discrete series associated to the strongly regular orbit \(\Omega\), then the property \(\mathcal{P}(\Omega)\) is equivalent to \(\pi\) being \(H\)-admissible. This answers positively a conjecture of M. Duflo in this case. Other results proved in the paper study the image \(p(\Omega)\) under certain conditions, and the decomposition to irreducible representations of \(\pi^G_g\) restricted to \(H\).