A commutativity criterion for certain algebras of invariant differential operators on nilpotent homogeneous spaces (Q1434536)

Let \(G\) be a connected, simply connected real nilpotent Lie group with Lie algebra \( \mathfrak g\), \( H\) a connected closed subgroup of \(G\) with Lie algebra \(\mathfrak h\) and \(f\) a linear form on \( \mathfrak g\) satisfying \(\langle f, [\mathfrak h, \mathfrak h] \rangle = (0)\). Let \(\chi_f\) be the unitary character of \(H\) defined by \( \chi_f( \exp X)= e^{if(X)},\;X\in \mathfrak h.\) Consider the unitary representation \( \tau = \text{Ind}_H^G\chi_f\) induced from the character \(\chi_f\) of \(H\). So, \(\tau \) decomposes into a continuous sum of unitary irreducible representations of \(G\), \[ \tau = \int ^\oplus _{\widehat {G}} m(\pi) \pi d\mu(\pi) \] where \( m(\pi)\) denotes the multiplicity of \( \pi\) and \( \mu\) a Plancherel measure of \(\tau \) on the unitary dual \(\widehat G\) of \(G\). The following fact is well known: either the multiplicity function \(m(\pi)\) is finite and uniformly bounded for \(\mu\)-almost all \(\pi \in \widehat G\), or it is infinite [cf. \textit{L. Corwin}, \textit{F. P. Greenleaf} and \textit{G. Grelaud}, Trans. Am. Math. Soc. 304, 549--583 (1987; Zbl 0629.22005); \textit{R. Lipsman}, Trans. Am. Math. Soc. 313, 433--473 (1989; Zbl 0683.22009)]. The paper under review deals with the algebra \(D_\tau (G/H)\) of linear \( C^\infty\)-differential operators keeping invariant the space \(C^\infty(G,H,f)\) of \(C^\infty \) complex functions \(\phi\) on the group \(G\) and satisfying the cocycle covariance relation \(\phi(gh)= \chi_f(h^{-1}) \phi(g)\) for all \(h\in H\) and \(g\in G\), and commuting with left translations of \(G\) on that space. \textit{L. Corwin} and \textit{F. P. Greenleaf} proved that under the assumption that \(\tau\) is of finite multiplicities the algebra \(D_\tau(G/H)\) is commutative [Commun. Pure Appl. Math. 45, 681--748 (1992; Zbl 0812.43004)] and conjectured later that the converse is also true. In this interesting paper, the authors prove this conjecture for an arbitrary connected simply connected nilpotent Lie group extending then earlier cases studied by \textit{H. Fujiwara}, \textit{G. Lion} and \textit{S. Mehdi} [Trans. Am. Math. Soc. 353, 4203--4217 (2001; Zbl 0978.43007)], the reviewer and \textit{H. Fujiwara} [Compos. Math 139, 29--65 (2003; Zbl 1035.22006)] and the reviewer and \textit{J. Ludwig} [Monatsh. Math. 134, 19--37 (2001; Zbl 0997.22007)]. In the setup of simply connected exponential solvable Lie groups, a similar result has been obtained by the reviewer and \textit{H. Fujiwara} in the case where \(H\) is a normal subgroup [Res. Expo. Math. 25, 127--134 (2002; Zbl 1015.43013)].