The space of bounded spherical functions on the free 2-step nilpotent Lie group (Q941089)

Let \( N\) be a connected and simply connected nilpotent Lie group and \(K\) a compact Lie group acting smoothly on \(N\) by automorphisms.The pair \((K,N)\) is said to be a Gelfand pair if the set of integrable \(K\)-invariant functions on \(N\) is an abelian algebra with respect to convolution. Let \(\Delta(K,N)\) be the set of all \(K\)-spherical and bounded functions for the Gelfand pair \((K,N)\). If \(G=K\times N \), the set \[ \widehat{G}_K=\{\pi\in\widehat{G}:\pi\text{ has one-dimensional space of \(K\)-fixed vectors\}} \] denotes the set of \(K\)-spherical representations of \(G\). Let \(\theta(\pi)\) be the coadjoint orbit associated to the representation \(\pi\). The authors of the present paper show first that \(\theta(\pi)\cap\mathfrak{n}^{*}\) consists only of one single point, denoted by \(K(\pi)\), where \(\mathfrak{n}^{*}\) designates the dual vector space of the Lie algebra of \(N\). They show afterwards that if \[ A(K,N)=\{K(\pi),\;\pi\in\widehat{G}_{K}\}, \] then there is a one-to-one correspondence between \(\Delta(K,N)\) and \(A(K,N)\) and conjecture that these sets are homeomorphic when \(A(K,N)\) is endowed with the trace topology of \(\mathfrak{n}^{*}/K\) and \(\Delta(K,N)\) with the compact open topology. The main upshot of the paper is to prove positively the conjecture in the case of Gelfand pairs given by the action of the orthogonal group on the free \(2\)-step nilpotent Lie group. Their method shows also in the studied case, how to embed the space \(\Delta(K,N)\) in a Euclidean space.