Formal dimension for semisimple symmetric spaces (Q2711522)

Let \(G\) be a connected semisimple Lie group, \(G/H\) a corresponding symmetric space and \((\pi_\lambda, {\mathcal H}_\lambda)\) a representation in the holomorphic discrete series of \(G/H\). Then the corresponding Harish-Chandra module is a highest weight representation of the complexified Lie algebra \(\mathfrak g_\mathbb C\). Let \(v_\lambda\) be a highest weight vector in \({\mathcal H}_\lambda\) and \(\eta \in {\mathcal H}_\lambda^{-\omega}\) a non-zero \(H\)-invariant hyperfunction vector (all others are multiples of \(\eta\)). The inverse of the number NEWLINE\[NEWLINE {1\over d(\lambda)} = {1\over |\langle \eta, v_\lambda \rangle|^2} \int_{G/ZH} |\langle \eta, \pi_\lambda(g^{-1}).v_\lambda \rangle|^2 d(gHZ) NEWLINE\]NEWLINE is called the formal dimension of the representation \(\pi_\lambda\). The main result of the present paper is an explicit formula for the formal dimension of all holomorphic discrete series representations of \(G/H\). For the case where \(G/H\) is a group, the formal dimension of the holomorphic discrete series representation was computed by \textit{Harish-Chandra} [Am. J. Math. 78, 564-628 (1956; Zbl 0072.01702)]. The meaning of the formal dimension is that it is the ``mass'' of the representation \(\pi_\lambda\) in the discrete part of the Plancherel formula of \(G/H\). As an intermediate result to the explicit formula for \(d(\lambda)\), Kroetz proves an ``averaging theorem'' asserting that, in the sense of hyperfunction vectors we have NEWLINE\[NEWLINE \int_H \pi_\lambda(h).v_\lambda dh = {\langle v_\lambda, v_\lambda\rangle \over \langle \eta, v_\lambda\rangle} c(\lambda + \rho) \eta, NEWLINE\]NEWLINE where \(c\) is the \(c\)-function of the dual symmetric space \(G^c/H^c\) of \(G/H\). The formula for \(d(\lambda)\) is now proved in two steps. First one considers only ``regular'' values on \(\lambda\) and then one uses an analytic continuation argument. For the ``regular'' case it is shown that \(d(\lambda) = d(\lambda)^G c(\lambda + \rho)\), where \(d(\lambda)^G\) is Harish-Chandra's formal dimensional for the group \(G\). Likewise the \(c\)-function has a product decomposition \(c = c_0 c_\Omega\), where \(c_0\) is the well known \(c\)-function of a Riemannian symmetric space, and \(c_\Omega\) is the \(c\)-function of the real form \(\Omega\) of the bounded symmetric domain \(G/K\). In view of these product decompositions, the explicit formula for \(d(\lambda)\) follows from Harish-Chandra's degree formula, the Gindikin-Karpelevic formula for \(c_0\), and a very recent product formula for \(c_\Omega\) due to the author and \textit{G. Olafsson} [``The \(c\)-function for non-compactly causal symmetric spaces'', Preprint].