A triple ratio on the Silov boundary of a bounded symmetric domain (Q2569836)

Summary: Let \(D\) be a Hermitian symmetric space of tube type, \(S\) its Silov boundary and \(G\) the neutral component of the group of bi-holomorphic diffeomorphisms of \(D\). Our main interest is in studying the action of \(G\) on \(S^{3} = S \times S \times S\). Section 1 and 2 are part of a joint work (see \textit{J.-L. Clerc} and \textit{B. Ørsted} [Transform. Groups 6, No. 4, 303--320 (2001; Zbl 1078.53076)]). In Section 1, as a pedagogical introduction, we study the case where \(D\) is the unit disc and \(S\) is the circle. This is a fairly elementary and explicit case, where one can easily get a flavour of the more general results. In Section 2, we study the case of tube type domains, for which we show that there is a finite number of open \(G\)-orbits in \(S^3\), and to each orbit we associate an integer, called the Maslov index. In the special case where \(D\) is the Siegel disc, then \(G\) is (isomorphic to) the symplectic group and \(S\) is the manifold of Lagrangian subspaces. The result on the orbits and the number which we construct coincides with the classical theory of the Maslov index (see \textit{G. Lion} and \textit{M. Vergne} [Progress in Mathematics, 6. Boston - Basel - Stuttgart: Birkhäuser. VIII, 337 p. (1980; Zbl 0444.22005)]), hence the name. We describe a formula for computing the Maslov index, using the automorphy kernel of the domain \(D\). In the special case of the Lagrangian manifold, this formula was obtained by \textit{B. Magneron} [J. Funct. Anal. 59, 90--122 (1984; Zbl 0548.57024)] in a different appoach. In Section 3, we study the case where \(D\) is the unit ball in a (rectangular) matrix space. There is now an infinite family of orbits, and we construct characteristic invariants for the action of \(G\) on \(S^3\). For the special case where \(D\) is the unit ball in \(\mathbb{C}^2\), this coincides with an invariant constructed by \textit{E. Cartan} [Comment. Math. Helv. 4, 158--171 (1932; Zbl 0005.11405)] for the \`\` hypersphere''. In all cases, we follow the following method: from an appropriate automorphy kernel for \(D\) we construct a kernel on \(D \times D \times D\), satisfying a simple transformation property under the action of \(G\). We then define a dense open set of \(S^3\) (the set of mutually transversal points in \(S\)), on which the kernel (or some function of it) can be extended continuously, and the resulting kernel is invariant or at least transforms nicely under the action of \(G\).