Endoscopy and cohomology growth on \(\mathrm{U}(3)\) (Q2877507)

Let \(\mathrm{U}(2,1,{\mathbb R})\) be the isometry group of an Hermitian form in three variables of signature \((2,1)\). A way to construct cocompact discrete subgroups of \(\mathrm{U}(2,1,{\mathbb R})\) is as follows: let \(E/F\) be a CM-extension of number fields, that is, \(F\) is a totally real number field and \(E\) is a quadratic extension without real places (i.e., at each infinite place of \(F\) the extension \(E/F\) is ramified). Let \(\Phi\) be an Hermitian form on \(E^3\) such that \(\Phi\) is of signature \((2,1)\) at exactly one infinite place \(v_0\) of \(F\) and definite at all other infinite places. Let \(\mathcal O\) denote the ring of integers in \(E\) and \(\Gamma\) be the stabilizer of the lattice \({\mathcal O}^3\) in \(U(\Phi)\); then \(\Gamma\) is a lattice in the Lie group \(G = \mathrm{U}(\Phi\otimes_{v_0} {\mathbb C})\cong \mathrm{U}(2,1)\). If \(F\not= {\mathbb Q}\) then \(\Gamma\) is actually cocompact (when \(E\) is an imaginary quadratic field the lattice \(\Gamma\) is never cocompact). The group \(\Gamma\) acts cocompactly on the contractible manifold \(\mathrm{U}(2,1)/(\mathrm{U}(2)\times \mathrm{U}(1))\), hence its virtual cohomological dimension is 4 and any of its torsion-free finite index subgroups is a Poincaré duality group of dimension 4.NEWLINENEWLINEIf \(\mathfrak n\) is an ideal in \(\mathcal O\) then one can define a congruence subgroup of level \(\mathfrak n\) in \(\Gamma\), which we denote by \(\Gamma({\mathfrak n})\). This is a finite index subgroup of \(\Gamma\) (of index \(V({\mathfrak n}) := [\Gamma:\Gamma({\mathfrak n})] \asymp |\mathfrak n|^8\)). The subject of the present paper is the growth rate of the cohomology groups \(H^i(\Gamma({\mathfrak n}, {\mathbb C})\) relatively to the index \(V({\mathfrak n})\). We denote by \(b^i(\Gamma(\mathfrak n)) = \dim H^i(\Gamma({\mathfrak n}, {\mathbb C})\). Then it is known by work of \textit{D. L. DeGeorge} and \textit{N. R. Wallach} [Ann. Math. (2) 107, 133--150 (1978; Zbl 0397.22007)] (in the compact case; the non-compact case was dealt with by \textit{G. Savin} [Invent. Math. 95, No. 1, 149--159 (1989; Zbl 0673.22003)] that NEWLINE\[NEWLINE \lim_{|\mathfrak{n}|\to+\infty} \frac{b^2(\Gamma(\mathfrak{n}))}{V({\mathfrak n})} NEWLINE\]NEWLINE exists and is equal to the Euler characteristic \(\chi(\Gamma)\). This implies that the first Betti number has a sublinear growth in \(V({\mathfrak n})\): The main result of the paper under review is the detemination of the exact growth rate, which is : NEWLINE\[NEWLINE \limsup_{|\mathfrak{n}|\to+\infty} \frac{b^1(\Gamma(\mathfrak{n}))}{V({\mathfrak n})^{3/8}} < +\infty, \quad \liminf_{|\mathfrak{n}|\to+\infty} \frac{b^1(\Gamma(\mathfrak{n}))}{V({\mathfrak n})^{3/8}} > 0 \qquad (\ast) NEWLINE\]NEWLINE (that the growth rate be dominated by a power \(<1\) of the index was conjectured in general by \textit{P. Sarnak} and \textit{X. Xue} [Duke Math. J. 64, No. 1, 207--227 (1991; Zbl 0741.22010)], and proved in a weaker form by them, using techniques essentially different from those of the present paper). In the case of a noncompact quotient (when \(F={\mathbb Q}\)) the same result hold at least when replacing \(b^1\) by \(b_{(2)}^1\) which is the dimension of the subspace of \(H^1\) made up of those classes that can be represented by square-integrable automorphic forms.NEWLINENEWLINEThe proof of \((\ast)\) depends on the classification of automorphic representations of \(\mathrm{U}(2,1)\) (the subject of \textit{J. D. Rogawski}'s book [Automorphic representations of unitary groups in three variables. Princeton, NJ: Princeton University Press (1990; Zbl 0724.11031)]). The cohomological representations of \(G\) which occur at \(v_0\) in automorphic representations of the adele group come via endoscopy from representations of smaller compact groups, and the multilplicities in \(L^2(G/\Gamma({\mathfrak n}))\) are related to traces of characters on these groups which can be easily controlled to yield \((\ast)\). (We note that the actual proof and even the formal statement of these heuristic remarks is very involved, and their summary in the paper at hand is somewhat terse.)NEWLINENEWLINEThere are further cocompact arithmetic lattices inside \(\mathrm{U}(2,1)\), the congruence subgroups of which are known (by a result of [Rogawski, loc. cit.]) to always have a null first Betti number; thus the present paper allows, as noted by the author, to completely determine the growth rate of Betti numbers of congruence towers of complex hyperbolic manifolds.