The explicit duality correspondence of (\(\text{Sp}(p,q),\text{O}^{*}(2n)\)) (Q1874452)

The main result of this paper is the fact that whenever \(p+q\leq n\), the theta correspondence for the dual pair \((\text{Sp}(p,q),\text{O}^*(2n))\) gives rise to an injection of the unitary dual of \(\text{Sp}(p,q)\) into the unitary dual of \(\text{O}^*(2n)\). This injection is described explicitly in terms of Langlands parameters. As an application, the authors obtain many previously unknown small unitary representations of \(\text{O}^*(2n)\), corresponding to known unitary representations of \(\text{Sp}(p,q)\) for \(p+q\leq n/2\) (the stable range). The latter include the trivial representation, the tempered representations and the unitary representations with nonzero cohomology. For \(n\geq 8\), this gives a complete classification of irreducible unitary representations of \(\text{O}^*(2n)\) with rank (in the sense of Howe) at most 3. The method employed is to use some known results about correspondences in the equal rank case (i.e., \(p+q=n\) or \(n-1\)), the induction principle of Kudla and Moeglin, and a careful analysis of lowest \(K\)-types. For the sake of completeness, the authors also work out the correspondence for the type I dual pairs over quaternions, \((\text{GL}(m,\mathbb{H}),\text{GL}(n,\mathbb{H}))\).