Pfaffians and determinants for Schur \(Q\)-functions (Q1924230)

Schur functions \(Q_\lambda\) were introduced by Schur in 1911: they are related to projective representations of the symmetric group as the ordinary Schur functions are related to its linear representations. Each partition \(\lambda\) consisting of a strictly decreasing sequence of positive integers has a corresponding symmetric function \(Q_\lambda\). Schur defined \(Q_\lambda\) for a general \(\lambda\) as the Pfaffian of the skew-symmetric matrix whose \((i,j)\)th entry for \(i>j\) is \(Q_{(\lambda_i,\lambda_j)}\). (See Section III.8 of \textit{I. G. Macdonald} [Symmetric functions and Hall polynomials, Oxford University Press, 2nd edition (1995; Zbl 0824.05059)].) In the 1980's it was shown by B. E. Sagan and by D. Worley that Schur \(Q\)-functions can be defined combinatorially using `shifted' tableaux, in which the \(i\)th row starts with a box in the \(i\)th column. Lattice path methods developed for Young diagrams by I. Gessel and G. Viennot have been extended by \textit{J. R. Stembridge} [Adv. Math. 83, No. 1, 96-131 (1990; Zbl 0790.05007)] to shifted tableaux. These methods are carried further in the present paper. The main result is a generalised Pfaffian formula for skew Schur \(Q\)-functions \(Q_{\lambda/\mu}\), where \(\lambda\) and \(\mu\) are strict partitions with \(\lambda_i\geq\mu_i\) for all \(i\). This formula is based on the notion of an `outside decomposition' of the shifted diagram of \(\lambda/\mu\), due to the author and \textit{I. P. Goulden} [Eur. J. Comb. 16, No. 5, 461-477 (1995; Zbl 0832.05097)]. This is a decomposition of the diagram into disjoint skew hooks, each running from the left or bottom edge of the diagram to its right or top edge. In the special case where these skew hooks are the rows of the diagram, the formula reduces to Schur's Pfaffian, as generalised to skew \(Q\)-functions by \textit{T. Jozefiak} and \textit{P. Pragacz} [J. Lond. Math. Soc., II. Ser. 43, No. 1, 76-90 (1991; Zbl 0761.20007)]. A second result of the same kind provides a determinantal formula for skew \(Q\)-functions, also based on an outside decomposition. This generalises a formula of S. Okada, which again corresponds to the simple decomposition of the diagram into its rows.