A note on a new description of invariant maximal nonnegative subspaces in an indefinite inner product space (Q1081100)

In this paper, A and B are self-adjoint \(n\times n\) matrices, with A assumed to be invertible. \(L^{\perp_ A}\) denotes the set of all vectors \(y\in C^ n\) such that \((Ax,y)=0\) for all \(x\in L\). The authors show that a subspace L is \(A^{-1}B\)-invariant maximal A-nonnegative if and only if L is maximal nonnegative and \(L^{\perp_ A}\subset L^{\perp_ B}\), and that such a subspace L always exists. A subspace L is called A-neutral if \((Ax,y)=0\) for all x,y\(\in L\). If L is an \(A^{-1}B\)-invariant A-neutral subspace, with dimension p, then L is also B-neutral, and \(p\leq n_ 0+\min (n_+,n_-)\), where \((n_+,n_-,n_ 0)\) is the inertia of B. If B is invertible and \(p=n/2\) (i.e., L is a hypermaximal neutral subspace), then \(n_+=n_- =n/2.\) The following theorem is also proved: A subspace L is \(A^{-1}B\)-invariant hypermaximal A-neutral if and only if dim L\(=n/2\) and L is both A-neutral and B-neutral.