A monotonicity property of \(h\)-vectors and \(h^*\)-vectors (Q1260775)

The \(h\)-vector \((h_ 0, \dots, h_ d)\) of a simplicial \((d-1)\)-complex \(\Delta\) is defined by \[ \sum^ d_{i = 0}f_{i - 1}(x - 1)^{d - i} = \sum^ d_{i = 0} h_ ix^{d - i}, \] where \(f_ i\) is the number of \(i\)-simplices in \(\Delta\); the same notation, with \(h_{e+1} = \cdots = h_ d = 0\), is used if \(\dim \Delta = e - 1<d - 1\). The author shows here that, if \(\Delta\) is a Cohen-Macaulay complex, and \(\Delta'\) is a Cohen-Macaulay \((e-1)\)-subcomplex no \(e+1\) of whose vertices form a facts of \(\Delta\), then \(h(\Delta') \leq h(\Delta)\). Similar techniques are employed to obtain classes of Gorenstein complexes with unimodal \(h\)- vectors. Most of these results have been obtained by \textit{G. Kalai} [DIMACS, Ser. Discret. Math. Theor. Comput. Sci. 4, 387-411 (1991; Zbl 0739.52017)] by less simple methods. However, the same ideas yield new results about lattice polytopes, namely polytopes \({\mathcal P}\) with vertices in a lattice \(L\) in \(\mathbb{R}^ m\). Writing \(i({\mathcal P},n)\) for the number of lattice points in \(n{\mathcal P}\) (with \(i({\mathcal P}, 0) = 1)\), the (integer) \(h^*\)-vector of \({\mathcal P}\) is defined by \[ \sum_{n \geq 0} i({\mathcal P}, n)x^ n = {h^*_ 0 + h^*_ 1x + \cdots + h^*_ d x^ d \over (1-x)^ d}. \] The same definition applies to an \(L\)- polyhedral complex \({\mathcal X}\) in \(\mathbb{R}^ m\), which is a complex whose cells are lattice polytopes. The author then shows, first, that the \(h^*\)-vector of an \(L\)-polyhedral complex \({\mathcal X}\) whose underlying point-set \(| {\mathcal X} |\) is Cohen-Macaulay is nonnegative, and second, if \({\mathcal Y}\) is another Cohen-Macaulay \(L\)-polyhedron with \({\mathcal Y} \subseteq {\mathcal X}\), and if \(\dim {\mathcal Y} = e - 1\) and \({\mathcal Y}\) is contained in an affine \((e-1)\)-space, then \(h^* ({\mathcal Y}) \leq h^* ({\mathcal X})\).