On the stability of cones in \(\mathbb R^{n+1}\) with zero scalar curvature (Q2573743)

Let \(F^n\to \mathbb R^{n+1}\) be an orientable hypersurface, and let \(S_j\), \(j=\overline{0,n}\), be the elementary symmetric functions of its principal curvatures \(k_1\), ..., \(k_n\). If \(S_{j+1}=0\), then \(F^n\) is a critical point of the functional \(A_j=\int_F S_j\, dM\) for variations with compact support \textit{R. Reilly}, [J. Differ. Geom.\, 8, 465--477 (1973; Zbl 0277.53030)]. This construction generalizes the notion of minimal hypersurfaces (\(j=0\)). A general notion of stability for hypersurfaces with \(S_{j+1}=0\), which also naturally generalizes the notion of stability for minimal hypersurfaces, was proposed in \textit{H. Alencar, M. P. do Carmo, M. F. Elbert}, [J. Reine Angew. Math. 554, 201--216 (2003; Zbl 1093.53063)]. Actually the authors analyze the stability of hypersurfaces in \(\mathbb R^{n+1}\) with \(S_2=0\). A bounded domain \(D\) on an orientable hypersurface \(F^n\to \mathbb R^{n+1}\) is stable, if \(d^2A_1/dt^2 (0) >0\) for all variations with support in \(D\), provided that the ellipticity condition \(S_3\not= 0\) holds. The authors discuss in detail the stability of \textsl{truncated cones}. Namely, let \(C(M)\) be a cone generated by a hypersurface in the unite sphere, \(M^{n-1}\to S^n_1\subset \mathbb R^{n+1}\). The part of \(C(M)\) which belongs to the closure of the ring bounded by the concentric spheres \(S^n_\varepsilon\) and \(S^n_1\), \(0<\varepsilon <1\), is called a truncated cone and denoted by \(C_\varepsilon(M)\). It is quite easy to see that \(C_\varepsilon(M)\) satisfies \(S_2=0\), \(S_3\not= 0\) if and only if the same conditions are satisfied by \(M\). The authors prove two theorems which reveal how the stability of truncated cones depends on the dimension \(n\). Theorem 1: Let \(M^{n-1}\), \(n\geq4\), be an orientable, compact hypersurface of \(S^n_1\) with \(S_2=0\) and \(S_3\not= 0\) everywhere. If \(n\leq 7\), then there exists an \(\varepsilon >0\) such that the truncated cone \(C_\varepsilon(M)\) is not stable. Theorem 2: If \(n\geq 8\), then there exist an orientable, compact hypersurface \(M^{n-1}\) of \(S^n_1\) with \(S_2=0\) and \(S_3\not= 0\), such that for all \(\varepsilon >0\) the truncated cone \(C_\varepsilon(M)\) is stable. An open question, which presents an additional motivation to study these theorems, is whether there exists an elliptic graph in \(\mathbb R^n\), \(n\geq 4\), with vanishing scalar curvature.