An index theory and existence of multiple brake orbits for star-shaped Hamiltonian systems (Q1101752)

Let \(x=(p,q)\in {\mathbb{R}}^ N\times {\mathbb{R}}^ N\) and \(H\in C^ 2({\mathbb{R}}^{2N},{\mathbb{R}})\). The paper is concerned with the existence of periodic solutions of the Hamiltonian system of differential equations \((HS)\quad \dot x=JH'(x)\) on a prescribed energy surface \(S=H^{-1}(1)\). Such solutions are called brake orbits if p(t) is odd and q(t) is even in t. Suppose that H satisfies the following conditions: (i) \(H(- p,q)=H(p,q)\); (ii) The set \(\{x\in {\mathbb{R}}^{2N}:\) H(x)\(\leq 1\}\) is compact, star-shaped with respect to the origin, and S is its boundary; (iii) \(x\cdot H'(x)\neq 0\) for all \(x\in S\). Let R be the smallest number for which \(| x| \in {\mathbb{R}}\) for all \(x\in S\) and \(\rho\) the largest number such that no tangent hyperplane to S intersects the ball \(| x| <\rho\). The main result of the paper asserts that if \(R^ 2<2\rho ^ 2\), then (HS) has at least N geometrically distinct brake orbits on S. For the proof the problem is reduced (in a standard way) to the one of finding critical points of the action functional on a manifold. In contrast to the usual situation, this functional is only partially \(({\mathbb{Z}}_ 2\)-) symmetric in the space considered. In order to obtain multiple critical points, we construct a new relative index which is a variant of the one introduced by \textit{H. Berestycki}, \textit{J.-M. Lasry}, \textit{G. Mancini} and \textit{B. Ruf} in the paper [Commun. Pure Appl. Math. 38, 253-289 (1985; Zbl 0542.58029)]. A special feature of our index is a strong dimension property (roughly speaking, it asserts that there exist sets of arbitrarily large index which are unbounded and radially homeomorphic to the unit sphere).