Restrictions on maximal invariant subgroups implying solvability of finite groups (Q1736329)

Actions of groups on groups by automorphisms play a major role in the study of finite groups. An interesting case of such actions is the coprime action: two groups \(A\) and \(G\) of coprime indices such that \(A\) acts on \(G\) as a group of automorphisms. The aim of this paper is to obtain the solubility of an \(A\)-group \(G\) such that \(\gcd(|A|, |G|)=1\) when we impose some restriction on its maximal \(A\)-invariant proper subgroups. The first main result can be understood as a generalisation of Schmidt's theorem on minimal non-nilpotent groups. Theorem A. Let \(G\) and \(A\) be finite groups of coprime orders such that \(A\) acts on \(G\) by automorphisms. If every maximal \(A\)-invariant proper subgroup of \(G\) is nilpotent but \(G\) is not, then \(G\) is soluble and \(|G|=p^aq^b\) for two distinct primes \(p\) and \(q\), and \(G\) has a normal \(A\)-invariant Sylow subgroup. The second main result generalises a theorem of Thompson. Theorem B. Let \(G\) and \(A\) be finite groups of coprime orders such that \(A\) acts on \(G\) by automorphisms. If \(G\) has a nilpotent maximal \(A\)-invariant subgroup of odd order, then \(G\) is soluble. Some extensions of Theorem A are also proved. Theorem C. Let \(G\) and \(A\) be finite groups of coprime orders such that \(A\) acts on \(G\) by automorphisms. If every non-nilpotent maximal \(A\)-invariant proper subgroup of \(G\) has prime index, then \(G\) is soluble. Theorem D. Let \(G\) and \(A\) be finite groups of coprime orders such that \(A\) acts on \(G\) by automorphisms and let \(p\) be a prime divisor of \(|G|\). If the indices of all non-nilpotent maximal \(A\)-invariant subgroups of \(G\) are powers of \(p\), then \(G\) is soluble. Theorem E. Let \(G\) and \(A\) be finite groups of coprime orders such that \(A\) acts on \(G\) by automorphisms. If every maximal \(A\)-invariant proper subgroup of \(G\) is supersoluble, then \(G\) is soluble.