On the second fundamental theorem of invariant theory for the orthosymplectic supergroup (Q1703227)

Suppose V is an orthosymplectic vector superspace of superdimension \((m|2n)\) on which the orthosymplectic supergroup acts \(\mathrm{OSp}(V)\). Let \(B_r(m-2n)\) be the Brauer algebra of degree \(r\) with parameter \(m-2n\). A result of \textit{G. I. Lehrer} and \textit{R. B. Zhang} [Commun. Math. Phys. 349, No. 2, 661--702 (2017; Zbl 1360.22027)] showed that there exits a surjective algebra homomorphism \(F_r^r:B_r(m-2n)\longrightarrow \mathrm{End}_{\mathrm{OSp}(V)}(V^{\otimes r})\) and the kernel \(\mathrm{Ker}(F_r^r)\) is a two-sided ideal of \(B_r(m-2n)\). To describle this kernel is usually a difficult task, which is called the second fundamental theorem of invariant theory in a category theoretical setting. The paper under review gives a sufficient and necessary condtion to when the kernel \(\mathrm{Ker}(F_r^r)\) is nonzero, completely confirming a conjecture of \textit{G. Lehrer} and \textit{R. Zhang} [Ann. Math. (2) 176, No. 3, 2031--2054 (2012; Zbl 1263.20043)]. The author also gives explicit generating sets for some special kernels. Some applications were obtained in the description for the endomorphism algebra of the tensor representation \(V^{\otimes r}\) of the orthosymplectic Lie superalgbra.