Integral with respect to nonadditive measure and estimates of indicator of entire functions (Q795202)

Let us consider a family X consisting of open sets \(\theta\) on \(S_ 1=\{z\in {\mathbb{R}}^ 2:| z| =1\},\) and assume that \(S_ 1\in X\). Let \(\delta\) (\(\theta)\) be a nonadditive measure on X. Denote \(K_{r,\theta}=\{z:| z|<r,\quad \arg \quad z\in \theta \}.\) Let \(n_ f(K_{r,\theta})\) be a number of zeros in \(K_{r,\theta}\) of an entire function f of normal type with a proximate order \(\rho\) (r)\(\to \rho\), and let \({\bar \Delta}'(\theta,f)=\lim_{r\to \infty}r^{- \rho(r)}n_ f(K_{r,\theta})\) be the upper angular density of zeros of f. Denote at last \(A'(\delta,X)=\{f:{\bar \Delta}'(\theta,f)\leq \delta(\theta),\theta \in X\}.\) The main result is the theorem: if the nonadditive measure \(\delta\) satisfies the equality \(\delta(\theta)=\delta({\bar \theta})\), for all \(\theta \in X\), then \[ \sup \{h_ f(\phi):f\in A'(\delta,X)\}=X_{(z)}-\int_{R^ 2}H^+(e^{i\phi}/\zeta,[\rho])d\delta_{(z)}(\zeta). \] There exists a function \(f\in A'(\delta,X),\) such that the equality sign holds for any \(\phi \in S_ 1\). Here \(h_ f\) is the Phragmén-Lindelöf indicator of f, \(H^+\) is the Weierstraß primary factor, and on the right side X-\(\int d\delta\) is the integral on a nonadditive measure introduced and studied by the author. This investigation continued and refines the works of \textit{A. A. Gol'dberg} [Mat. Sb., Nov. Ser. 58(100), 289-334 (1962; Zbl 0121.291), ibid. 61(103), 334-349 (1963; Zbl 0141.076), ibid. 65(107), 414-453 (1964; Zbl 0141.077), ibid. 66(108), 411-457 (1965; Zbl 0141.077)) and \textit{V. S. Azarin} in Teor. Funkts. Funkts. Anal. Prilozh. 18, 18-50 (1973; Zbl 0285.30019).