Encoding of data sets and algorithms (Q6546951)

The insurance of the quality of the output of a machine learning algorithm as well as its reliability in comparison to the complexity of the algorithm used constitutes a key element in many applications. In this contribution, the authors present a mathematically rigorous theory in order to take a decision about what models as algorithms applied on data sets are close to each other in terms of certain metrics, such as performance and the complexity level of the algorithm. The mathematical control of the decision is given in terms of metric entropy and capacity on certain functional spaces.\N\NGiven a normed linear space \(X\) and \(K,\) a compact subset, for any \(\epsilon>0\) let \(\mathcal{N}_{\epsilon}(K)\) be the minimal value of \(n\) such that there exists an \(\epsilon\)-net of \(K\) consisting of \(n\) points. The entropy of \(K\) is defined as \(H_{\epsilon}(K)= log \mathcal{N}_{\epsilon}(K).\) On the other hand, let \(\mathcal{M}_{\epsilon}(K)\) be the maximum value of \(m\) for which there exist \(m\) \(\epsilon\)-separable points for \(K\). The capacity of \(K\) is defined as \(C_{\epsilon}(K)= log \mathcal{M}_{\epsilon}(K).\)\N\NAnalytic functions on an ellipsoid in \(\mathbb{C}^{q}\) as well as entire functions of exponential type defined in \(\mathbb{C}^{Q}\) are introduced. In such a way, compact spaces of analytic and entire functions are defined. Estimates of the metric entropy of a compact subset of functions of infinitely many variables that arise in the definition of these spaces constitute the core of the contributions of the paper under review.\N\NThe metric entropy and capacity related to a finite number of balls of a given radius to cover a compact set are the basic tools in order to get lower and upper bounds for such estimates. On the other hand, the probability measures analyzed therein have densities which are analytic while the functionals are entire functions of exponential type defined on an infinite-dimensional sequence space. These functionals can be viewed as acting on the sequence of Fourier coefficients of the input function with respect to multivariate Chebyshev polynomials.\N\NFinally, computational issues are discussed in the framework of the generation of analytic and bandlimited functions as well as the generation of \(\epsilon\)-nets on ellipsoids, see Kernel-based analysis of massive data [\textit{H. N. Mhaskar}, Front. Appl. Math. Stat. 6, Article ID 30, 18 p. (2020; \url{doi:10.3389/fams.2020.00030})].