Rotation invariant subspaces of Besov and Triebel-Lizorkin space: Compactness of embeddings, smoothness and decay of functions. (Q1809935)

Let \(A^s_{pq} (\mathbb{R}^n)\) be either \(B^s_{pq} (\mathbb{R}^n)\) or \(F^s_{pq} (\mathbb{R}^n)\), the spaces mentioned in the title. Let \(RA^s_{pq} (\mathbb{R}^n)\) be the respective rotationally invariant subspaces. It is remarkable that the embedding \[ RA^{s_0}_{p_0 q_0} (\mathbb{R}^n) \hookrightarrow RA^{s_1}_{p_1 q_1} (\mathbb{R}^n), \quad s_0 > s_1, \quad s_0 - n/p_0 > s_1 - n/p_1 , \] is not only continuous as for the full spaces \(A^s_{pq} (\mathbb{R}^n)\), but even compact. Furthermore, elements of \(RA^s_{pq} (\mathbb{R}^n)\) with \(s > n/p\) show a specific singularity behaviour at \(0\) and \(\infty\). It is the main aim of this paper to extend these assertions to closed subgroups \(H\) of the orthogonal group \(O(n)\), where the orbits \[ \{ y\mid y= Hx {\text{ for some}} \;x \in \mathbb{R}^n \} \] play a role. The considerations are based on atomic decompositons.