Harmonic analysis on nilpotent groups and singular integrals. II: Singular kernels supported on submanifolds (Q1103143)

This paper is the second in a series of papers by the two authors [begun in ibid. 73, 179-194 (1987; Zbl 0622.42010)]. The principal objective of the paper is the study of singular convolution operators on nilpotent Lie groups whose kernels are supported on lower dimensional varieties and which may possibly contain exponential polynomial oscillatory factors. To be more specific, let G be a simply connected nilpotent Lie group with Lie algebra \({\mathfrak g}\), and define dilations \(D_{\delta}\), \(\delta >0\), on \({\mathfrak g}\) by \(D_{\delta}X_ j=\delta^{a_ j}X_ j\), if \(X_ 1,...,X_ n\) is a fixed basis of \({\mathfrak g}\), where \(\alpha_ j>0\), \(j=1,...,n\). For a given decomposition \({\mathfrak g}={\mathfrak g}_ 1\oplus...\oplus {\mathfrak g}_ k\) of \({\mathfrak g}\) into dilation-invariant subspaces \({\mathfrak g}_ 1,...,{\mathfrak g}_ k\), define canonical coordinates by the mapping exp Y\(=\exp Y_ 1...\exp Y_ k\), if \(Y=Y_ 1+...+Y_ k\) with \(Y_ j\in {\mathfrak g}_ j\). Assume that V is a (connected) analytic homogeneous submanifold of \({\mathfrak g}\) not containing the origin, let \(d\sigma\) denote the surface measure of V, and let K be a smooth function on V such that the measure Kd\(\sigma\) is homogeneous of critical degree \(-Q=-\sum \alpha_ j\) and has mean value zero in an appropriate sense. Let P(x,y) be a real polynomial on \(G\times G\). The main theorem states that the operator \[ Tf(x)=p.v.\int_{V}e^{iP(x,\exp Y)}K(y)f(x(\exp Y)^{-1})d\sigma (Y) \] is bounded on \(L^ p(G)\) for \(1<p<\infty\), with a bound that depends only on K,p and the degree of P. This result is proved first in the special case where the dilations \(D_{\delta}\) are automorphisms of G and where P vanishes identically. The general case is derived from this case by a beautiful application of the ``method of transference''. Making use of this result respectively the technics devised for deriving it, the authors are also able to prove \(L^ p\)-estimates for certain maximal functions associated to the submanifold V.