Penalized likelihood-type estimators for generalized nonparametric regression
type: Item
instance of: scholarly article
title: Penalized likelihood-type estimators for generalized nonparametric regression
zbMATH Open document ID: 0849.62022
author: Dennis D. Cox
author: Finbarr O'Sullivan
published in: Journal of Multivariate Analysis
publication date: 1996-08-05T00:00:00Z
review text: The method of maximum penalized likelihood has proved useful for a wide variety of nonparametric function estimation problems. In this method a smooth estimator of the parameter \(\theta\) is obtained by minimization of a ``penalized likelihood-type functional.'' To describe the method, suppose we are given data \((X_1, Y_1)\), \((X_2, Y_2), \dots, (X_n, Y_n)\). Then the penalized likelihood-type functional is \(l_{n \lambda} (\theta) = l_n (\theta) + (\lambda/2) J (\theta)\). The three ingredients of \(l_{n \lambda}\) are: (i) The smoothing parameter is \(\lambda > 0\). (ii) The likelihood component (which depends on the data) is \(l_n (\theta)\). We take it to be of the form \(l_n (\theta) = n^{-1} \sum_i \rho (Y_i \mid X_i, \theta)\) for some criterion function \(\rho\) which measures ``goodness of fit'' or ``fidelity to the data'', such as \(\rho (y \mid x, \theta) = [y - \theta (x)]^2\). (iii) The penalty functional is \(J (\theta)\). If \(\theta\) is real valued and \(x\) is one dimensional, then the most commonly used penalty functional is \(J (\theta) = \int [\theta^n (x)]^2 dx\), which gives rise to estimates which are cubic smoothing splines. Estimators of this type have been considered by a number of authors. The purpose of this paper is to develop first-order asymptotic approximations for such vector-valued nonparametric regression function estimators. The results are based on linear Taylor series expansions in infinite-dimensional spaces. An application of these approximations is to derive rates of convergence for the integrated squared error of the estimator and its derivatives. The approximations also provide insight into the estimation error which can be approximately decomposed into the sum of a bias (deterministic) term and a random term. The result on rates of convergence is stated in Section 2, along with assumptions that are used throughout the paper. In Section 3 we give two theorems that provide the details on the asymptotic linearization of t…
Mathematics Subject Classification ID: 62G07
Mathematics Subject Classification ID: 41A35
Mathematics Subject Classification ID: 41A25
Mathematics Subject Classification ID: 45M05
Mathematics Subject Classification ID: 65R20
Mathematics Subject Classification ID: 47A53
zbMATH DE Number: 892476
zbMATH Keywords: maximum penalized likelihood
zbMATH Keywords: nonparametric regression
zbMATH Keywords: multiple classification
zbMATH Keywords: smoothing splines
zbMATH Keywords: smoothing parameter
zbMATH Keywords: penalty functional
zbMATH Keywords: first-order asymptotic approximations
zbMATH Keywords: vector-valued nonparametric regression function estimators
zbMATH Keywords: linear Taylor series expansions
zbMATH Keywords: infinite-dimensional spaces
zbMATH Keywords: rates of convergence
zbMATH Keywords: integrated squared error
zbMATH Keywords: estimation error
zbMATH Keywords: Gaussian approximations
MaRDI profile type: Publication
OpenAlex ID: W2028204517
DOI: 10.1006/JMVA.1996.0010
description: scientific article; zbMATH DE number 892476