Class 20: Introduction to Smoothing Splines

Review from previous class

• We reviewed the various methods for smoothing, working from the class handout.

Smoothing Splines

Lisp script for today's class

Class 21

Handout

This is a draft. We'll add details in class for some of the proofs, etc.

Loss Function

The loss function which leads to splines is a penalized sum of squares. Among all functions with the same fidelity to the data, cubic splines minimize this integrated second derivative penalty, and are thus the smoother of choice for this criterion.

Computing a Spline

Forget smoothing for a moment. How does on compute a spline. We have lots of choices, most of which resemble a regression function of sorts, with various choices for the regression design matrix. But how does one use these things for smoothing. You have to add in the loss function. One formulation, the one which uses B-splines, gives a useful form for the loss function. An important alternative is to remove knots. This different strategy leads to the "turbo smoother" of Friedman and Silverman.

Bayesian Spin

Though it may be surprising, other arguments lead to the same time of estimating equations. These penalized least squares estimators can also be motivated from a "shrinkage perspective" (as in ridge regression) or by posing a prior distribution located at zero.

Next time