2. Residuals

Residuals are vital to regression because they establish the credibility of the analysis. Never accept a regression analysis without having checked the residual plots.

Residuals come in many flavors:

Plain vanilla residual: $e_i = (y_i - \hat y_i)$.

Standardized residual:

$$s_i = \frac{e_i}{\sqrt{\widehat{Var}(e_i)}}.$$

Studentized residual: t_i.