Outlier residuals can have an undue effect on our ability to minimize errors (in part because they are squared) - especially for small datasets.

Outlier residuals - predictions that are especially far from their observed values, relative to other observations

Outlier observations - observed values are especially far from observed values, relative to other observations

Remember, outlier residuals are not the same as outlier observations!