Exercise 2.5#

A very flexible approach (versus a less flexible)#

Advantages#

  • Less bias.

  • Given enough data, better results.

Disadvantages#

  • More risk of overfitting.

  • Harder to train.

  • Longer to train.

  • Computational more demanding.

  • Less clear interpretability.

When is one approach preferable?#

More flexible#

  • Large sample size and small number of predictors.

  • Non-linear relationship between the predictors and response.

Less flexible#

  • Small sample size and large number of predictors.

  • High variance of the error terms.