The Bias–Variance Trade-off

In short

• •The session demonstrates how polynomial regression is used to fit data generated from a sinusoidal curve.
• •Higher-degree polynomials can fit every training data point exactly but often fail to capture the underlying ground truth curve.
• •This phenomenon illustrates the risk of overfitting, where low training error does not guarantee accurate generalization between data samples.

Key concepts

• Least Squared Error0:45
• Polynomial Degree Influence1:58
• Overfitting2:29

Chapters

1. 0:15Introduction to Polynomial Fitting
2. 0:36Understanding the Data Set
3. 0:58Error Term and Regression
4. 1:29Polynomial Degree Comparison
5. 2:29Overfitting Explained

Check yourself

1. 1. What is the ground truth curve in the experiment?

   Answer: A sinusoidal curve

   The speaker identifies the green curve as the sinusoidal curve from which the data points were drawn. 0:00

2. 2. How is the regression performed on the dataset?

   Answer: By using a least squared error term

   The session explains that a least squared error term is used to perform the regression. 0:45

3. 3. What is the result of using a 0-degree polynomial for fitting?

   Answer: It acts as a constant term

   The 0-degree polynomial is described as nothing but a constant term. 1:29

4. 4. What happens when a 9th-degree polynomial is used?

   Answer: The fit passes through every blue data point

   With a 9th-degree polynomial, the resulting red curve goes through each one of the blue data points. 1:58

5. 5. Why is the 9th-degree polynomial fit considered problematic?

   Answer: It is far from the ground truth between the sample points

   The model performs well on training data but deviates from the ground truth in the gaps between data points. 2:44