Machine Learning — Overfitting and Underfitting

Rahul S
3 min readAug 23

In the realm of machine learning, the critical challenge lies in finding a model that generalizes well from a given dataset. This generalization is essential for the model’s ability to make accurate predictions on unseen data. However, two common pitfalls can hinder this goal: overfitting and underfitting.

Overfitting: The Curse of Excessive Complexity

Overfitting occurs when a machine learning model captures not only the underlying patterns in the data but also the noise and random fluctuations. As a result, an overfit model performs exceptionally well on the training data but fails to generalize to new, unseen data.

Overfitting typically arises when a model is too complex for the amount of training data available or when the model is allowed to train for too many iterations. The key challenge in dealing with overfitting is finding the right balance between model complexity and data fitting.

Underfitting: The Pitfall of Oversimplification

An underfit model is too simple to capture the underlying patterns in the data. It fails to learn from the training data effectively, resulting in poor performance both on the training data and unseen data. This could be due to selecting an overly simplistic algorithm or not providing the model with enough training data.

Detecting Overfitting:

  1. If the model’s performance on the validation set is significantly worse than on the training set, it might be overfitting.
  2. Create learning curves by plotting the model’s training and validation performance (e.g., accuracy or loss) over time (usually epochs). If the training performance improves while the validation performance plateaus or degrades, overfitting is likely occurring.

Addressing Overfitting:

Reduce Model Complexity:

  • Decrease the number of model parameters or layers.
  • Use simpler model architectures.
  • Reduce the complexity of neural networks by adding dropout layers or reducing the number of hidden units.

Improving Data Quality: The Foundation for Accurate and Reliable Models

Machine Learning: Confusion matrix in classification problems

Machine Learning: Data Drift and Concept Drift

Machine Learning- Data Leakage

Machine Learning — Cost Function, An Introduction

Machine Learning: Cross Entropy and Cross-Entropy Loss

Machine Learning: Interpretation of Loss Function with Cross-Entropy Loss

Introduction to Gaussian Mixture Models (GMM) with Expectation-Maximization (EM)

DBSCAN: Intution, Advantages, and Points to Remember

Machine Learning: Regularization for Overfitting

Rahul S

I learn as I write | LLM, NLP, Statistics, ML