Overfitting and Underfitting in Deep Learning

Hello,

In this article, I will discuss the concepts of “Overfitting” and “Underfitting,” which are commonly encountered in deep learning.

While working on my master’s thesis in deep learning, I faced similar problems. I decided to write this article to share some small details for those who experience this issue while progressing with their models trained on a given dataset.

First, let’s understand these concepts. Overfitting refers to the situation where the error rate of our developed model during the learning process is very low or almost non-existent. On the other hand, Underfitting can be described as the opposite, where the developed model has a high error rate that is considered unsuccessful.

So, what can we do to deal with both situations? To address the Overfitting problem, we can reintroduce our model to the learning process by increasing the erroneous data augmentation in the dataset, which is a classic approach. Additionally, we can apply L1/L2 regularization.

L1/L2 regularization involves two loss functions, L1 and L2, aimed at reducing errors in machine learning. You can refer to this source on the internet for a numpy example of its implementation.

To tackle Underfitting, we can provide solutions from two perspectives. We can increase the number of neurons in the layers used during the learning process or add more layers. Another solution could be to extend the learning process slightly.

I hope this information proves useful to you. If you have any further questions, feel free to ask.