Loss function

What is a loss function?

When training a machine learning model, we typically try to reduce the number of errors it makes. That is, our objective is to minimize the error of the model. A loss function (also called a cost function) is a function that allows us to evaluate this error. It maps the predictions of the model onto a single number that represents the cost of a mistake. The smaller is the number produced by the loss function, the better is the model’s performance.

There are many various loss functions used for different algorithms. Some of the most commonly used are: mean squared error, hinge loss, cross-entropy, etc. 

Why is it important?

Loss functions are an integral part of any optimization algorithm. They show how well the model performs and how it should be adjusted. It is vital to choose a correct loss function since their results will differ depending on the task.