What is meant by fine-tuning of neural network?

Question

In many research articles on neural network, I read the term "fine-tuning". As far as I understood, this is not a selection of parameters. Could someone please explain what it exactly means?

Answer 1

Finetuning means taking weights of a trained neural network and use it as initialization for a new model being trained on data from the same domain (often e.g. images). It is used to:

1. speed up the training
2. overcome small dataset size

There are various strategies, such as training the whole initialized network or "freezing" some of the pre-trained weights (usually whole layers). The article A Comprehensive guide to Fine-tuning Deep Learning Models in Keras provides a good insight into this. Also have a look at the following threads:

• Fine Tuning vs Joint Training vs Feature Extraction
• CNN: ReTraining and Fine Tuning