fine-tuning

Continuing to train some or all of a pretrained model's own weights on a new task, usually at a low learning rate.

Fine-tuning continues training a pretrained model's own weights (some or all layers, not just a newly added head) on a new target task, typically at a low learning rate so the pretrained features are nudged toward the new task rather than overwritten. Fine-tuning generally needs more target-task data than plain transfer learning with a frozen backbone, but can reach higher accuracy when enough is available.

Examples

After transfer learning with a frozen backbone plateaus, unfreezing the backbone's last few layers and continuing training at a low learning rate squeezes out further accuracy.