self-supervised pretraining

Learning useful features from unlabeled images alone, with no labels at all during pretraining.

Self-supervised pretraining learns useful features from unlabeled images alone, with no labels at all during pretraining, unlike ordinary supervised pretraining (such as ImageNet-style pretraining), which trains directly on labeled examples. Because a self-supervised backbone's features never depended on having abundant labels in the first place, they often transfer better to a target task with only a small labeled dataset, sometimes usable frozen or with only a lightweight fine-tune.

Examples

A backbone pretrained with self-supervised learning on unlabeled photos can transfer more readily to a rare-disease dataset of only a few hundred labeled images than a supervised ImageNet backbone would.