Skip to content
Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

Recipe

Build an image classifier

Use when: you're handed a labeled image dataset and asked to build a working classifier from it

Goal: a trained, honestly-evaluated image classifier, with a documented reason behind every major choice along the way

Taught in Transfer learning and fine-tuning

Quick recipe

  1. Split the data before looking at it further
  2. Decide -- transfer learning, or train from scratch?
  3. Train with augmentation on the training set only
  4. Watch the train/validation loss curves and react
  5. Evaluate exactly once on the test set
  1. 1

    Set aside training, validation, and test sets.

    Why: every later decision needs an honest validation signal. The test set has to stay untouched until the very end (Section 3).

  2. 2

    Given the training set's size, apply Step 1 of the transfer-learning recipe above.

    • no suitable pretrained backbone exists for this domain, and there's enough labeled data — build and train a CNN from scratch (Sections 1–2's own recipe).
    • a pretrained backbone exists for a similar domain (the common case) — follow the transfer-learning-and-fine-tuning recipe above, start to finish.

    Why: transfer learning is far more data- and compute-efficient whenever a suitable pretrained backbone exists. It's the default choice unless a real reason rules it out.

  3. 3

    Apply label-preserving transformations to the training images as training runs; leave validation and test untouched.

    Why: augmentation fights overfitting by denying the model any one exact training image to memorize (Section 3).

  4. 4

    Diagnose overfitting or underfitting as soon as the gap (or lack of progress) appears.

    • overfitting appears — more augmentation, earlier stopping, or, if fine-tuning, freeze more of the backbone.
    • underfitting appears — train longer, unfreeze more layers, or, if training from scratch, increase capacity.

    Why: catching a bad training run early saves the rest of the compute budget. Otherwise it's spent on a network that was already failing (Section 3).

  5. 5

    Report this number, and only this number, as the model's real performance.

    Why: Section 3's own rule, and the transfer-learning recipe's own final step -- the test set is the only honest number, spent exactly once.

Common mistakes

  • Choosing an architecture or tuning hyperparameters by watching the TEST set instead of the validation set.
  • Skipping straight to training without checking whether a pretrained backbone already exists for a similar domain.
  • Declaring victory the moment training loss looks good, without ever checking the validation curve.