Skip to content
Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

Recipe

Retrieve images similar to a query

Use when: you have a query image, a gallery of candidate images, and need the closest matches

Goal: a ranked list of the k closest gallery images, with their similarity scores

Taught in Retrieval pipelines and why generic embeddings win

Quick recipe

  1. Choose and freeze an encoder
  2. Embed the whole gallery once, offline
  3. Embed the query with the identical encoder
  4. Rank the gallery by cosine similarity to the query
  5. Return the top k
  1. 1

    Pick a trained image-embedding encoder. Section 3's own case below makes this almost always a strong, broad pretrained one. It's rarely something trained from scratch for this one task.

    Why: The encoder defines the latent space every later step operates in. Swapping it partway through invalidates every embedding already computed against the old one.

  2. 2

    Run every gallery image through the encoder and store its embedding vector.

    • the gallery is fixed (a static product catalog) — embed it once and cache every vector.
    • the gallery grows over time (new uploads) — embed each new item as it arrives and append its vector to the stored set.

    Why: Gallery embeddings don't depend on any particular future query. Computing them once, and reusing them for every search, is what makes retrieval fast at query time.

  3. 3

    Run the query image through the exact same encoder used in Step 1, with the same preprocessing (resizing, normalization).

    Why: Cosine similarity is only meaningful when both embeddings came from the same map into the same latent space. A different encoder places points in a different, incomparable space. That's true even if the two spaces happen to share a dimension count.

  4. 4

    Compute cosine similarity between the query embedding and every stored gallery embedding. Sort the results in decreasing order.

    Example

    With query \mathbf{q}=(4.2,0.8) against three stored gallery embeddings (4,1), (5,0), (3.5,1.5), Section 2's worked examples already computed this exact ranking: \approx0.998, \approx0.982, \approx0.977. That's decreasing order, so the first vector is the top match.

    Why: This is the step that turns "how similar are these two points" into an actual ordered list of candidates. It uses exactly the formula Section 2 derived.

  5. 5

    Keep the k highest-scoring gallery images as the result, along with their similarity scores.

    Why: A user or a downstream system needs a short, usable answer, not a full ranked list of the entire gallery.

Common mistakes

  • Re-embedding the gallery with an updated or different encoder, without recomputing every stored vector. Comparing embeddings from two different encoders is meaningless, even though nothing crashes.
  • Ranking with plain Euclidean distance on un-normalized embeddings. This silently reintroduces the distortion Section 2's worked example showed.
  • Forgetting the query needs the identical preprocessing the gallery images went through before embedding: resize, crop, and normalization.