Skip to content
Free public beta until December 4, 2026: every course is open, no card needed. After that, the first chapter of each course stays free and full access is a one-time $39 ($19 for up to 100 founding members chosen for helpful feedback). Your progress is kept. How the beta works

Recipe

Match features between two images

Use when: you have two images of a related scene and need point correspondences reliable enough to feed a downstream fit: a homography, a fundamental matrix, or any other relationship between the two views

Goal: a set of point correspondences clean enough that a downstream geometric fit can trust every one of them

Taught in The feature-matching recipe

Quick recipe

  1. Detect keypoints in both images
  2. Compute a descriptor at every keypoint
  3. Find each keypoint's nearest neighbor
  4. Apply the ratio test
  5. Fit a model with RANSAC
  6. Accept the final inlier set
  1. 1

    Run a corner or keypoint detector: Harris, Shi-Tomasi, or FAST (Chapter 4), independently on each image.

    Why: a correspondence only makes sense between points that are themselves repeatable and easy to pin down. The pipeline starts from keypoints, never from arbitrary pixels.

  2. 2

    Build a descriptor vector around each keypoint in both images: SIFT's gradient histogram, or a binary descriptor like ORB/BRIEF (Chapter 4).

    • the descriptor is a real-valued vector (SIFT-style) — Step 3 compares it by Euclidean distance.
    • the descriptor is a bit string (ORB, BRIEF) — Step 3 compares it by Hamming distance instead.

    Why: two keypoints can only be compared once each one has a fixed-length list of numbers for its own local area.

  3. 3

    For every keypoint descriptor in image A, search image B's descriptors. Keep the single closest one, using whichever distance Step 2 picked.

    Example

    A toy SIFT-style descriptor (0.2, 0.5, 0.1, 0.8) against a candidate (0.3, 0.4, 0.2, 0.9) gives a Euclidean distance of 0.2, Section 1's own worked example.

    Why: the closest descriptor is the strongest single candidate for "the same physical point," before any filtering happens.

  4. 4

    Compare the best match's distance d_1 to the second-best's distance d_2. Keep the match only if r = d_1/d_2 is below a threshold (usually 0.7-0.8).

    Example

    d_1 = 0.2, d_2 = 0.5 gives r = 0.4, well under 0.7, so this match survives to the next step.

    Why: a good match should be clearly closer than every other candidate. A near-tie means the descriptor check genuinely can't tell two candidates apart.

  5. 5

    Sample a minimal set from the matches that survived Step 4, many times over. Fit the geometric model each time, count inliers, and keep the best-scoring fit across k iterations (Section 2).

    Why: even past the ratio test, sometimes a match is wrong in a sneaky way. It looks fine on its own, but it doesn't fit the shape every other match agrees on. RANSAC is the only step that checks matches against EACH OTHER, instead of one at a time.

  6. 6

    The points RANSAC's best-fitting model marked as inliers are the correspondences the rest of the pipeline can trust. Discard everything else.

    Why: this is the actual deliverable: a correspondence set clean enough for that downstream fit (Chapter 6 onward) to build on without re-checking it.

Common mistakes

  • Running RANSAC before the ratio test, instead of after, wastes iterations. It fits models against matches that were already unclear at the descriptor level.
  • Comparing SIFT descriptors with Hamming distance, or ORB/BRIEF descriptors with Euclidean distance, is a mismatch. Each descriptor family needs its own matching distance.
  • Treating "passed RANSAC" as a sure thing, instead of just a likely one, is a mistake. The RANSAC consensus guarantee (Section 2) gives a confidence level chosen by k, not a promise.
  • Picking a ratio-test threshold with no regard for the actual data is a mistake too: 0.7-0.8 is a common default, not a rule that always applies.