Recipe

Calibrate a camera

Use when: you have a real camera whose intrinsics, extrinsics, and lens distortion are unknown. You also have a calibration target (often a checkerboard) whose true geometry you already know exactly

Goal: a camera matrix P = K[R|t] per observed pose, plus distortion coefficients. Both need to be good enough that a downstream computation (Chapters 6, 8, 9) can trust every pixel measurement this camera produces

Taught in The camera calibration recipe

1. 1

   Photograph a calibration target -- often a checkerboard -- at several different poses (angles and distances). Mark every corner's pixel position in each photo.

   Example

   Three photos of an 8\times8 checkerboard mark roughly 50 interior corners per photo. That gives 150 known-3D-position-to-observed-pixel correspondences to work from.

   Why: Calibration is a controlled experiment. The target's true 3D geometry has to already be known, ahead of time. Otherwise there is nothing to compare an observed image against.

2. 2

   For every observed corner, write one equation. It relates the corner's known 3D position, in the target's own coordinate frame, to its observed pixel position. This goes through the unknown \mathbf{P}=K[R\mid\mathbf{t}] for that pose.

   • the target is planar (a flat checkerboard, the common case) — the homography Chapter 6 built between the target's own plane and each image supplies a fast starting guess for K. That guess comes before the full nonlinear solve.
   • the target is a genuine 3D rig (corners at varying depths) — every corner's 3D position already varies in Z. The equations are set up directly, with no planar shortcut available.

   Why: Every correspondence is one equation constraining the unknown parameters. More correspondences across more poses is what makes an otherwise unsolvable system over-determined enough to solve.

3. 3

   Solve the resulting system for K -- shared across every single pose -- and one R, \mathbf{t} pair per pose. Typically this uses least squares. Find the parameters that minimize the total gap between every predicted and every observed pixel position. Do this across every pose at once.

   Why: Pin K identical across every pose. Let R,\mathbf{t} vary freely per pose instead. That combination is exactly the constraint that breaks a problem: the focal-length-versus-distance problem Section 2's own worked example names. A single pose alone can't do this.

4. 4

   Fit the radial (k_1, k_2, k_3) and tangential (p_1, p_2) distortion coefficients alongside K and every pose's R,\mathbf{t}. This is usually part of the same least-squares solve, rather than a separate afterthought.

   ⚡ Shortcut

   For a lens with only mild distortion -- a standard, non-wide-angle lens -- fitting k_1 alone first is often enough. Use it as a single-term model, for an initial working guess. Add k_2 and, rarely, k_3, only if the reprojection error in Step 5 stays too high.

   Why: Leaving distortion out of the solve doesn't make it disappear. It gets silently absorbed into a slightly wrong K and slightly wrong R,\mathbf{t} instead. That corrupts every later measurement, in a way nothing about the numbers alone would flag.

5. 5

   Project every known 3D corner back through the solved \mathbf{P}, with the fitted distortion applied. Measure how far each predicted pixel sits from where that same corner was actually observed. That gap is the reprojection error e, averaged across every corner and pose.

   Why: Reprojection error is the one genuine external check, that the whole solve actually explains the data. It is not just a number a solver reports with no way to audit it.