Bayer filter and demosaicing

Most camera sensors place a Bayer filter -- a repeating mosaic of red, green, and blue filters, with twice as many green cells as red or blue -- directly over the sensor grid, so each physical pixel records only one color channel. Demosaicing is the interpolation step that estimates the two missing color values at every pixel from its neighbors, reconstructing a full RGB image from the raw mosaic. The simplest version, bilinear interpolation, averages a missing channel's value from the nearest neighboring pixels -- at most four, directly up/down/left/right, called the orthogonal neighbors -- that actually recorded it.