Recipe

Produce an instance segmentation map with Mask R-CNN

Use when: you need a separate mask for each individual object, not just a per-class map

Goal: a list of detected objects, each with its own class, confidence score, and pixel mask

Taught in FCN, U-Net, and Mask R-CNN

1. 1

   Run the detection stage exactly as Chapter 13 taught it: propose candidate boxes likely to contain an object.

   Why: Mask R-CNN's first stage is an ordinary two-stage detector. Nothing about proposing regions changes just because a mask comes later.

2. 2

   Assign a class and a confidence score to each proposed box, then adjust its coordinates. This is exactly Chapter 13's own classify-and-refine step.

   Why: the mask branch needs to know which class it's segmenting before it runs. A mask predicted with no class attached isn't useful on its own.

3. 3

   Run non-max suppression at the chosen confidence and IoU thresholds. This is exactly Chapter 13's own procedure, and it collapses duplicate boxes down to one per real object.

   Example

   Two boxes over the same car, scores 0.92 and 0.81, IoU 0.9 between them, threshold 0.5: the 0.92 box is kept, the 0.81 box is discarded before it ever reaches the mask branch.

   Why: running the mask branch on every proposed box, before removing duplicates, wastes computation. It would predict several near-identical masks for the same physical object.

4. 4

   For every box that survives non-max suppression, crop that region and run it through the added mask branch. That branch is a small per-pixel classifier that predicts a binary object/not-object mask inside that one box only.

   Example

   Two surviving boxes -- one over each of two touching cars -- each get their own 28\times 28-style crop and their own independently predicted mask. The two cars touch in the original photo. But their two crops -- and so their two masks -- never overlap in the output, because each mask is confined to its own box.

   Why: this is the one genuinely new piece Mask R-CNN adds. Running the mask branch separately per surviving box automatically gives each detected object its own private mask. So two same-class objects never share pixels in the final output.

5. 5

   List out every surviving box's class label, confidence score, and predicted mask as one separate detected instance. Scoring each predicted mask against ground truth is Section 3's own subject.

   Why: that list -- object identity plus a mask for each -- is exactly instance segmentation's own definition from Section 1. It's assembled here from pieces this recipe already computed.