Conformal Prediction for Image Segmentation Using Morphological Prediction Sets

MICCAI 2025. 📄 arXiv. bibtex.

Luca Mossina,¹ Corentin Friedrich¹

¹ IRT Saint Exupéry, Toulouse, France.

• Research Lab: DEEL, Dependable, Explainable & Embeddable Learning for trustworthy AI.
• Lab's open-source software and papers

Idea

We use morphological operations (dilation, sequences of dilations, etc.) to add a margin $\mu_{\lambda}(\hat{Y})$ around a predicted (binary) segmentation mask $\hat{Y}$, such that the ground-truth mask $Y$ is covered with high probability, and false negative pixels are statistically controlled.

To make this statistically rigorous, we use conformal prediction: using calibration data, we find the minimal number of dilations $\lambda$ (applied to the predicted mask) needed to cover the ground truth, on average. We write $\delta^{\lambda}(\hat{Y})$ to say that we apply dilation $\lambda$ times, at each step adding a margin of pixels to the predicted mask $\hat{Y}$. The choice of structuring element (e.g., a cross, square, or disk) is arbitrary, the users can craft any kind of element or sequence of operations, as long as $\hat{Y}$ can "grow" in all directions and cover the whole ground truth mask $Y$.

This gives us a prediction set $C_{\lambda}(\hat{Y}) = \hat{Y} \cup \mu_{\lambda}(\hat{Y}) = \delta^{\lambda}(\hat{Y})$, which is a set of pixels that are either predicted or added by the dilation operation. The prediction set is guaranteed to cover the ground truth with a user-defined probability $\geq 1 - \alpha$, e.g., 90%:

$$\mathbb{P}(Y \subseteq C_{\lambda}(\hat{Y})) \geq 1 - \alpha$$

This is a nonparametric method, which does not require any training or hyperparameter tuning, and is model-agnostic: it can be applied to any segmentation model, including deep learning models, classical methods, or even human annotators.

• requirement: having a set of (previously unseen) annotated calibration pairs $(X_i, Y_i)_{i=1}^n$, that are i.i.d. samples from the same distribution as the test data.

Synthetic example

The following example illustrates the idea of conformal prediction with morphological operations.

In the following image, we have a ground truth mask (in red) and a predicted mask (in blue). In purple, we have the pixels that were correctly predicted. The remaining red ones, are false negatives, i.e. pixels that belong to the ground truth but were not predicted.

The animation shows a sequence of four dilations by a $(3 \times 3)$ cross structuring element, which expand the margin of the predicted mask (darker blue, fig. above). Four iterations is the minimal number of iterations needed, i.e. the nonconformity score for this specific image: all missing pixels are recovered (shown in orange).

Examples

The directory notebooks contains complete examples for the datasets:

• WBC and OASIS, using the UniverSeg segmentation model
• polyps tumors dataset, using PraNet (we use precomputed predictions as distributed by A. Angelopoulos.

References & sources

Starting points for datasets:

• WBC
• OASIS
• polyps tumor data

Models used:

• UniverSeg. code, paper
• PraNet. paper

Citation

@article{Mossina_2025_conformal,
  title={Conformal Prediction for Image Segmentation Using Morphological Prediction Sets},
  author={Mossina, Luca and Friedrich, Corentin},
  journal={arXiv preprint arXiv:2503.05618},
  year={2025}
}