#788·imgaug

Segmentation Maps Border Treatment broken

Author: thetoby9944Created Sep 13, 2021Updated Jul 2, 2024

Summary: Applying augmentations with SegmentationMapOnImage ignores the mode parameter. Any transformations that create a void image area (affine, pad, ...) is executed with constant border treatment only on the labels.

Example:

python
def segmentation_map_aug(
    self, 
    image: np.array, 
    label: np.array
) -> (np.array, np.array):

    label = SegmentationMapsOnImage(label, shape=image.shape)
    image, label = iaa.Affine(
        rotate=(-180, 180),
        mode="reflect"
    )(
        image=image, 
        segmentation_maps=label
    )
    return image, label.get_arr()

Observed Behavior: The image content gets reflected where the rotation would leave a void area - But the corresponding labels fill the void area black, instead of also reflecting the content. This creates a mismatch in image content and segmentation map content.

Expected Behavior: When the content of an image is reflected during rotation or padding, the corresponding segmentation map should be reflected as well. This also applies to other border treatment modes.