Segmentation Maps Border Treatment broken
Author: thetoby9944Created Sep 13, 2021Updated Jul 2, 2024
Summary: Applying augmentations with SegmentationMapOnImage ignores the mode parameter. Any transformations that create a void image area (affine, pad, ...) is executed with constant border treatment only on the labels.
Example:
def segmentation_map_aug(
self,
image: np.array,
label: np.array
) -> (np.array, np.array):
label = SegmentationMapsOnImage(label, shape=image.shape)
image, label = iaa.Affine(
rotate=(-180, 180),
mode="reflect"
)(
image=image,
segmentation_maps=label
)
return image, label.get_arr()Observed Behavior: The image content gets reflected where the rotation would leave a void area - But the corresponding labels fill the void area black, instead of also reflecting the content. This creates a mismatch in image content and segmentation map content.
Expected Behavior: When the content of an image is reflected during rotation or padding, the corresponding segmentation map should be reflected as well. This also applies to other border treatment modes.
Source: aleju/imgaug