fix(_automation): oriented rectangle can omit pixels from rotated masks

This was generated by AI during a kaizen pass and updated after review.

Summary

When AI Assist or AI Text Prompt converts a Mask to an oriented-rectangle Shape, compute_oriented_rectangle_from_mask fits the minimum-area rectangle to foreground pixel centers. compute_polygon_from_mask uses the half-pixel boundary instead. The two conversion paths therefore use different Mask extent conventions.

The center-based result is not always a user-visible under-size:

• An axis-aligned 10 by 6 Mask produces continuous side lengths of 9 by 5.
• The resulting Shape rasterizes back to the exact original 60 Mask pixels because labelme uses inclusive pixel coordinates.

Rotated Masks have a real coverage problem. With the existing positive 30-degree test fixture, the source Mask has 83 pixels. Rasterizing the generated oriented-rectangle Shape gives 84 pixels, with 6 source pixels missing and 7 extra pixels.

This issue is therefore about two related points:

1. The package needs one documented Mask extent contract.
2. The current rotated Shape does not reliably enclose the source Mask pixels.

Where

labelme/_automation/_geometry.py:

• compute_oriented_rectangle_from_mask builds the hull from ys, xs = np.nonzero(mask). These coordinates are pixel centers.
• _min_area_rect uses u_max - u_min and p_max - p_min without an outer boundary margin.
• compute_polygon_from_mask traces the Mask at its half-pixel boundary.

labelme/_automation/_shape_builders.py:

• _oriented_rectangle_for_detection converts the result into the oriented-rectangle Shape used by AI Assist and AI Text Prompt.

Reproduction

import math

import numpy as np

from labelme._automation._geometry import compute_oriented_rectangle_from_mask
from labelme._utils.shape import shape_to_mask

# Axis-aligned Mask: vector dimensions are 9 by 5, but raster coverage is exact.
mask = np.zeros((20, 20), dtype=bool)
mask[5:11, 3:13] = True
corners = compute_oriented_rectangle_from_mask(mask)
output = shape_to_mask(
    img_shape=mask.shape,
    points=corners.tolist(),
    shape_type="oriented_rectangle",
)
assert np.array_equal(output, mask)

# Existing positive 30-degree fixture.
angle = math.pi / 6
cos_a = math.cos(angle)
sin_a = math.sin(angle)
ys, xs = np.mgrid[0:40, 0:40]
dx = xs - 20.0
dy = ys - 20.0
local_x = dx * cos_a + dy * sin_a
local_y = -dx * sin_a + dy * cos_a
mask = (np.abs(local_x) <= 10.0) & (np.abs(local_y) <= 2.0)

corners = compute_oriented_rectangle_from_mask(mask)
output = shape_to_mask(
    img_shape=mask.shape,
    points=corners.tolist(),
    shape_type="oriented_rectangle",
)

assert mask.sum() == 83
assert output.sum() == 84
assert (mask & ~output).sum() == 6
assert (~mask & output).sum() == 7

Extent contract

For an axis-aligned Mask, a pixel-center rectangle with inclusive endpoints can reproduce the Mask exactly. Expanding it by half a pixel on each edge is not necessary for that raster round-trip.

For a rotated rectangle, a flat 0.5 margin on each fitted edge is not the exact pixel-cell expansion. For a unit axis vector u, the required margin per edge is:

0.5 * (abs(u.x) + abs(u.y))

At 30 degrees, this is approximately 0.683 per edge, not 0.5. Also, expanding the current fitted angle does not prove that the result remains the minimum-area rectangle of the full pixel cells. A correct implementation must define the target contract before it selects an algorithm.

Existing tests

The current tests deliberately require the pixel-center coordinates:

• An 11 by 21 Mask produces corners with continuous dimensions 10 by 20.
• An 11 by 11 Mask produces corners with continuous dimensions 10 by 10.

A behavior change must update these tests deliberately and add a rotated coverage regression test.

Maintainer decision

Choose one contract:

1. The oriented-rectangle Shape must enclose the full pixel cells of the source Mask.
2. The Shape must enclose only the foreground pixel centers.
3. The main requirement is raster coverage: converting the Shape back to a Mask must not omit source Mask pixels.

The third contract states the observed user requirement most directly. It does not require polygon and oriented-rectangle Shapes to use identical continuous coordinates.

This decision should be made with #2336, which covers the related Mask bounding-box extent convention.

Affected files

• labelme/_automation/_geometry.py
• labelme/_automation/_shape_builders.py
• tests/unit/_automation/_geometry_test.py
• tests/unit/_automation/_shape_builders_test.py
• tests/unit/utils/shape_test.py