#5490·detectron2

Issue with Onnx'd detectron2 mask-rcnn bounding boxes

Author: cwheeler-rrCreated Sep 17, 2025Updated Sep 17, 2025

I have build a mask-rcnn in detectron2 with COCO-InstanceSegmentation/mask_rcnn_X_101_32x8d_FPN_3x.yaml as a back bone but have a bounding box miss match compared to the original image

When using the detectron2 model through the inbuilt methods like so:

python
cfg = get_cfg()
# Use a pre-trained Mask R-CNN model from the model zoo
cfg.merge_from_file("../../outputs/models/config_custom_2025-09-13 05:57:14.510549.yaml")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5  # confidence threshold
cfg.MODEL.WEIGHTS = "../../outputs/models/best_model.pth"
cfg.MODEL.DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

predictor = DefaultPredictor(cfg)

img = get_masks_no_norm("../temp_data/full_inspection/img1.png")
outputs = predictor(img)
instances = outputs["instances"].to("cpu")
boxes = instances.pred_boxes.tensor.numpy()
boxes_dict[name] = boxes

This will give bonding boxes for an image that are accurate and line up well with the prediction visulisations from training and testing.

However when using an onnx model created using the commands from detectron2s documentation:

python3 ./detectron2/tools/deploy/export_model.py --config "../../outputs/models/config_custom_2025-09-12 17:13:50.286475.yaml" --output ./ --export-method tracing --sample-image ../datasets/dataset1/test.png --format onnx MODEL.WEIGHTS ../../outputs/models/best_model.pth MODEL.DEVICE cuda

I get an onnx output but it performs differently to the original model.

I am then running this model with:

python
session = ort.InferenceSession(model_path)

ai_image = image_pre_process()

# Run inference
inp_dict = {"x.1": ai_image}
outputs = session.run(None, inp_dict)

Ive given an example of how these detection are different below the object we are detecting are small so even being 2-3 pixels out could lead to missing the thing we want to find.

Detecron2 boxes:

[277.0877,  222.06036, 291.78217, 248.23471
 [253.8955, 175.23312. 271.17606. 197.9112 ]

Onnx boxes:

[274.79288, 219.97232, 289.00266, 246.60608]
[250.75594, 174.43523, 268.79462, 197.45988]

Is there any reason you can think of why there would be a difference in the outputs from the Onnx model and the original detectron2 model?

Thank you for any help

Source: facebookresearch/detectron2