Baike.dev
Log in
Back to tool/Back to issues
#3266·mmpose

Low inference FPS on custom trained model

Author: EladProjectCreated Jun 13, 2026Updated Jun 13, 2026

Prerequisite

Environment

I'm trying to build a hand keypoint detector based on depth images. I'm using this base: mmpose::hand_2d_keypoint/rtmpose/coco_wholebody_hand/rtmpose-m_8xb32-210e_coco-wholebody-hand-256x256.py

My images are 640x480 depth images which I preprocess to 640x480 color images. The training goes well and inference is with good accuracy. I convert the resulting pth file to onnx and use rtmlib for inference with on cpu and on gpu. My hardware is i5-13400f and RTX4070

This is the code I use to time the inference:

       hand_kwargs = dict(
            det_input_size=(320,320),
            pose_input_size=(256,256),
            mode="lightweight",
            device="cuda", # sometimes it cpu
            backend="onnxruntime,
            pose=<my custom model>
        )
       
        self._detector = Hand(**hand_kwargs)
        det_start_time = time.perf_counter()
        bboxes = self._detector.det_model(color_image)
        det_elapsed_ms = (time.perf_counter() - det_start_time) * 1000.0
        pose_start_time = time.perf_counter()
        self.keypoints, self.scores = self._detector.pose_model(color_image, bboxes=bboxes)
        #self.keypoints, self.scores = self._detector(color_image)
        pose_elapsed_ms = (time.perf_counter() - pose_start_time) * 1000.0
        print(f"RTMPose detection took {det_elapsed_ms:.2f} ms, pose estimation took {pose_elapsed_ms:.2f} ms. Num boxes: {len(bboxes)}, Num hands: {len(self.keypoints)}")

A typical result looks like this: RTMPose detection took 16.66 ms, pose estimation took 45.18 ms. Num boxes: 2, Num hands: 2

and it seems to occur both on cpu and gpu (gpu being a little bit faster). This is much higher than what the documentation say should be the performance.

What am I doing wrong?

Reproduces the problem - code sample

You can see the code I'm using above. It will be hard to reproduce without my model.

Reproduces the problem - command or script

Was hoping you have any ideas that can help me

Reproduces the problem - error message

There is no error. I do have these messages:

hand onnxruntime cuda
2026-06-13 13:39:18.1880169 [W:onnxruntime:, session_state.cc:1367 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Some nodes were not assigned to the preferred execution providers which may or may not have an negative impact on performance. e.g. ORT explicitly assigns shape related ops to CPU to improve perf.
2026-06-13 13:39:18.1945236 [W:onnxruntime:, session_state.cc:1369 onnxruntime::VerifyEachNodeIsAssignedToAnEp] Rerunning with verbose output on a non-minimal build will show node assignments.
load C:\Users\eladp\.cache\rtmlib\hub\checkpoints\rtmdet_nano_8xb32-300e_hand-267f9c8f.onnx with onnxruntime backend
load G:\My Drive\Projects\hand_depth_model\outputs\20260610-191649_rtmpose_hand\best_coco_AP_epoch_180.onnx with onnxruntime backend
ModelKeypointExtractor: Loaded RTMPose backend
RTMPose detection took 479.45 ms, pose estimation took 128.75 ms. Num boxes: 1, Num hands: 1

Additional information

No response

Source: open-mmlab/mmpose

View original on GitHubView discussion on GitHub