Proposal: CUDA-accelerated implementation of TrackerCSRT

Description

cv::TrackerCSRT currently runs on the CPU even when OpenCV is built with CUDA support. I would like to propose and work on a CUDA-accelerated CSRT implementation.

CSRT provides good localization accuracy, but its CPU runtime can be limiting in real-time video applications, particularly when tracking multiple objects or running alongside a detector. A CUDA implementation could accelerate its computationally expensive stages while keeping frames and tracker state in GPU memory.

Proposed API

My initial suggestion is a separate CUDA tracker rather than changing the behavior of the existing CPU implementation:

cv::Ptr<cv::cuda::TrackerCSRT> tracker =
    cv::cuda::TrackerCSRT::create(params);

tracker->init(gpuFrame, boundingBox);
bool found = tracker->update(gpuFrame, boundingBox);

The tracker would accept cv::cuda::GpuMat frames so that applications already performing decoding or inference on the GPU would not need to download every frame to CPU memory.

I would like maintainers’ feedback on:

1. Whether a CUDA CSRT implementation would be accepted.
2. Whether it should be placed in an existing CUDA module, the tracking module, or a new CUDA tracking module.
3. Whether a separate cv::cuda::TrackerCSRT API is preferred.
4. Whether the initial implementation should target the current OpenCV 4.x API or OpenCV 5.x.

Proposed implementation scope

The implementation would attempt to move the main CSRT stages to the GPU:

• Image preprocessing and ROI extraction
• HOG, grayscale and Color Names feature extraction
• Windowing and feature normalization
• FFT-based correlation operations
• Spatial and channel reliability calculations
• ADMM filter optimization
• Scale estimation

Existing OpenCV CUDA primitives would be reused where appropriate, with custom CUDA kernels added only for CSRT-specific operations.

Frames, templates, filters, features and temporary buffers would remain in GPU memory between calls. Avoiding repeated CPU–GPU transfers would be a primary design requirement.

Compatibility

The existing CPU cv::TrackerCSRT implementation and API would remain unchanged. The CUDA implementation would only be built when CUDA support is enabled.

The goal would be to preserve the behavior and parameters of the CPU implementation as closely as practical. Small numerical differences are expected because of floating-point ordering and CUDA implementations of operations such as FFT.

Testing and evaluation

I plan to provide:

• Correctness tests comparing CPU and CUDA tracking results within suitable tolerances
• Tests for initialization, update, target loss and different image formats
• Tracking-quality evaluation on an established single-object tracking dataset
• Performance benchmarks across different frame, target and batch/object counts
• Separate timing for computation and CPU–GPU transfers
• Documentation and a minimal C++ example

Benchmarks would also include small targets and single-object cases where CUDA launch overhead may outweigh the speedup.

Questions before implementation

Before beginning the full implementation, I would appreciate feedback on the proposed API, repository/module placement and expected testing requirements.

I am willing to work on the implementation and submit it incrementally if the proposal is considered suitable.