optflow: calcOpticalFlowSparseToDense segfaults when k exceeds the number of sparse matches

System information

• OpenCV: 4.10.0 with opencv_contrib, built from source, static
• Platform: Ubuntu 22.04, x86_64
• Compiler: GCC 11.4, TBB parallel backend
• Also seen on Windows 10 x64 with an MSVC build of the same version

The same code is on the 4.x and 5.x branches of opencv_contrib today, so this is not specific to 4.10.

Detailed description

cv::optflow::calcOpticalFlowSparseToDense reads a null pointer and the process dies with SIGSEGV when k is large relative to the number of sparse matches the image can supply. The call below uses the documented default parameters on a legal input.

The function samples the image on a grid of grid_step pixels. It tracks those points with calcOpticalFlowPyrLK and keeps the ones that survive. The survivors go to EdgeAwareInterpolator, configured with setK(k).

A 64x48 image at the default grid_step of 8 yields 48 samples, and the tracker drops some of them. The default k is 128. So the interpolator is asked to fit a model over the 128 nearest matches when fewer than 48 exist.

In modules/optflow/src/sparsetodenseflow.cpp the only check on k is:

CV_Assert( grid_step>1 && k>3 && sigma>0.0001f && fgs_lambda>1.0f && fgs_sigma>0.01f );

Nothing compares k with points.size() or points_filtered.size() before gd->setK(k).

The crash is not deterministic. The routine parallelizes internally, so the same input dies on some runs and completes on others. That is why a small image with the defaults can look fine in a quick test.

Steps to reproduce

#include <opencv2/core.hpp>
#include <opencv2/optflow.hpp>

#include <cstdio>
#include <cstdlib>

int main(int argc, char** argv)
{
    const int seed = argc > 1 ? std::atoi(argv[1]) : 1;
    cv::setRNGSeed(seed);

    cv::Mat from(48, 64, CV_8UC1);
    cv::Mat to(48, 64, CV_8UC1);
    cv::Mat flow;
    cv::randu(from, cv::Scalar(0), cv::Scalar(255));
    cv::randu(to,   cv::Scalar(0), cv::Scalar(255));

    // grid_step 8 and k 128 are the documented defaults
    cv::optflow::calcOpticalFlowSparseToDense(from, to, flow, 8, 128);

    std::printf("ok\n");
    return 0;
}

Run one seed per process, because a crash ends the process:

for s in $(seq 1 300); do ./repro $s || echo "crashed on seed $s"; done

Measured on the build above, 300 runs per row:

image     k     grid_step   crashes
64x48     128   8           5 / 300
64x48     48    8           0 / 300
64x48     44    8           0 / 300
64x48     32    8           0 / 300
320x240   128   8           0 / 300

On Windows with an MSVC build the rate is higher and the threshold is lower. With one fixed input, k=128 crashed 23 times in 100. k=46 crashed 6 times in 60. k=44 and below stayed clean in 100 runs each.

So the crash needs two things together: a small image, and a k close to or above the number of matches. Either one alone is fine. Lowering k or enlarging the image removes it.

Suggested fix

Clamp k to what the image can supply before configuring the interpolator, or refuse with a message:

if ((int)points_filtered.size() <= k)
    k = std::max(4, (int)points_filtered.size() - 1);
gd->setK(k);

A caller who leaves the defaults alone on a small image should get a result or an error, not a dead process.