#2036·buck

Buck default java test runner may behave differently if --debug and --filter are specified or not

Author: romanoidCreated Sep 25, 2018Updated Nov 2, 2023
LabelsP3

https://github.com/facebook/buck/issues/2007 was one of the examples of such behavior observed (and fix fixed this behavior discrepancy in regards to some errors)

See this line: https://github.com/facebook/buck/blob/a6c7d7391a7a8e575c6372cac6690a240b5907ea/src/com/facebook/buck/testrunner/JUnitRunner.java#L221

This leads to using DelegateRunnerWithTimeout or not depending on if defaultTestTimeoutMillis is 0 or filter are specified.

Now --debug flag implies defaultTestTimeoutMillis.

This leads to both --filter and --debug changing test runner behavior, potentially causing discrepancies for test failures in normal "run everything" mode vs when you try to debug specific failure.

On the first glance i don't see why filters will not work with delegation using DelegateRunnerWithTimeout. In addition, not observing timeouts if only subset of tests are run may be potentially dangerous.

It is slightly more complicated with debug, we may want to keep timeout "infinite" when debugging, but retain other behaviors.

Potentially replacing defaultTestTimeoutMillis with Integer.MAX_VALUE rather than 0 may do the trick.

I'll be happy to investigate filtering and do the fixes if you think this approach is reasonable.