#2028·benchmark

Run all tests without arguments

Author: senyaiCreated Aug 14, 2025Updated Aug 15, 2025

Hi! I find it confusing that the tests require arguments. I want to change it.

Here are my reasons:

Not all tests check for required arguments https://github.com/google/benchmark/blob/7697796fe5a70edbc3394d0593ef82afeebb7ddf/test/spec_arg_test.cc#L63-L70 and so it's easy to run a test incorrectly.
There are GitHub issues mentioning that a test fails when the actual problem was that the correct arguments were not passed.
Arguments need to be duplicated between test/CMakeLists.txt and test/BUILD and they are actually not in sync now (perf_counters_test is run with different arguments).

My solution:

Create test/default_arguments.h:

cpp

// ...

void AddTestArguments(int &argc, char **&argv, std::initializer_list<char const*> args={}) {
  if (argc > 1) {
    std::cout << "Warning: User is not expected to pass any command line arguments\n";
  }
  static std::vector<char const*> new_argv;
  new_argv.insert(new_argv.end(), argv, argv + argc);
  new_argv.insert(new_argv.end(), args.begin(), args.end());
  new_argv.push_back("--benchmark_min_time=0.01s");
  argv = const_cast<char**>(new_argv.data());
  argc = static_cast<int>(new_argv.size());
}

Call it after each main function like this

cpp

int main(int argc, char* argv[]) {
  AddTestArguments(argc, argv, {"--benchmark_counters_tabular=true"});
  benchmark::MaybeReenterWithoutASLR(argc, argv);
  RunOutputTests(argc, argv);
}

Cleanup the code

Sounds good?

Source: google/benchmark

View original on GitHub View discussion on GitHub