newman exits 0 despite printing an argument validation error (invalid -n, --timeout, --color, or unknown subcommand)
Description
bin/newman.js prints error: <message> and calls program.help() in two places when it hits an argument error: on an unrecognized subcommand, and on any error thrown while parsing options.
https://github.com/postmanlabs/newman/blob/develop/bin/newman.js#L92-L96 https://github.com/postmanlabs/newman/blob/develop/bin/newman.js#L128-L137
Commander's Command.prototype.help(contextOptions) only exits with a non-zero code when called as help({ error: true }):
help(contextOptions) {
this.outputHelp(contextOptions);
let exitCode = process.exitCode || 0;
if (exitCode === 0 && contextOptions && typeof contextOptions !== 'function' && contextOptions.error) {
exitCode = 1;
}
this._exit(exitCode, 'commander.help', '(outputHelp)');
}Both call sites call it bare (program.help()), so contextOptions is undefined and it always exits process.exitCode || 0 — i.e. 0 — even though an error was just printed to console.error.
This affects any option that validates its value via a custom coercion function that throws on bad input: --iteration-count/-n, --timeout, --timeout-request, --timeout-script (all use util.cast.integer), and --color (uses util.cast.colorOptions). It also affects typing an unrecognized subcommand.
Impact
A CI pipeline that runs newman run collection.json -n -5 (or any other invalid value for the options above, or a typo'd subcommand) gets error: The value must be a positive integer. printed to stderr, the run never actually executes, and $? is 0. Any pipeline gating on the exit code will treat this as a successful run.
Repro
$ node ./bin/newman.js run collection.json -n -5; echo "exit=$?"
error: The value must be a positive integer.
...
exit=0
$ node ./bin/newman.js run collection.json --color purple; echo "exit=$?"
error: invalid value `purple` for --color. Expected: auto|on|off
...
exit=0
$ node ./bin/newman.js frobnicate; echo "exit=$?"
error: invalid command `frobnicate`
...
exit=0For comparison, commander's own built-in validation errors (e.g. missing required option values, unknown options) already exit 1 correctly, since those go through commander's internal _exit machinery rather than this catch/help path.
Fix
Pass { error: true } at both call sites:
program.help({ error: true });PR with fix + CLI-level regression tests (which fail against current develop and pass with the fix): #3374
Source: postmanlabs/newman