CLI key-value parser preserves whitespace in argument names

Author: yilin-succeedCreated Sep 12, 2026Updated Sep 12, 2026

Description

The new MergeDictAction accepts comma-separated key=value arguments, but key_val_to_dict() does not trim whitespace around each key. As a result, the natural CLI spelling with a space after a comma silently creates an argument whose name begins with a space.

Reproduction

python
from lm_eval._cli.utils import key_val_to_dict

print(key_val_to_dict("do_sample=false, temperature=1.0"))

Current output:

python
{"do_sample": False, " temperature": 1.0}

Expected output:

python
{"do_sample": False, "temperature": 1.0}

This is silent rather than a parse error: downstream constructors receive the wrong keyword name, so an otherwise valid --model_args or --gen_kwargs value may be rejected or ignored depending on the consumer.

Proposed fix

Strip surrounding whitespace from keys before inserting them into the result, reject keys that become empty after trimming, and add focused tests for comma-separated arguments with spaces and empty keys. Value whitespace is already normalized by handle_cli_value_string().

I checked open and closed issues/PRs for key_val_to_dict, model/gen-argument whitespace, leading-space keys, and related parser changes, and did not find an existing report or implementation. I would like to submit the minimal fix and regression tests.

Source: EleutherAI/lm-evaluation-harness