GenerationConfig should validate that pad_token_id is not in eos_token_id list

Problem

When a model is fine-tuned with TRL's SFTTrainer, the saved generation_config.json can end up with pad_token_id set to a token that is also in the eos_token_id list. This causes model.generate() to emit EOS immediately, producing empty completions.

Reproduction

We fine-tuned nvidia/NVIDIA-Nemotron-Nano-9B-v2 with TRL SFTTrainer. The base model has:

"pad_token_id": 0,
"eos_token_id": 12

After SFT training, the saved generation_config.json had:

{
  "eos_token_id": [2, 11, 12],
  "pad_token_id": 12
}

Token 12 (<|im_end|>) is both pad AND eos. When model.generate() is called with pad_token_id=12, the model sees token 12 as EOS and stops immediately. HumanEval scored 0.0% (0/164) — every completion was empty.

The base model scored 76.2% (125/164) with the same eval script. The SFT model itself learned correctly (loss 0.94, token accuracy 78%) — the 0% is purely a config bug.

Root Cause

TRL's SFTTrainer sets tokenizer.pad_token = tokenizer.eos_token when no pad token exists. For Nemotron, eos_token is <|im_end|> (token 12), which is also in the eos_token_id list [2, 11, 12]. The trainer saves this collision into generation_config.json.

GenerationConfig.validate() checks various parameter combinations (do_sample vs top_p, etc.) but does not check for pad/eos collision.

Proposed Fix

Add a validation check in GenerationConfig.validate():

if self.pad_token_id is not None and self.eos_token_id is not None:
    eos_ids = self.eos_token_id if isinstance(self.eos_token_id, list) else [self.eos_token_id]
    if self.pad_token_id in eos_ids:
        warnings.warn(
            f"pad_token_id ({self.pad_token_id}) is in eos_token_id list ({eos_ids}). "
            f"This may cause model.generate() to stop immediately. "
            f"Consider setting pad_token_id to a token not in eos_token_id."
        )

This would warn users at config load time, before they waste hours running evals that produce 0%.

Environment

• transformers: 5.16.0.dev0 (main branch)
• trl: 1.10.0
• Model: nvidia/NVIDIA-Nemotron-Nano-9B-v2 (NemotronH hybrid Mamba-2 + Attention)
• GPU: NVIDIA GB10 (119GB VRAM)

Related

• TRL issue #1384: "SFTTrainer should not set tokenizer.pad_token_id = tokenizer.eos_token_id" (open since 2023)
• TRL PR #3200: Fix SFT masking EOS when equal to PAD (merged — fixes training, but not the saved config)
• Full writeup: https://github.com/davidnichols-ops/claude-yolo-vibes-v5/blob/master/EVALUATION.md