[Bug] GMPOTrainer with use_liger_kernel=True trains on the GRPO objective

Reproduction

With use_liger_kernel=True, GMPOTrainer silently trains on the GRPO objective. The reproduction below compares a full optimizer update with the GMPO and GRPO objectives.

Related: Issue #6808 already reports the dispatch problem; PR #7163 proposes rejecting the configuration at initialization, and PR #6817 was an earlier attempt.

GMPOTrainer overrides _compute_loss but inherits compute_loss from GRPOTrainer, which dispatches before _compute_loss is reached:

def compute_loss(self, model, inputs, return_outputs=False, num_items_in_batch=None):
    if return_outputs:
        raise ValueError("The GRPOTrainer does not support returning outputs")
    if self.use_liger_kernel:
        unwrapped_model = self.accelerator.unwrap_model(model)
        return self._forward_redirection(model, unwrapped_model, self.compute_liger_loss, unwrapped_model, inputs)
    return self._compute_loss(model, inputs)

So with use_liger_kernel=True, GMPO trains on the fused GRPO loss.

To reproduce: GMPOTrainer.train with a two-layer LLaMA (hidden size 64, vocabulary 128), a fixed initial checkpoint and a fixed rollout generated by a separate behavior policy, one SGD update (lr 0.05, momentum 0.9), once with use_liger_kernel=False and once with True.

The Liger-enabled GMPO run is identical to a plain GRPO run in loss, full gradient and final state. Compared with the GMPO objective, its gradient differs by 0.1996558210 relative L2. Overriding compute_loss in GMPOTrainer to call its own _compute_loss brings that to 1.93049e-7; GRPO runs with and without Liger serve as controls for ordinary kernel differences. Reproduced on a second host.

Expected: an execution flag should either keep the GMPO objective or reject the configuration, not silently switch to GRPO. Rejecting the flag as in #7163 is a reasonable API choice. Routing to the ordinary GMPO loss, as tested here, keeps the objective but not the fused kernel's memory savings. This is about TRL's integration, not the GMPO paper code, and the one-update test says nothing about long-run reward.

System Info

TRL: 32fc9d71333c0390bed5b74051526aab0361441e
PyTorch: 2.8.0+cu128
Transformers: 5.17.0
liger-kernel: 0.8.3
Triton: 3.4.0
GPU: NVIDIA RTX A6000

Source: gmpo_trainer.py, grpo_trainer.py.

Checklist

• I have checked that my issue isn't already filed (the related issue #6808 reports this problem; this report adds a full-update reproduction).
• I have included my system information
• Any code provided is minimal, complete, and reproducible (a code excerpt and reproduction protocol are included; a standalone executable reproducer is not attached).
• Any code provided is properly formatted in code blocks, (no screenshot, more on code blocks)
• Any traceback provided is complete (no exception; this is a silent objective switch)