[Bug]: ignore_eos_token keeps EOS in DPO length normalization

软件环境

• paddlepaddle: 3.3.1 (CPU)
• paddleformers: current develop
• Python: 3.12.10

重复问题

• I have searched the existing issues

错误描述

paddleformers.nn.criterion.dpo_loss.dpo_logps excludes one EOS token from each chosen/rejected log-prob sum when ignore_eos_token=True, but the average_log_prob and normalize_logps branches still divide/scale by response lengths that include that excluded token.

This makes the normalized policy and reference log-probs internally inconsistent. It directly changes the objectives that enable average_log_prob (simpo, ipo, and or), and also affects explicit normalize_logps=True. Because chosen and rejected responses can have different lengths, the error does not cancel in their log-ratio.

稳定复现步骤 & 代码

For a packed pair with boundaries [0, 0, 3, 7], suppose the per-token negative log-probs are:

chosen:  [-1, -2, EOS]
rejected: [-3, -4, -5, EOS]

With ignore_eos_token=True, the aggregated log-probs are -3 and -12. Their effective token counts are therefore 2 and 3, so averaging should return -1.5 and -4.0.

The current implementation divides by the original lengths 3 and 4 and instead returns -1.0 and -3.0. The same stale lengths are used by normalize_logps.

I have a focused fix and regression coverage ready that reuse EOS-adjusted chosen/rejected lengths in both normalization branches.