Qwen2.5 1.5B with GRPO meets Entropy Collapse

Author: BUGLI27Created Jun 12, 2026Updated Jun 27, 2026

I tried clipping higher and disabling the KL loss, but neither helped. Does anyone know what might be causing this?

Source: Jiayi-Pan/TinyZero