Eagle3 peaks at ~4.6 epochs then degrades; also 6% short of the released checkpoint (DSpark/DFlash reproduce)

Trained all three drafters from the unmodified configs in config/ on the documented data pipeline (Qwen3-4B target, full open-perfectblend split, regen params from scripts/data/README.md, 8 GPUs, 10 epochs). Two things about Eagle3.

1. It peaks around 4.6 epochs and degrades from there

Eval is 200 held-out prompts at temperature=0 with a fixed seed, so it's deterministic — re-running a checkpoint gives the same number.

epoch   train loss   accept_len
  0.8      2.69         3.28
  2.3      1.97         3.34
  4.6      1.73         3.34   <- peak
  7.6      1.59         3.24
 10.0      1.50         3.18   <- final, -4.9%

Loss keeps going down the whole time. Four consecutive evals go down after the peak and the per-position acceptance rates go down with them.

DSpark and DFlash off the same cache, same schedule: every per-1000-step increment stays positive through the final step. Only Eagle3 turns over.

Also shows up on wall clock. Measured tokens/s against a plain autoregressive baseline: our step-12000 checkpoint is ~1.5% off the released one's speedup, our final checkpoint is ~8% off.

So the README line

Each drafter is trained for 10 epochs to ensure full convergence.

holds for DSpark and DFlash here but not for Eagle3.

2. Even at its peak it's 6.2% short, while the other two land within 1%

            ours (best ckpt)   released   ratio
DSpark            4.85           4.89     99.2%
DFlash            4.44           4.45     99.7%
Eagle3            3.34           3.56     93.8%

Best-to-released, not final-to-released, so this isn't double-counting the overfitting above.

Checked and matching: architecture config (all keys, incl. ttt_length, step_loss_decay, draft_num_hidden_layers, target_layer_ids), tensor set/shapes/dtypes, bit-identical frozen embed_tokens and lm_head, regen temperature/top-p/top-k/min-p/non-thinking, min_loss_tokens, config seed, epoch count.

Two things ruled out by measuring rather than arguing:

• Not data-limited. Retrained Eagle3 on a ~170x smaller subset and compared peaks: full data buys Eagle3 at most +1.10 accept_len, vs +2.30 for DSpark. It benefits less from data, not more.
• Not missing regularisation. The released checkpoint's trained-weight norm is ~5% larger than our final's, and weight decay would make it smaller. Shipped config is weight_decay=0.0.

Could still be something on our end, but DSpark and DFlash come out within 1% on the same pipeline, so it looks specific to Eagle3.

Questions

1. Are the released Eagle3 checkpoints the last step of a 10-epoch run, or picked on a validation signal? README says "the direct output of the corresponding training configuration", which reads like the former.
2. Do you see the turnover in 1 on your side? If so, the Table 1 Eagle3 row is measured past its peak.
3. Anything in the Eagle3 training path not in config/eagle3/eagle3_qwen3_4b.py? A train_config.py next to the released checkpoint, or just the hyperparams used for it, would let us close 2 ourselves.
4. Worth documenting a shorter schedule for Eagle3, or adding val-based checkpoint selection?

Let us know if you need more information. Looking forward to your reply.