MoE 前馈每层每向前处理约 3×num_experts 设备→主机同步

The cost is paid per expert, per layer, per forward, and it is spent on routing bookkeeping rather than on any expert math. [README.md:566] attributes the MoE slowdown to kernel launch and scheduling overhead, and suggests that optimizing it requires a fused-MoE operator library — Triton custom kernels, DeepSpeed-MoE, Megatron-LM — which the project declines in order to stay native-PyTorch, accepting ~50% slower than dense at 4 experts / top-1. Host syncs are a separate cost from launch overhead, and this one is removable without leaving native PyTorch. I am not claiming it accounts for the whole 50%; I am pointing out that a measurable part of it is bookkeeping that does not need to be there.