Llama 4 MoE routing: has profile-based expert selection been explored as an alternative to learned routers?

Llama 4 Maverick uses 128 experts with top-1 routing — a massive MoE deployment. The learned router W_r must be enormous at that scale.

One architectural question: the learned router co-trains with experts, so the expert pool is locked post-training. If a year from now someone builds a better coding expert, can it be swapped into Maverick without retraining the entire router?

We've been exploring profile-based routing as an alternative: each expert carries a calibrated benchmark vector. Router uses cosine similarity instead of learned weights. Zero router parameters (cosine sim is pure math). Experts become swappable — update one profile array, done. The bias mechanism (like DeepSeek-V3's auxiliary-loss-free balancing) remains fully compatible.

At 128 experts × 16 MoE layers × d_model, the parameter savings alone are significant.

We have a working proof-of-concept: https://github.com/washingtoneimae-dot/profile-moe

Would love the Llama team's perspective — was profile-based routing ever considered, or is the learned router considered sufficient for post-training expert management?