Expert utilization is a necessary MoE metric, but it can conceal a router that is not learning from the task. Cerebras demonstrates the failure on a 124-million-parameter GPT-2-style model with four experts trained on FineWeb.
The first run looked healthy. Learned routing beat the dense baseline by roughly 1%, and utilization converged close to 25% for each expert. The expected gain was about 2%, so the team inspected gradient norms instead of treating balanced load as proof of specialization.
The router received effectively no cross-entropy gradient. With top_k=1, the selected gate is normalized by the sum of selected gates. Because that sum is the same single value, the weight becomes gate / gate = 1. The model can learn to distribute traffic through the auxiliary balancing loss while losing the task signal that should teach which expert fits each token.
Using top_k=2 preserves relative weights and restores gradient flow, but activates two experts. Cerebras instead adds a tunable null, or phantom, expert for the top-one case. It is never executed and adds no model parameters or expert computation, yet it gives the real expert score a contrastive alternative during normalization.
With the null expert, the cross-entropy gradient returned and the four-expert model reached the expected 2% improvement over the dense baseline at 2.4 billion training tokens. The result came from repairing one router operation rather than increasing model size or running a broad hyperparameter sweep.
For New Runtime, the broader lesson is an evaluation pattern: monitor the causal training signal, not only a tidy downstream distribution. When an aggregate metric looks ideal but quality stalls, trace the gradient or decision path through the component that is supposed to learn.