moe

Mixture-of-Experts architectures, routing behavior, training stability, scaling properties, and serving tradeoffs for expert-based models.