For higher-interactivity scenarios, execution time for MoE models is bound by expert weight load time. By splitting, or sharding, the experts across multiple GPUs across NVL72 nodes, this bottleneck is reduced, improving end-to-end performance.
大多数人认为MoE模型的主要瓶颈在于计算能力，但作者指出专家权重加载时间是真正的瓶颈，并提出通过跨GPU分片专家权重来解决问题，这挑战了AI模型优化的传统认知，暗示了I/O可能比计算更重要。