SiliconANGLE reports that Ai2 says the framework supports growing an expert pool from eight to 128 while selecting four experts per token, and can scale to models exceeding one trillion parameters.
In benchmarks reported by Ai2, a 47-billion-parameter model processed 52,000 tokens per second on Nvidia B300 GPUs, compared with about 19,400 tokens per second for Nvidia’s Megatron-core. Ai2 attributes the efficiency to expert parallelism, splitting layers across GPU groups, distributing optimizer state, and support for MXFP8. These are reported benchmark figures, not a general guarantee across hardware or workloads. The project and related systems are available to developers and the open-source community on GitHub, according to SiliconANGLE.