Prime Intellect Releases prime-rl 0.6.0 to Train Trillion-Parameter MoE Models on Agentic RL Workloads
131k sequence length. Sub-5-minute step times. Prime Intellect just open-sourced the playbook for training trillion-parameter models at scale.
Why it matters
Prime Intellect's prime-rl framework removes a critical bottleneck in agentic RL training for massive MoE models, enabling researchers and companies to train at scales previously reserved for well-funded labs. The open-source release democratizes trillion-parameter training infrastructure.
The key facts
6 to knowprime-rl 0.6.0 framework for asynchronous RL on trillion-parameter MoE models
GLM-5 trained on SWE tasks at 131k sequence length
Sub-5-minute step times with 256 rollouts
Deployed on 28 H200 nodes
Optimizations: FP8 inference, Wide Expert Parallelism, prefill/decode disaggregation, router replay, 3-D parallelism (FSDP, EP, CP)
Open-source framework (reduces barriers to entry for trillion-parameter training)
Go to the source
MarkTechPostmarktechpost.com
Publisher excerpt: Prime Intellect has released prime-rl 0.6.0, an open framework for asynchronous reinforcement learning on trillion-parameter Mixture-of-Experts models. It trained GLM-5 on SWE tasks at up to 131k sequence length, with sub-5-minute step times and 256 rollouts, on 28 H200 nodes. This breakdown covers…