Prime Intellect Launches Prime Inference: Serverless and Reserved Serving for Frontier Open Models
Prime Intellect's inference platform hits 101 tokens/sec per user on GLM-5.3. Here's what changes for open-model deployment.
Why it matters
Prime Intellect launches Prime Inference, an OpenAI-compatible serving platform optimized for NVIDIA Blackwell, using vLLM and KV compression to achieve measurable throughput on frontier open models. Practitioners deploying open-weight models at scale now have a vendor option beyond proprietary inference APIs.
The key facts
13 to knowOpenAI-compatible API for serving frontier open models
NVIDIA Blackwell hardware target
GLM-5.3 deployment: 66 sessions per prefill group, 101 tokens/sec per user
Uses Dynamo, vLLM, NVFP4 KV compression
Serverless and reserved serving options
No pricing disclosed
No GA/preview status clearly stated; appears to be announcement/launch
Prime Intellect launches Prime Inference—OpenAI-compatible serving platform
Targets frontier open models on NVIDIA Blackwell hardware
GLM-5.3 deployment case: 66 sessions per prefill group, 101 tokens/sec per user
Stack: Dynamo, vLLM, NVFP4 KV compression
Offers both serverless and reserved serving modes
Benchmarked throughput assumes prefill-grouped workloads; real-world variability not disclosed
Go to the source
MarkTechPostmarktechpost.com
Publisher excerpt: Prime Intellect has launched Prime Inference, an OpenAI-compatible platform for serving frontier open models on NVIDIA Blackwell. Its GLM-5.3 deployment uses Dynamo, vLLM and NVFP4 KV compression to serve 66 sessions per prefill group at 101 tok/s per user.