Model WarsJuly 9, 2026via MarkTechPost
NVIDIA Releases Nemotron-Labs-3-Puzzle-75B-A9B: A Compressed Hybrid MoE LLM Delivering 2.03x Server Throughput at Matched User Throughput
Why it matters
NVIDIA is shipping production-grade model compression that solves the inference efficiency problem at scale. For operators, this means 8x better concurrency on existing hardware—a direct path to lower latency and higher utilization without new capex.
Key signals
- Nemotron-Labs-3-Puzzle-75B-A9B: compressed from 120.7B → 75.3B total parameters
- Active parameters: 12.8B → 9.3B (25% reduction in active compute)
- 2.03x total throughput on 8xB200 node at matched user throughput (100 tok/s per user)
- H100 concurrency: 1 request → 8 requests at 1M-token concurrency
- Compression method: iterative Puzzle (hardware-aware structural compression + knowledge distillation recovery)
- Released by NVIDIA Labs
- Published: July 9, 2026
The hook
2.03x throughput. NVIDIA's new Nemotron-Labs compresses a 120B model to 75B without losing performance—and servers can now handle 8x more concurrent requests on H100s.
NVIDIA has released Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super. Iterative Puzzle alternates hardware-aware structural compression with short knowledge distillation recovery phases. The model drops from 120.7B total / 12.8B active parameters to 75.3B / 9.3B. On a single 8xB200 node it delivers 2.03x Super's total throughput at 100 tok/s per user. On one H100, 1M-token concurrency rises from 1 request to 8.
The post NVIDIA Releases Nemotron-Labs-3-Puzzle-75B-A9B: A Compressed Hybrid MoE LLM Delivering 2.03x Server Throughput at Matched User Throughput appeared first on MarkTechPost.