The Agent RaceJuly 9, 2026via MarkTechPost
Meet Nemotron Labs 3 Puzzle 75B A9B: A Compressed Hybrid MoE LLM Delivering 2.03x Server Throughput
Why it matters
NVIDIA ships a production-ready compressed MoE variant that delivers massive throughput gains on B200/H100 hardware. For infrastructure teams, this is a direct cost-per-inference win that could reshape deployment economics.
Key signals
- Nemotron-Labs-3-Puzzle-75B-A9B: compressed from 120.7B total / 12.8B active to 75.3B / 9.3B parameters
- 2.03x throughput improvement over Nemotron-3-Super on 8xB200 node
- 100 tokens/sec per user on single 8xB200 node
- 8x concurrency increase on single H100 (1 → 8 requests at 1M-token context)
- Compression method: iterative Puzzle alternating hardware-aware structural compression with knowledge distillation
- Published: July 9, 2026
The hook
2.03x throughput. NVIDIA's new Nemotron compress 75B model just changed the inference efficiency playbook.
NVIDIA has released Nemotron-Labs-3-Puzzle-75B-A9B, a compressed variant of Nemotron-3-Super. Iterative Puzzle alternates hardware-aware structural compression with short knowledge distillation recovery phases. The model drops from 120.7B total / 12.8B active parameters to 75.3B / 9.3B. On a single 8xB200 node it delivers 2.03x Super's total throughput at 100 tok/s per user. On one H100, 1M-token concurrency rises from 1 request to 8.