ChipsAugust 27, 2026via AWS Machine Learning Blog
Reduce ASR inference costs by 75% with NVIDIA MPS on Amazon EC2
Why it matters
Compute economics shift: multi-process GPU sharing cuts infrastructure spend for speech AI workloads without sacrificing latency. Practitioners running ASR at scale should re-evaluate their deployment strategy.
Key signals
- 75% GPU infrastructure cost reduction
- 92.1 requests per second per GPU throughput
- Sub-second latency maintained
- NVIDIA CUDA Multi-Process Service (MPS) + Triton Inference Server
- Amazon EC2 GPU instances
- ASR (automatic speech recognition) use case
- 75% reduction in GPU infrastructure cost
- NVIDIA CUDA Multi-Process Service (MPS) + Triton Inference Server on EC2
- ASR model serving use case
- Shared GPU utilization technique
The hook
75% ASR inference cost cut. NVIDIA MPS on EC2 hits 92.1 req/s per GPU at sub-second latency.
Serving automatic speech recognition (ASR) models at scale is costly when each request uses only a fraction of a GPU. Learn how NVIDIA CUDA Multi-Process Service (MPS) with NVIDIA Triton Inference Server on Amazon EC2 GPU instances cuts GPU infrastructure by 75% while holding sub-second latency at 9…