ChipsThe story, in brief
Block-sparse GPU kernels
Orders of magnitude faster. OpenAI just open-sourced GPU kernels that could reshape how models run inference.
The KeyNews take
Why it matters
Block-sparse GPU kernel optimization is foundational infrastructure that reduces compute cost and latency for neural networks at scale. This directly impacts the economics of model deployment and inference efficiency.
The key facts
5 to knowBlock-sparse GPU kernels released by OpenAI
Performance improvement: orders of magnitude faster than cuBLAS/cuSPARSE
Applied to text sentiment analysis and generative modeling (text and images)
Addresses underexplored neural network architecture class
Published December 6, 2017
Go to the source
OpenAI Blogopenai.com
Publisher excerpt: We’re releasing highly-optimized GPU kernels for an underexplored class of neural network architectures: networks with block-sparse weights. Depending on the chosen sparsity, these kernels can run orders of magnitude faster than cuBLAS or cuSPARSE. We’ve used them to attain state-of-the-art results…