How fast is 10 tokens per second really?
10 tokens per second sounds fast. Here's what it actually means for your AI infrastructure decisions.
Why it matters
A technical deep-dive on token throughput metrics that challenges how companies should evaluate AI model performance and cost efficiency—critical context for infrastructure planning decisions.
The key facts
8 to knowFocus on tokens-per-second as a performance metric
Published by Simon Willison (Datasette creator, influential AI technologist)
Addresses infrastructure evaluation methodology
May 2026 publication date (recent analysis)
Focus on demystifying tokens-per-second as a performance metric
Analysis of perceived vs. actual speed in practical deployments
Relevant to infrastructure decision-making and model selection
Published by Simon Willison (trusted AI technical analyst)
Go to the source
Simon Willisonsimonwillison.net