There are no lossless transformations of natural-language text
A fundamental limit on language models: transformations always lose information. Here's what it means for reasoning and context windows.
Why it matters
Research reveals that no transformation of natural language can be truly lossless—a finding with direct implications for how LLMs process, compress, and reason over text. This reshapes thinking about tokenization, context windows, and the theoretical limits of language model capabilities.
The key facts
8 to knowNo lossless transformation of natural-language text exists
Implication for tokenization strategies and context compression
Affects reasoning chains and information preservation in long contexts
Published by Simon Willison (Datasette creator, noted AI researcher)
Core claim: all transformations of natural-language text lose information
Implications for tokenization, embeddings, and model training
Challenges the assumption that any encoding scheme is lossless
Published August 11, 2026 on Simon Willison's blog (established AI thought leader)
Go to the source
Simon Willisonsimonwillison.net