FrontierAugust 11, 2026via Simon Willison
There are no lossless transformations of natural-language text
Why it matters
Research reveals that no transformation of natural language can be truly lossless—a finding with direct implications for how LLMs process, compress, and reason over text. This reshapes thinking about tokenization, context windows, and the theoretical limits of language model capabilities.
Key signals
- No lossless transformation of natural-language text exists
- Implication for tokenization strategies and context compression
- Affects reasoning chains and information preservation in long contexts
- Published by Simon Willison (Datasette creator, noted AI researcher)
- Core claim: all transformations of natural-language text lose information
- Implications for tokenization, embeddings, and model training
- Challenges the assumption that any encoding scheme is lossless
- Published August 11, 2026 on Simon Willison's blog (established AI thought leader)
The hook
A fundamental limit on language models: transformations always lose information. Here's what it means for reasoning and context windows.