FrontierSeptember 6, 2026via MarkTechPost
H Company Releases NeoMME: A Family of 260M and 800M Single-Tower Multimodal Encoders That Drop the Vision Tower and Causal Decoder
Why it matters
A shift in multimodal retrieval architecture: single-tower encoders without pretrained vision components are emerging as competitive alternatives to vision-tower-plus-text approaches, with real implications for retrieval and embedding model design.
Key signals
- NeoMME family: 260M and 800M parameter models
- Single Transformer processes multilingual text tokens and 32×32 raw image patches
- No pretrained vision tower, no causal decoder
- Masked discrete-diffusion pretraining objective
- Dual dense and late-interaction retrieval heads
- ViDoRe v3: 260M model reaches 0.523 nDCG@10
- 255× index compression
- 51.3 pages per second indexing throughput on one L40S GPU
- Authors acknowledge text-retrieval performance gaps
The hook
H Company's NeoMME drops the vision tower entirely—260M and 800M multimodal encoders that process text and raw image patches in a single Transformer, hitting 0.523 nDCG@10 on ViDoRe v3.
We look at NeoMME, a family of 260M and 800M bidirectional encoders from H Company. Unlike ColPali-style retrievers, it processes multilingual text tokens and raw 32×32 image patches in a single Transformer, with no pretrained vision tower and no causal decoder. We cover the masked discrete-diffusio…