VideoPrism: A foundational visual encoder for video understanding
Google just released VideoPrism: a single frozen model that outperforms specialized video AI models across 15+ benchmarks—from classification to scientific discovery.
Why it matters
Google's VideoPrism represents a major shift toward general-purpose video foundation models that can handle diverse tasks without task-specific fine-tuning, signaling where enterprise video AI is heading and potentially disrupting the market for specialized video models.
The key facts
8 to knowTrained on 618 million video clips (36M high-quality video-text pairs + 582M noisy/machine-generated text)
State-of-the-art performance across classification, localization, retrieval, captioning, and QA tasks
Outperforms previous foundation models: VideoCLIP, InternVideo, VideoCoCa, UMT, VATT, and CLIP
Single frozen model—no task-specific fine-tuning required for adaptation
Surpasses domain-specific models on scientific benchmarks (Fly vs. Fly, CalMS21, ChimpACT, KABR)
Two-stage training: contrastive learning + masked video modeling
Vision Transformer (ViT) with factorized spatial-temporal encoding
Compatible with language models (text encoder + language decoder for retrieval, captioning, QA)
Go to the source
Google Research Blogblog.research.google
Publisher excerpt: Posted by Long Zhao, Senior Research Scientist, and Ting Liu, Senior Staff Software Engineer, Google Research An astounding number of videos are available on the Web, covering a variety of content from everyday moments people share to historical moments to scientific observations, each of which…