MELON: Reconstructing 3D objects from images with unknown poses
Google just solved a 'chicken and egg' problem that's blocked 3D reconstruction for years. MELON needs just 4-6 images.
Why it matters
Google Research released MELON, a computer vision breakthrough that reconstructs 3D objects from unposed images without pre-training or initial pose estimates. This addresses a fundamental bottleneck in e-commerce, autonomous vehicles, and robotics—where camera positions are unknown.
The key facts
15 to knowMELON requires only 4-6 images to reconstruct 3D objects
Achieves 27.5 dB peak signal-to-noise ratio after 50k training steps
Works on extremely noisy images (σ=1.0 where objects are visually indistinguishable)
Cameras converge to globally optimal poses within first 1,000 training steps
No pre-training on labeled data required
Spotlighted at 3DV 2024 conference
Integrates into existing NeRF methods as modular approach
Uses lightweight CNN encoder requiring no pre-training initialization
Achieves competitive PSNR of 27.5 dB after 50k training steps
Works with extremely noisy images (tested up to σ=1.0 Gaussian noise)
Eliminates need for pre-training on labeled datasets
Uses lightweight CNN encoder with no pre-training required
Modulo loss considers N=2 views (or N=4 for symmetric objects)
Solves pose-inference problem that previously required initial estimates close to global minima
Applications: e-commerce 3D models, autonomous vehicle navigation, robotics
Go to the source
Google Research Blogblog.research.google
Publisher excerpt: Posted by Mark Matthews, Senior Software Engineer, and Dmitry Lagun, Research Scientist, Google Research A person's prior experience and understanding of the world generally enables them to easily infer what an object looks like in whole, even if only looking at a few 2D pictures of it. Yet the…