ToolsSeptember 3, 2026via MarkTechPost
Perplexity Open Sources Lily: A Rust + Metal Inference Engine for Qwen3.6-35B-A3B on Apple Silicon
Why it matters
A practitioner running local inference on Apple Silicon can now adopt a faster, open-weight runtime; the release reflects the growing competition for on-device model serving and optimization for specific hardware.
Key signals
- Perplexity open-sourced Lily inference engine
- Built in Rust with custom Metal kernels
- Optimized for Qwen3.6-35B-A3B on Apple Silicon
- 1.23x prefill throughput vs MLX-LM baseline
- 1.35x decode throughput vs MLX-LM baseline
- Tested on 40-core, 128GB M5 Max
- Powers Hybrid Compute in Perplexity Computer product
- Perplexity open sourced Lily inference engine
- Built in Rust with custom Metal kernels for Apple Silicon
- Optimized for Qwen3.6-35B-A3B model
- 1.23x MLX-LM prefill throughput improvement
- 1.35x MLX-LM decode throughput improvement
- Benchmarked on 40-core, 128 GB M5 Max
- Powers Hybrid Compute in Perplexity Computer
The hook
Perplexity open-sourced Lily, a Rust+Metal inference engine that runs 35B models on Apple Silicon 35% faster than the current standard.
Perplexity has open sourced Lily, the local inference engine behind Hybrid Compute in Perplexity Computer. Built in Rust with custom Metal kernels for one model on one chip family, it averages 1.23x MLX-LM's prefill throughput and 1.35x its decode throughput on a 40-core, 128 GB M5 Max.