Perplexity Open Sources Lily: A Rust + Metal Inference Engine for Qwen3.6-35B-A3B on Apple Silicon
Perplexity open-sourced Lily, a Rust+Metal inference engine that runs 35B models on Apple Silicon 35% faster than the current standard.
Why it matters
A practitioner running local inference on Apple Silicon can now adopt a faster, open-weight runtime; the release reflects the growing competition for on-device model serving and optimization for specific hardware.
The key facts
14 to knowPerplexity open-sourced Lily inference engine
Built in Rust with custom Metal kernels
Optimized for Qwen3.6-35B-A3B on Apple Silicon
1.23x prefill throughput vs MLX-LM baseline
1.35x decode throughput vs MLX-LM baseline
Tested on 40-core, 128GB M5 Max
Powers Hybrid Compute in Perplexity Computer product
Perplexity open sourced Lily inference engine
Built in Rust with custom Metal kernels for Apple Silicon
Optimized for Qwen3.6-35B-A3B model
1.23x MLX-LM prefill throughput improvement
1.35x MLX-LM decode throughput improvement
Benchmarked on 40-core, 128 GB M5 Max
Powers Hybrid Compute in Perplexity Computer
Go to the source
MarkTechPostmarktechpost.com
Publisher excerpt: Perplexity has open sourced Lily, the local inference engine behind Hybrid Compute in Perplexity Computer. Built in Rust with custom Metal kernels for one model on one chip family, it averages 1.23x MLX-LM's prefill throughput and 1.35x its decode throughput on a 40-core, 128 GB M5 Max.