FrontierSeptember 17, 2026via Vercel Blog
Run Terminal-Bench and other Harbor evals on Vercel Sandbox
Why it matters
Vercel's Sandbox integration makes it frictionless for practitioners to run standardized AI benchmarks (SWE-bench, Terminal-Bench, OSWorld) against any model via AI Gateway. This lowers the barrier to rigorous capability comparison and speeds up the eval cycle—useful for anyone choosing between models or tracking frontier progress.
Key signals
- Harbor evals (SWE-bench, Terminal-Bench, tau3-bench, OSWorld) now executable on Vercel Sandbox
- Firecracker microVM isolation enables parallelization beyond local hardware
- Network policy enforced at sandbox firewall; credential injection keeps secrets outside the VM
- AI Gateway integration allows benchmarking hundreds of models from multiple providers with single command swap
- Example: --model vercel_ai_gateway/openai/gpt-5.6-luna flag switches eval target
- Harbor v0.22.0 or later required
- Harbor (open-source eval harness) integrated into Vercel Sandbox
- Supports SWE-bench, Terminal-Bench, tau3-bench, OSWorld
- Execution in isolated Firecracker microVMs enables parallelization beyond local capacity
- Network policy enforced at sandbox firewall; credential injection kept outside VM
- Works with AI Gateway to reach hundreds of models from multiple providers
- Single command to swap between models/providers (--model flag)
- Requires Harbor 0.22.0 or later
The hook
Harbor evals now run on Vercel Sandbox—benchmark hundreds of models in parallel without touching your laptop.
You can now run Harbor evals on Vercel Sandbox.
is the open-source harness behind , whose registry includes many other benchmarks such as SWE-bench, tau3-bench and OSWorld. Pass to and each trial executes in its own isolated Firecracker microVM, so you can parallelize far beyond what your local …