How Ora benchmarks every major AI agent on Vercel
Ora benchmarks every major agent framework side-by-side on live websites. The results: 7% fewer steps, 2x native success, 99% of the web still isn't agent-ready.
Why it matters
A production agent benchmarking platform reveals where agents fail on real websites and how framework choice (Claude Code, ChatGPT, Gemini, eve) affects task completion. Real deployment data for practitioners choosing agent stacks.
The key facts
9 to knowOra runs agents on live customer websites to measure discovery, navigation, and transaction capability
7% fewer steps to goal; 2x native success rate (tasks completed on customer site vs falling back to web search); 9% more valid endpoints
Agents tested: Claude Code, ChatGPT, Gemini, Hermes, OpenClaw, eve (Vercel's framework)
99% of the web estimated as not yet agent-ready by Ora's measure
eve (Vercel framework) passed Ora's benchmarks; Ora now uses eve for internal agent infrastructure
Sandbox override feature in eve allows Ora to instrument and trace eve agents in their own execution environment
16-person engineering team at Ora ships hundreds of commits per day using coding agents on Vercel stack
Prompt-caching fix in eve reduced total cost by ~15% in subsequent benchmarks
Platform runs entirely on Vercel: frontend, backend, and agent runtime share deployment, logs, authentication
Go to the source
Vercel Blogvercel.com
Publisher excerpt: Ora on Vercel Every harness expects its own infrastructure One platform under every harness Testing eve like any other harness The framework behind Ora's own agents Front end, back end, and agent runtime on one platform Every major agent tested side by side on live sites Hundreds of commits a day…