k
keynews.ai
keynews.ai
FrontierThe story, in brief

Supabase Releases Evals: an Open Source Benchmark That Scores Claude Code, Codex and OpenCode on Real Supabase Tasks

Supabase just open-sourced a benchmark that grades Claude Code, Codex, and OpenCode on real-world database tasks — deterministic scoring, no handwaving.

Reporting by MarkTechPost 1 min brief
Illustration of a transparent lens revealing connected networks across layers of paper.
Exploring the next frontier of AI research.AI illustration by KeyNews
The KeyNews take

Why it matters

A vendor-authored eval framework for coding agents on domain-specific tasks (database operations, Edge Functions, RLS policy fixes). Practitioners get a reproducible benchmark; enthusiasts see how capability measurement is shifting from general math/code contests to task-specific, containerized workflows.

The key facts

6 to know

  1. Open-source benchmark: supabase/evals (Apache 2.0)

  2. Models evaluated: Claude Code, Codex, OpenCode

  3. Task domains: schema building, Edge Function debugging, RLS policy fixes

  4. Evaluation method: deterministic checks + LLM-as-judge scoring

  5. Infrastructure: containerized stacks for reproducibility

  6. Scope: real Supabase tasks (not synthetic)

Go to the source

MarkTechPostmarktechpost.com

Publisher excerpt: Supabase has open sourced supabase/evals, an Apache-2.0 benchmark and framework that runs coding agents including Claude Code, Codex and OpenCode against real Supabase tasks — building schemas, debugging Edge Functions, fixing RLS policies — inside containerized stacks, then scores them with…
Read original report
Back to today's editionMore frontier news

The wider picture

View all
Illustration of a transparent lens revealing connected networks across layers of paper.
AI illustration by KeyNews
Frontier01

Alibaba Unveils Zhenwu V900 — and Plans Qwen Models With Up to 10 Trillion Parameters

Alibaba is advancing on two fronts simultaneously: announcing a custom AI accelerator (Zhenwu V900) and committing to massive model scale (10T parameters for future Qwen releases). For practitioners, this matters as a credible third-party capability play outside the US-China licensing squeeze; for enthusiasts, it's a significant lab-race signal about training compute and parameter scaling as competitive levers.

TechRepublic
Illustration of a transparent lens revealing connected networks across layers of paper.
AI illustration by KeyNews
Frontier02

Kyutai Releases Voice of Reason: A Speech-Native Model that Solves Spoken Math with Reinforcement Learning

A new capability frontier: models that reason directly in speech without transcription bottlenecks. This changes how we think about multimodal reasoning and what's possible with open-weight releases at scale.

MarkTechPost
Illustration of a transparent lens revealing connected networks across layers of paper.
AI illustration by KeyNews
Frontier03

Claude Opus 5.5, GPT-6 Sol, GPT-6 Luna, and a new price war

Frontier labs are shipping upgraded reasoning and multimodal models in rapid succession, signaling acceleration in the capability race. Simultaneous price cuts reshape AI economics for practitioners.

Simon Willison