AI Gateway adds confidence-based decision fallbacks
Vercel's AI Gateway now escalates to a fallback model when confidence dips below your threshold—catch low-confidence answers before they reach users.
Why it matters
A practitioner feature for reliability: set confidence conditions on choice/score/boolean questions, and the gateway automatically runs a second model if the primary answer is uncertain. Useful for production workflows where a low-confidence answer is worse than a fallback; both stages bill.
The key facts
15 to knowFeature: confidence-based decision escalation in Vercel AI Gateway
Supports Choice, Score, and Boolean question types
Conditions can be combined; Boolean questions escalate on probability range
Fallback configured via providerOptions.gateway.models conditional object
Example: { "confidenceBelow": 0.6 } escalates any Choice or Score answer below 60% confidence
Both primary and fallback decisions are billed
Currently in beta; documented in decision fallbacks documentation
Existing model-name fallbacks unaffected by the new feature
Feature: confidence conditions on Choice, Score, and Boolean questions
Fallback triggered when confidence signal (e.g., confidenceBelow: 0.6) is met
Conditions can be combined for multi-signal escalation
Escalated requests bill both primary and fallback model calls
Status: beta
Supports optional per-question filtering; defaults to all matching question types if no question field specified
Configured via providerOptions.gateway.models.fallback conditional object
Go to the source
Vercel Blogvercel.com
Publisher excerpt: can now escalate a decision request to a fallback model when the primary model's answer trips a confidence condition you set.AI Gateway Conditions can be combined, so an escalation can depend on more than one signal. The plain model names you already list in keep catching outright errors, so…