Show HN: Forge – Guardrails take an 8B model from 53% to 99% on agentic tasks
An 8B model just matched a $20/month API. Not with better weights—with better guardrails. Forge takes local LLMs from 53% to 99% on agentic tasks.
Why it matters
Open-source reliability layer proves that serving infrastructure and system-level guardrails can close the capability gap between local 8B models and frontier APIs on multi-step agentic workflows—reshaping the economics of self-hosted AI deployment.
The key facts
8 to knowMinistral 8B + Forge: 99.3% vs Claude Sonnet alone: 87.2% on agentic tasks
8B local model with Forge outperforms frontier API without guardrails by 12 percentage points
Retry nudges alone account for 24-49 point accuracy swings in ablation study
Same model weights show 75-point accuracy variance across serving backends (llama-server vs Llamafile)
Error recovery mechanism scores 0% for all models tested without guardrails—architectural absence, not capability gap
Paper accepted to ACM CAIS '26, peer-reviewed across 97 model/backend configurations, 18 scenarios, 50 runs each
Serving backend infrastructure can swing accuracy by 7% to 83% with identical weights
Open-source repo with eval harness, interactive dashboard, and proxy server mode
Go to the source
Hacker Newsgithub.com
Publisher excerpt: Hi HN, I'm Antoine Zambelli, AI Director at Texas Instruments. I built Forge, an open-source reliability layer for self-hosted LLM tool-calling. What it does: - Adds domain-and-tool-agnostic guardrails (retry nudges, step enforcement, error recovery, VRAM-aware context management) to local models…