Presentation: From OTEL to SLMs: Distilling Frontier Model Behaviour from Production Telemetry
Not a pilot. Teams are now distilling frontier model capabilities into local SLMs using production telemetry — cutting costs while keeping performance.
Why it matters
Organizations are moving beyond expensive frontier models by instrumenting AI agents with OpenTelemetry to create continuous feedback loops that train cheaper, specialized smaller language models. This represents a shift toward cost-efficient, localized AI infrastructure.
The key facts
12 to knowOpenTelemetry instrumentation for AI agent tracking
Implicit labeling from user actions (accept/dismiss/regenerate)
Continuous data flywheel for model distillation
Frontier model capabilities distilled into smaller language models (SLMs)
Cost reduction through localized model deployment
Language Server Protocol (LSP) use case for code intelligence
Distilling frontier model capabilities into smaller local models (SLMs)
OpenTelemetry instrumentation for AI agent behavior tracking
User actions (accept/dismiss/regenerate) as implicit training labels
Continuous data flywheel approach to model optimization
Cost reduction through inference optimization
Custom Language Server Protocol (LSP) implementations
Go to the source
InfoQ AI/MLinfoq.com
Publisher excerpt: Ben O'Mahony discusses building custom AI-powered Language Server Protocols (LSPs) that go beyond standard rule-based checkers. He explains how to instrument AI agents natively with OpenTelemetry to track concrete user actions (accepting, dismissing, or regenerating code fixes) as implicit labels,…