Evaluating Deep Agents using LangSmith on AWS
Deep agents hitting production. Here's how to evaluate them before they break.
Why it matters
AWS and LangChain are publishing practical tooling for evaluating agentic AI systems in production—a critical gap as enterprises move agents from research to real workflows. This signals growing maturity in the agent deployment lifecycle.
The key facts
12 to knowFive evaluation patterns for deep agents documented
Offline evaluation using pytest and LangSmith
Online monitoring for production agents
Text-to-SQL deep agent walkthrough on Amazon Bedrock
Full dev-to-production lifecycle guidance
AWS, LangChain, and Anthropic collaboration on evals framework
Offline evaluation approach using pytest and LangSmith
Online production monitoring configuration included
Text-to-SQL deep agent use case demonstrated
Amazon Bedrock integration for inference
Full development-to-production lifecycle covered
Combines learnings from LangChain and Anthropic
Go to the source
AWS Machine Learning Blogaws.amazon.com
Publisher excerpt: This post combines learnings from LangChain’s work on evaluating deep agents and Anthropic’s guide to demystifying evals for AI agents into a practical guide. In this post, you will learn how to: 1) apply five evaluation patterns for deep agents, 2) build offline evaluations using pytest and…