Why We Built LangSmith for Improving Agent Quality

Harrison Chase (CEO of LangChain) sits down with Bagatur (LangSmith Engineer) and Tanushree (Product Manager) for a technical roundtable on bringing production agents from prototype to rigor. They discuss the evolution of LangSmith's platform, dive deep into the new Insights Agent feature for automatically discovering patterns in production traces, and explore Multi-turn Evaluations for understanding end-to-end user interactions.

00:00 - Introductions + the evolution of LangSmith
02:39 - Introducing Insights Agent
03:49 - Real-world use cases for Insights Agent
04:44 - Customizing insights for your specific use case
05:22 - The algorithm behind Insights Agent
06:30 - The hardest part of getting Insights to work
07:13 - Tips for getting started with Insights
08:47 - Evals vs Insights - what's the difference
09:36 - What are Threads and why do they matter?
11:59 - Offline vs online evals
12:46 - Multi-turn evals for measuring agent performance in production
13:19 - Thread-level metrics and workflows
14:22 - The hot take: "Are evals dead?"
16:08 - The future of testing
17:08 - Closing thoughts

Read more about our latest LangSmith updates: https://bit.ly/3WrUNDZ
Learn more about Insights Agent: https://docs.langchain.com/langsmith/insights
Learn more about Multi-turn Evals: https://docs.langchain.com/langsmith/online-evaluations#configure-multi-turn-online-evaluators