LangChain has published a practical evaluation workflow for teams that need to catch agent regressions before and after deployment. Its recommendation is to start with one important behavior, collect 10 to 20 representative traces and record a baseline rather than attempting a comprehensive test suite immediately.
A trace captures the user input, final response and every model or tool call in between. That matters because an agent can produce an acceptable answer after taking an unsafe or wasteful path. Tests can therefore inspect individual tool calls, the final response, a complete conversation or the end state of a sandboxed environment.
Offline evaluations run a fixed dataset before a release and can compare a changed prompt, model or tool with the baseline. Online evaluations score production traces and expose failures the prepared dataset missed. LangChain recommends adding those failures to the offline set, creating a loop between monitoring and regression testing.
Deterministic code checks are preferred when a rule can be expressed precisely, such as staying under a travel budget. Model judges can grade less structured qualities, but teams should compare them with human decisions and use narrow pass-or-fail questions. Human review remains the reference for subjective quality and for detecting when an automated judge has drifted.