LangSmith has added Jev as an evaluator for AI-agent traces, giving developers a new option between rigid code checks and full language models used as judges. Jev is available from the Evaluators tab in LangSmith tracing projects and can score both stored tests and live traffic.
Jev does not generate prose. TypeSafe AI describes it as a “System One” model that reads a state, such as an agent trace or message, and returns typed probabilities, choices or scores. A team could use it to classify user intent, flag possible personal-data leakage or rate frustration without converting a judge’s free-form explanation into structured data afterward. It can also answer several questions about the same trace in parallel.
LangChain says its tests found Jev faster, cheaper and more consistent than the language-model judges it compared against. TypeSafe AI claims gains of up to about 200 times in speed and 450 times in cost on classification tasks, but those figures are vendor-reported and will depend on the workload.
The addition does not replace every evaluator. Deterministic checks remain useful for exact conditions, while an LLM judge is still better when reviewers need written reasoning. Jev is aimed at narrow, repeatable judgments that teams want to run across far more production traces.