A specialized decision model came close to a frontier chat model on a banking classification test while using far less time and money. OpenRouter ran 3,080 customer-support messages from the Banking77 dataset through TypeSafe’s Jev 1.13 and Claude Opus 5.
Jev assigned the correct one of 77 intents 81 percent of the time, compared with 84.4 percent for Opus. Its median response took 175 milliseconds rather than 2.27 seconds. The full run cost $0.34 for Jev and $7.44 for Opus, equal to $0.11 and $2.42 per thousand requests. Neither model returned malformed output.
The accuracy gap was statistically meaningful: OpenRouter calculated a 95 percent confidence interval of 2.3 to 4.4 percentage points. Opus alone got 175 disputed examples right, while Jev alone got 72. Both remained below fine-tuned encoders that score in the low 90s on this dataset.
The comparison covers one narrow, short-text task and was conducted by the platform selling access to both models. It shows a useful cost-quality trade, not general superiority. Teams could also use Jev’s confidence values to send uncertain cases to a larger model or a person.