The Jev benchmark, in plain numbers

Every Jev benchmark you will see today is about speed, cost and calibration — not a public accuracy leaderboard. This page explains what the published figures actually mean and how to test them.

What the numbers are

The headline figures are performance, not accuracy. Jev returns a decision in 70 to 500 ms, roughly 40 to 200 times faster than a frontier LLM. On cost, it is $0.042 per million input tokens, output free — about $0.0004 per decision.

• Latency: 70 to 500 ms per call.
• Speed: about 40 to 200 times faster than a frontier LLM.
• Cost: $0.042 per million input tokens, output free.
• Per decision: roughly $0.0004.
• Throughput: about 250,000 tokens per second and 1,200 requests per minute.
• Context: up to about 64k tokens, with 32k for state plus one longest question.

Calibration is the real Jev benchmark

Because Jev is trained with RLCD for calibration rather than fluency, the meaningful measure is whether its probabilities match reality. Every answer carries per-option probabilities and a confidence, where confidence describes how concentrated the answer is, not whether it is right.

What the figures do not include

Be honest about the gaps: TypeSafe AI has not published a full accuracy leaderboard, so no benchmark table ranks Jev against named models on labelled tasks. Treat the published numbers as speed and cost claims you can verify, not as a quality score.

Run your own Jev benchmark

The only Jev benchmark that matters for you is on your own labelled data. Send a batch of real cases, compare Jev's decisions to your ground truth, and calibrate your confidence thresholds. That turns generic figures into numbers you can trust in production.

• Collect a labelled sample from your own domain.
• Call Jev with your real states and questions.
• Compare decisions to ground truth and tune your thresholds.

See also: Pricing · How to use the API · Playground

Related: Jev vs an LLM · System One model · Jev architecture