Using Jev as a judge
LLM-as-a-judge usually means prompting a big model, parsing its reply, and trusting a score it made up. Jev as a judge returns exactly one of your grades, a calibrated probability for each, and a confidence value — in one round trip, with nothing to parse.
Jev is TypeSafe AI's first System One model, and the choice primitive turns it into a judge. You define the verdict space — pass / fail, a 1–5 rubric, which of two answers is better — send the thing being judged as state, and Jev returns one key locked to your options, a probability distribution across them, and a single confidence score. Because the output is typed, a Jev judge never invents a grade, never trails off, and never hands you text you have to clean up.
Why calibration matters for a judge
A judge is only useful if its confidence tracks how often it is right. Jev is trained with RLCD, so its reported probabilities are calibrated — a 0.9 verdict really is right about nine times in ten. That is what lets you route on the score: auto-accept high-confidence verdicts, and send only the low-confidence tail to a human or a heavier model.
- One verdict: locked to the grades you defined
- Calibrated probabilities across every grade
- Confidence you can threshold on for review routing
- Roughly 70–500ms per verdict, about $0.001 each
Jev judge vs prompting an LLM to grade
| Prompting an LLM | Jev as a judge | |
|---|---|---|
| Output | Free text to parse | One typed grade |
| Invalid grades | Possible | Impossible |
| Calibration | Uncalibrated | RLCD-calibrated |
| Latency | Seconds | ~70–500ms |
| Cost | Cents+ | ~$0.001 |
See it running
PR Judge scores a pull request, and Live Post Judge grades a social post in real time — both are Jev as a judge over the hosted endpoint. Point your own evaluations at POST https://jevtypesafeai.com/api/v1/decide with a jv_live_ key (or the official POST https://api.typesafe.ai/v1/systemone) and you get the same typed, calibrated verdict.
See also: PR Judge · Live Post Judge · Jev classifier
Run Jev as a judge
Grab a jv_live_ key and return a typed, calibrated verdict in one call.