Deploy OpenJev locally

OpenJev is the closest thing to a self-hosted Jev server: it speaks the same request shape as Jev and serves it over HTTP. Run the full vLLM stack in Docker for a drop-in endpoint, or pull the GGUF builds into Ollama or LM Studio for an easy cross-platform setup.

What OpenJev is

OpenJev is an open, community decision model and server: each request supplies a state, an instruction and 2–16 candidates, and the model returns one candidate label plus the distribution over them. It's not a chat model. There's also a lightweight OpenJev Verdict variant (a 151M ModernBERT classifier) that adds an abstention probability for low-evidence inputs.

Relationship to official Jev

OpenJev is independent, not TypeSafe's Jev — OpenJev's own README says so. Its value is that it exposes a Jev-compatible HTTP contract on open weights, so you get a local endpoint that behaves like the hosted API without the calibration guarantees. Use it when you need a self-hosted, network-reachable decision server.

Requirements

• Drop-in HTTP API: Docker + an NVIDIA GPU with ~24 GB VRAM (vLLM, NVFP4 weights).
• Easy / laptop route: the APUS-OpenJev GGUF builds (4B or 9B) via Ollama, LM Studio or llama.cpp — Linux, Windows, or macOS (Metal).
• Q8_0 is the recommended default quantization; the 4B build suits lighter machines.

Self-host the Jev-compatible API (Docker + vLLM)

git clone https://github.com/razorback16/openjev && cd openjev
docker compose up -d          # brings up vLLM + a Jev-compatible API
curl localhost:8080/v1/models

# then POST a state + typed questions to the decision endpoint on :8080

Run it in Ollama or LM Studio (GGUF)

No 24 GB GPU? Pull the APUS-OpenJev GGUF and drive it with the OpenJev decision harness — this is the route that answers "can I run a Jev-like model in LM Studio / Ollama?" Yes:

# Ollama (also works in LM Studio / llama.cpp; macOS Metal, Windows, Linux)
ollama pull hf.co/apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q8_0

# drive it as a decision model via the OpenJev harness:
python examples/openjev_local.py --backend ollama \
  --model hf.co/apus-ailab/APUS-OpenJev-v1-9B-GGUF:Q8_0
# lighter machines: swap in the 4B build (APUS-OpenJev-v1-4B-GGUF)

Local API endpoint

The Docker/vLLM route gives you a real HTTP endpoint on 127.0.0.1:8080 that mirrors the Jev contract, so Jev-compatible client code (and the official SDK via base_url) can target it directly. The Ollama/LM Studio route runs the model through the harness rather than a Jev-shaped HTTP server — great for local experimentation.

Reported latency

No official latency number. On the vLLM route, latency and throughput track your GPU; the GGUF builds (Q8_0) run interactively on laptops but are slower than a datacenter GPU. Benchmark your chosen build and quantization on your own hardware.

Calibration caveats

OpenJev's distributions are its own, and quantization (Q8_0 and below) can shift them further. It gives you a Jev-shaped contract, not Jev's calibration — validate on your labelled data, and remember lower quant = smaller/faster but less faithful. The Verdict variant's abstention probability can help you route low-evidence inputs to a human.

Sources

• GitHub — razorback16/openjev — server (Docker / vLLM)
• Hugging Face — APUS-OpenJev-v1-9B-GGUF — GGUF build
• Hugging Face — APUS-OpenJev-v1-4B-GGUF — lighter GGUF build

Check the repo for the current compose file, model revisions and licenses before deploying.

See also: Jev on Hugging Face · Is Jev open source? · ← Compare all local Jev alternatives · Prefer hosted Jev (no GPU)

Other local models: Laya · NanoJev · DiffusionGemma-as-Jev