Loading…
Topic
Decision Models
3 posts about Decision Models. Every summary links to the original.
- 3 posts
- 2 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Benchmarking AI decision models against traditional guardrails
Enterprise generative AI applications require guardrails that balance flexibility with reliability. Decision models such as TypeSafe AI’s Jev produce fixed decisions with guaranteed schema and type safety, operating faster and cheaper than LLM-as-a-judge approaches. The Red Hat AI Safety team benchmarked nine guardrails across four methodologies for prompt injection and content safety tasks: pre-trained CPU-scale classifiers like Red Hat’s DeBERTa-v3 and Granite Guardian HAP-125m, older zero-shot models like BART-large-mnli, modern LLM-as-a-judge safety models including Shieldstral and Nemotron-3.5, and Jev-style decision models including Laya and DiffusionGemma. Each guardrail was evaluated using stock and custom risk policies, with Jev and DiffusionGemma using zero-shot questions. The results highlight trade-offs: Jev provides zero-shot flexibility and low latency, but purpose-built classifiers often match its speed and cost while delivering superior task-specific performance, and LLM-as-a-judge models can be tuned for higher accuracy.
Introducing Clef: our open-source decision models, and new RL fine-tuning platform
Cloudflare released two open-source decision models, Clef and Clef-flash, hosted on Workers AI under an Apache 2.0 license and fully compatible with the Jev API. These models produce bounded structured outputs for agentic workflows, classifying inputs like customer support messages or website domains with typed probabilities and confidence scores. Clef includes a vision encoder and a 64k context window, distinguishing it from text-only decision models such as Jev, and currently leads the Jev Decision Index benchmark. The models demonstrated lower latency than alternatives, with Clef classifying a website in 2.2 seconds versus 4.7 seconds for Cloudflare’s fastest general LLM, while Clef-flash offers even faster inference at 38.8 ms median latency. Cloudflare also debuted a reinforcement learning fine-tuning platform that lets customers adapt Clef to their specific use cases using their own data.
Run decision models on vLLM and Red Hat AI using DiffusionGemma
A new category of AI model called ‘System One’ makes fast, typed, probabilistic decisions from unstructured input, exemplified by TypeSafe AI’s Jev, which offers guaranteed structure, default probabilities, and low latency (70–500 ms) for high-volume decisions like routing and moderation. Jev is a hosted API without self-hosting options, prompting the vLLM community to adapt DiffusionGemma, an open block-diffusion model, as a self-hostable decision engine. The approach uses a fixed-length canvas seeded with answer templates, a single denoising step, and calibrated logprobs to return decisions with confidence scores, achieving over 300 decisions per second on a Cloud Run deployment. DiffusionGemma is already validated on Red Hat AI, and the structured-read support in vLLM nightly enables teams to run decision models on their own infrastructure. The post provides a recipe using vLLM nightly and a decision server that exposes a Jev-compatible endpoint for prototyping.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.