Topic

Red Hat AI

1 post about Red Hat AI. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Posts

Red Hat
Run decision models on vLLM and Red Hat AI using DiffusionGemma
A new category of AI model called ‘System One’ makes fast, typed, probabilistic decisions from unstructured input, exemplified by TypeSafe AI’s Jev, which offers guaranteed structure, default probabilities, and low latency (70–500 ms) for high-volume decisions like routing and moderation. Jev is a hosted API without self-hosting options, prompting the vLLM community to adapt DiffusionGemma, an open block-diffusion model, as a self-hostable decision engine. The approach uses a fixed-length canvas seeded with answer templates, a single denoising step, and calibrated logprobs to return decisions with confidence scores, achieving over 300 decisions per second on a Cloud Run deployment. DiffusionGemma is already validated on Red Hat AI, and the structured-read support in vLLM nightly enables teams to run decision models on their own infrastructure. The post provides a recipe using vLLM nightly and a decision server that exposes a Jev-compatible endpoint for prototyping.

The best engineering reads, once a week.

Topic

1 post about Red Hat AI. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Red Hat
Run decision models on vLLM and Red Hat AI using DiffusionGemma
A new category of AI model called ‘System One’ makes fast, typed, probabilistic decisions from unstructured input, exemplified by TypeSafe AI’s Jev, which offers guaranteed structure, default probabilities, and low latency (70–500 ms) for high-volume decisions like routing and moderation. Jev is a hosted API without self-hosting options, prompting the vLLM community to adapt DiffusionGemma, an open block-diffusion model, as a self-hostable decision engine. The approach uses a fixed-length canvas seeded with answer templates, a single denoising step, and calibrated logprobs to return decisions with confidence scores, achieving over 300 decisions per second on a Cloud Run deployment. DiffusionGemma is already validated on Red Hat AI, and the structured-read support in vLLM nightly enables teams to run decision models on their own infrastructure. The post provides a recipe using vLLM nightly and a decision server that exposes a Jev-compatible endpoint for prototyping.

The best engineering reads, once a week.