Loading…
Red Hat
Major software company that provides open-source technology, such as the Linux operating system, to businesses and large organizations.
- 82 posts summarised
- latest
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
From their blog
Latest articles
Evaluating LLM guardrail configs locally with EvalHub
This installment evaluates local NeMo Guardrails configurations against large-scale risk datasets using EvalHub, following earlier manual testing. EvalHub provides a unified interface for AI evaluation, with provider integrations, reusable benchmarks, Hugging Face dataset inputs, and both local and Kubernetes deployment options. The benchmark configuration maps dataset prompt and label columns to block and pass labels, while balanced safe and risky examples prevent an always-block guardrail from appearing perfect. On the prompt-injection benchmark, the nuanced regex configuration achieved 51.1% accuracy with 13.7 ms median latency; DeBERTa reached 88.93% accuracy with 45.2 ms median latency. The conclusion favors layered guardrails: targeted regex checks can provide speed, with DeBERTa handling other cases, and EvalHub supplies reproducible measurements of the latency–efficacy trade-off.
Evaluate AI agents with IBM CLEAR & EvalHub on OpenShift AI
Reliable AI agents can fail in production through incorrect tool use, off-topic reasoning, or plausible but incorrect answers that unit tests, static benchmarks, and logs may not expose. This tutorial builds an eval-driven trace-evaluation pipeline on Red Hat OpenShift AI 3.4 or later, using MLflow to store structured agent traces and evaluation artifacts, IBM CLEAR to score reasoning quality, relevance, and tool usage with an LLM judge, and EvalHub to orchestrate OpenShift jobs. It uploads eight prerecorded multistep research-agent traces, each containing at least 30 spans, then verifies them in MLflow before deploying EvalHub, registering the ibm-clear provider, and configuring judge-model credentials. Standard mode evaluates steps holistically, while SPARC mode gives tool calls separate scoring rows; the resulting workflow produces critiques and ranked failure patterns with frequency, severity, and step or agent attribution.
Extend Layer 2 networks into Red Hat OpenShift Virtualization with BGP and EVPN
The article describes how to extend Layer 2 networks from a legacy data-center fabric into Red Hat OpenShift Virtualization using BGP and EVPN. It uses a three-node OpenShift lab with secondary NICs on a provider network, static IPv4 addressing, a VTEP CR for 172.19.0.0/24, FRRouting peering, and a Layer 2 primary CUDN with VNI 100. The configuration preserves imported VMs’ IP and MAC identities while maintaining isolation for tenant networks that may have overlapping address spaces. In the example, an external FRR router learns the VM MAC through EVPN and reaches 22.100.0.3; after the VM moves from node 3 to node 1, the MAC remains unchanged and the advertised VTEP changes from 172.19.0.102 to 172.19.0.100. The stated result is phased migration with Layer 2 adjacency to legacy application tiers and end-to-end tenant segmentation.
LoRA backdoor threat: How OpenShift AI mitigates the risk
LoRA adapters can carry semantic backdoors even when the underlying open-weight model is clean, creating a supply-chain risk for customized AI assistants. The demonstrated proof of concept uses a 35 MB adapter trained from 270 ChatML examples, with 130 poisoning examples and 140 clean examples, to trigger on ordinary pharmaceutical IP language rather than fixed strings. With Qwen/Qwen2.5-0.5B-Instruct served by vLLM 0.25.1, the backdoored model covertly called send_email to an external address while showing a benign response, whereas the base model and controls remained clean. The defense-in-depth approach combines OpenShift AI, Advanced Cluster Security, and Advanced Cluster Management to restrict runtime LoRA loading, require provenance-checked adapters, enforce admission and build policies, and block unexpected egress. Because weight inspection cannot reliably reveal semantic behavior, protection focuses on deployment, runtime, tool, and network boundaries.
LLM quantization guide: How to do it, and how it helps
LLM quantization addresses the gap between rapidly growing model sizes and GPU memory by storing weights and activations at lower precision than BF16. It explains how quantization targets linear layers in transformer blocks, distinguishes weight quantization for memory-bound decoding from activation quantization for compute-bound prefill, and maps formats such as GGUF, FP8, NVFP4, AWQ-INT4, and MLX to their inference engines. Calibrated methods including GPTQ, AWQ, and SmoothQuant protect important weights using representative data, while reported NVFP4 results recover about 99% of BF16 accuracy for 70B–235B models and 95–98% for 7B–14B models. In a Llama 3 70B benchmark on two H100s, FP8 weights-and-activations increased throughput from 158 to 474 input tokens/sec and reduced time to first token under load from over 30,000ms to about 4,800ms.
Orchestrate production RAG with OpenShift AI
This guide shows how to move a streaming retrieval-augmented generation (RAG) workflow from a monolithic Ray Data script to modular AI pipelines on OpenShift AI. It addresses failure recovery and run traceability by separating parsing and chunking, ingestion, model download, and deployment into five reusable components. Parsed and chunked JSONL is stored in S3-compatible storage so KFP can rerun ingestion without repeating document processing, and each run records parameters, timestamps, logs, and status. The data chain uses Docling’s HybridChunker and Ray map_batches with ActorPoolStrategy, while the model chain downloads a Hugging Face model and deploys it through vLLM and KServe in parallel. For embedding ingestion, the workflow supports local CPU processing with granite-embedding-125m-english or a dedicated GPU vLLM service for larger collections and offers notebooks for deployment and query validation.
Developing LLM guardrail configs locally with NeMo Guardrails
The article presents a local workflow for developing and testing NeMo Guardrails configurations in a Jupyter notebook, avoiding repeated Kubernetes deployments and requiring no LLM, GPU, or cloud resources. It defines rails, configurations, and servers, then starts with a case-insensitive regex rail that blocks the exact phrase “ignore all previous instructions”; the test catches a direct attack but allows an accented “Ignóre” variant. A broader regex configuration is followed by a DeBERTa-based prompt-injection classifier and a manually assembled set of 13 prompts, producing 12 correct decisions out of 13, with one subtle attack allowed. The article recommends iterating locally, potentially layering targeted regexes before classifiers for fail-fast, defense-in-depth filtering, while noting that 13 prompts cannot meaningfully measure performance and pointing to standardized EvalHub evaluations as a next step.
Red Hat OpenShift autoscaling using MachineSet autoscaler with KEDA
The demonstration compares OpenShift MachineSet Autoscaler with KEDA against the previously used Cluster Autoscaler, using metrics to scale a MachineSet before pods remain pending. After installing Red Hat’s Custom Metrics Autoscaler operator (KEDA 2.19), granting permissions for MachineSet scaling and Thanos/Prometheus access, it creates a ScaledObject with a Prometheus trigger targeting demo-p4p95-worker-eastus3. Because a MachineSet has no pods, the HPA path cannot use metricType: Value, so AverageValue is paired with a PromQL query that multiplies CPU request utilization by the current node count. With nine one-CPU test pods, the MachineSet scaled from one to two to three nodes as the metric crossed 75%, and all pods reached Running; two additional Standard_D4s_v3 VMs were provisioned in zone 3.
AI-powered multicluster management: Querying fleet health with OpenShift Lightspeed and Red Hat Advanced Cluster Management
Managing modern large-scale Kubernetes fleets across distributed clusters can make troubleshooting slow because administrators must build manual search queries and sift scattered outputs. The post presents a Technology Preview MCP server for Red Hat Advanced Cluster Management that translates natural-language prompts into structured searches across managed clusters, deployed with Helm in the hub cluster. OpenShift Lightspeed is configured through its OLSConfig custom resource to call the MCP endpoint, use a Kubernetes service-account token for internal authorization, and optionally skip approval for read-only searches. The examples cover finding clusters below OpenShift 4.21, locating pods in a specified cluster and namespace, and identifying an unhealthy VM across the fleet; one result finds dev-vm in an ErrorUnschedulable condition and provides diagnostic guidance.
Stop wasting GPU allocation in Kubernetes with GPU-pruner
GPU-pruner addresses idle GPU allocation in Kubernetes by identifying workloads whose hardware activity remains below a configured threshold, rather than relying on browser sessions, network traffic, or pod lifetime. It queries Prometheus for NVIDIA Data Center GPU Manager metrics, including engine activity and GPU utilization, and treats peak performance below 1% throughout a default 35-minute observation window as idle. After tracing an idle pod through ownerReferences or KServe labels, it scales the parent resource to zero, preserving workload metadata for later restoration. Slack notifications, grace periods, lookback checks, and dry-run mode provide safeguards, while Prometheus label handling must preserve the workload namespace and pod identity in DCGM telemetry.
Self-service backup for VMs and containers on OpenShift: No cluster-admin required
In multi-tenant Red Hat OpenShift clusters, Velero Backup and Restore resources live in the OADP installation namespace, leaving non-admin application teams dependent on platform staff. OADP self-service addresses this with namespace-scoped NonAdminBackup and NonAdminRestore resources that proxy Velero resources, while a controller validates requests against DataProtectionApplication policies and synchronizes phase, progress, and queue status back to the tenant namespace. Administrators can enforce retention, snapshot and file-system backup settings, resource inclusion or exclusion, and approval for tenant-provided backup storage; cluster-scoped resources such as SCCs, ClusterRoles, CRDs, and KubeVirt cluster preferences are automatically excluded. OADP 1.6 supports the same workflow for containers, stateful applications, and OpenShift Virtualization VMs, but self-service has no native schedules, cross-namespace restores, or short-lived credentials for tenant-provided storage locations.
Building hermetic notebook images for Open Data Hub and Red Hat OpenShift AI
Open Data Hub and Red Hat OpenShift AI adopted hermetic container builds to eliminate network failures, satisfy Conforma compliance, and ensure reproducible notebook environments. Instead of pulling packages during image creation, dependencies are pinned in lockfiles, prefetched into a local cache using tools like Cachi2, and installed with the network disabled. The workflow relies on a unified Containerfile capable of running locally, within GitHub Actions, and on Konflux via Tekton tasks. To pass audit checks and avoid generic tarball downloads, the team packaged awkward binaries like ripgrep and Pandoc into multi-arch Python wheels while patching Node postinstall scripts that fetch files at install time. Upstream CentOS and downstream Red Hat Enterprise Linux dependencies are maintained in distinct lock trees to prevent subtle packaging mismatches.
llm-d flow control: Priority queuing for shared GPU inference
Shared GPU model pools allow platform teams to maximize utilization, but mixed workloads often contend for serving capacity with differing service requirements. Generally available in Red Hat AI Inference 3.5, llm-d flow control introduces explicit admission control prior to backend routing. An Endpoint Picker sits in the request path and evaluates active priority bands from highest to lowest integer values whenever pool saturation is detected. Saturation metrics combine per-replica queue depth and key-value cache pressure to determine when dispatch gates should engage. Once a priority band is admitted, tenant fairness policies distribute dispatch turns, and ordering rules select the next request to forward to a vLLM replica via Envoy.
How AI observability works with MLflow
Traditional application monitoring treats AI workflows as black boxes, leaving engineering teams unable to determine whether incorrect responses result from model hallucinations or erroneous upstream tool data. MLflow provides AI observability by capturing distributed traces containing model calls, prompts, retrieved context, execution latencies, and tool interactions. In an example mortgage lending application, trace inspection revealed that an AI assistant accurately summarized raw JSON data returned by an internal pipeline tool rather than hallucinating conflicting metrics. Beyond latency attribution and tool debugging, MLflow is compatible with OpenTelemetry and provides automatic tracing integrations for frameworks such as LangChain and Claude Code. Capturing these traces enables teams to evaluate subsequent application versions, detect model provider shifts, and identify tool regressions before reaching users.
Try the Ansible playbook generation lab with Gemini and OpenAI
The official Visual Studio Code extension for Red Hat Ansible Automation Platform includes an automation coding assistant, formerly called Ansible Lightspeed. Initially integrated with IBM watsonx, the extension now supports Google Gemini and OpenAI-compatible endpoints through Red Hat AI. An updated interactive lab demonstrates these capabilities through an end-to-end scenario involving database installation and HTML landing page deployment across two nodes. Learners write natural language prompts to generate Ansible playbooks and roles, evaluate output, and utilize real-time automated linting and integrated documentation. By serving an open-source model through a preconfigured OpenAI-compatible endpoint, the refreshed environment removes external credential setup steps.
Automating Red Hat OpenShift AI installations with Helm and GitOps
Deploying Red Hat OpenShift AI manually across multiple environments is error-prone due to the numerous required dependencies, including certificate managers, job queuing systems, and GPU operators. To streamline this lifecycle, the odh-gitops repository introduces the rhai-on-openshift-chart Helm chart. The chart automates operator provisioning through Operator Lifecycle Manager and configures custom resources such as DataScienceCluster. Dependency resolution is controlled via a tri-state model that automatically installs prerequisites like cert-manager and Red Hat Connectivity Link when dependent components are enabled. Using this chart, teams can deploy full platform installations or minimal inference stacks through standard Helm CLI commands or Argo CD GitOps pipelines.
GRPO fine-tuning on Red Hat OpenShift AI: Reinforcement learning from verifiable rewards with Training Hub
Supervised fine-tuning for structured reasoning tasks like tool calling often demands expensive, labor-intensive dataset curation. Group Relative Policy Optimization (GRPO) resolves this by using programmatic reward functions to score multiple candidate responses without requiring human labels or a dedicated reward model. On Red Hat OpenShift AI, developers can implement GRPO using Training Hub, Kubeflow Trainer, and the Kubeflow SDK to fine-tune models like Qwen3-4B. The setup utilizes vLLM for candidate generation alongside LoRA weight updates, sharing storage via ReadWriteMany persistent volume claims. In a practical tool-calling test, five training iterations raised task accuracy from 33% to 67%, teaching the model to emit parseable function invocations rather than fabricated conversational text.
Red Hat OpenShift autoscaling with Cluster Autoscaler
Red Hat OpenShift supports compute scaling using Cluster Autoscaler integrated with the OpenShift machine API. The mechanism relies on paired ClusterAutoscaler and MachineAutoscaler custom resources to manage cluster-wide limits and per-MachineSet replica boundaries. Scaling decisions evaluate pending pods against node allocatable capacity and hypothetical node capacity annotations while respecting expander strategies like Random, LeastWaste, or Priority. Cluster Autoscaler checks for unnecessary nodes every 10 seconds, scaling down when utilization falls below configured thresholds and workloads can safely migrate. To mitigate provisioning delays, clusters can adopt overprovisioning via low-priority pause containers or use APIs like ProvisioningRequest and CapacityBuffer for predictive scaling.
Run LoRA fine-tuning on Red Hat OpenShift AI with Ray
Red Hat OpenShift AI 3.5 integrates Ray cluster runtime images with preinstalled Training Hub packages to support distributed fine-tuning algorithms including LoRA, SFT, OSFT, and GRPO. Using the CodeFlare software development kit within a Jupyter workbench, developers can provision an elastic single-GPU Ray cluster attached to shared ReadWriteMany persistent storage containing model weights and training datasets. The tutorial demonstrates fine-tuning the Qwen2.5-1.5B-Instruct model on the public sql-create-context dataset to produce direct, bare SQL output without explanatory conversational text. After training completes, the workbench loads and merges the resulting LoRA adapter checkpoints with the base model to evaluate output format correctness against sample database schema questions. The RayCluster automatically tears down resources once the job concludes, while the shared pipeline architecture allows switching between supported fine-tuning algorithms by modifying dataset inputs and configuration parameters.
Improved failure reports on Red Hat OpenShift with the event-driven diagnostic operator
During major infrastructure incidents, critical system logs and diagnostic evidence are frequently overwritten, lost, or erased by emergency recovery efforts before engineers can investigate. To solve this visibility gap in multi-cluster environments, an open-source event-driven diagnostic operator was developed as an automatic first information report collector. Deployed on a central hub cluster, the operator monitors Kubernetes warning events rather than relying on heavy polling. When an event matches a predefined regular expression pattern, the operator identifies the target spoke cluster, copies its kubeconfig secret, and spins up an independent must-gather job. Diagnostic data is written to off-site or shared storage with a configurable retention window, preserving forensic context without human intervention.
Topics they write about
All topics →Weekly digest
The best engineering reads, once a week.