Loading…
Red Hat
Major software company that provides open-source technology, such as the Linux operating system, to businesses and large organizations.
- 81 posts summarised
- latest
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
From their blog
Latest articles
Constraining AI agents with Red Hat AI: Containment, identity, and governance
AI agents running on laptops or clusters may inherit broad filesystem, network, and credential access, making faulty permissions or prompt injection potential security risks. The guide uses Red Hat OpenShift to build layered controls around an OpenClaw agent: namespace isolation, default-deny network policy, ResourceQuota, and optional Kata Containers contain workloads and limit escape or resource abuse. SPIFFE and SPIRE issue rotating SVIDs based on workload attributes such as namespace and ServiceAccount, allowing services to verify an agent without shared secrets. Governance adds OPA Gatekeeper admission policies, an MCP Gateway with risk-tiered tool authorization, NeMo Guardrails for prompt and completion filtering, and MLflow traces for auditability. The recommended rollout starts with namespace isolation and quotas, then adds sandboxing, identity, and the more involved governance controls.
Red Hat Developer Hub: Preventing compliance violations with AI coding agents
AI coding agents can generate services quickly, but they may miss organization-specific architecture, ownership, and compliance rules stored in an internal developer portal. A demo connects Claude Code to a Backstage-based Red Hat Developer Hub through Backstage CLI actions, allowing the agent to query domains, components, APIs, software templates, and TechDocs before proposing a design. In a simulated Parasol Insurance catalog with more than 500 entities across 14 domains, the agent finds an existing but unrelated reconciliation service and resolves a Claims–Billing messaging conflict by keeping IBM MQ at the payment boundary and publishing internal events to Kafka. It then uses catalog relationships and specifications to identify integrations and produces a Technology Decision Summary that cites the sources behind each choice, making the resulting decisions reviewable and traceable.
Understanding W8A8 INT8 LLM quantization: Accuracy and performance results
The post evaluates whether W8A8 INT8 quantization can reduce the size of a Llama 3.1 8B Instruct model without sacrificing accuracy or inference performance. Using SmoothQuant to address activation outliers and GPTQ with error compensation for BF16 weights, the model was compressed from 14.9 GB to 8.0 GB after calibration on WikiText-2. Identical zero-shot lm-eval-harness tests across MMLU, ARC Easy, HellaSwag, and IFEval found MMLU, ARC Easy, and HellaSwag deltas smaller than their standard errors, while the IFEval change was described as likely within a similar noise range; the post concludes no measurable degradation. vLLM and GuideLLM tests on one NVIDIA L40S reported 29% more concurrent requests, 24% lower synchronous TTFT, and 44% faster token generation at maximum load, while ITL degraded more quickly under high concurrency because of dynamic activation quantization overhead.
Python 3.14 free-threaded build is now available in RHEL
Red Hat Enterprise Linux 9.8 and 10.2 now provide a free-threaded Python 3.14 build alongside the regular interpreter, enabling developers to test parallel CPU execution with threads. The build disables the GIL, allowing Python threads to execute on multiple CPU cores in one process, while built-in dict, list, and set operations retain internal locking for individual operations. Developers install the python3.14-freethreading package from the CodeReady Linux Builder repository; it provides the separate python3.14t executable and commands to verify free-threading support and GIL status. Existing code may expose race conditions, and unsupported C-extension dependencies can re-enable the GIL, so the source recommends testing, dependency checks, explicit synchronization, and workload measurement. The build carries a stated 5%-10% single-threaded performance overhead and mainly targets CPU-bound threaded workloads.
Unlock a LUKS root over SSH on Fedora and Red Hat Enterprise Linux
Remote Fedora or Red Hat Enterprise Linux servers with encrypted LUKS roots can stop in the initramfs, waiting for a passphrase at an unattended console. The documented setup installs dracut-sshd from EPEL, adds dracut’s network-manager module, enables rd.neednet=1 and ip=dhcp in GRUB, then rebuilds the initramfs and boot configuration. NetworkManager brings up the initramfs network, while SSH exposes systemd-tty-ask-password-agent; the SSH session closes after unlocking because the root transition destroys the initramfs sshd. The rd.neednet=1 argument creates the neednet flag that gates nm-initrd.service, and the generated connection profile carries into the real root. Initramfs credentials sit unencrypted on /boot, so dedicated host keys, separate known_hosts entries, and physical testing are recommended before relying on remote recovery.
Bringing custom knowledge to agents with AutoRAG
Enterprise LLMs can hallucinate when they lack company-specific vocabulary and documentation, so the guide builds a bank agent grounded in internal data. It configures Red Hat OpenShift AI 3.5 Technology Preview features, enables OGX, AutoRAG, and the MCP catalog, and deploys PostgreSQL, Milvus, and MinIO for metadata, vector embeddings, and artifacts. AutoRAG processes documents, evaluates pipeline configurations against a benchmark dataset, and identifies a preferred setup; the example uses Qwen3-8B and embeddinggemma-300m models. A MariaDB MCP server adds access to customer records, with a patch supplying writable logging storage and a warning that unrestricted ALLOWED_HOSTS is for demo/development only. The resulting Streamlit agent answers a customer-information query and a Gold-card-limit question using RAG context and database tools.
Red Hat edge platforms: Choosing the right one for your use case
Choosing an edge platform requires balancing hardware footprint, workload needs, high availability, and operational complexity; the options range from RHEL with Podman or KVM to OpenShift clusters. The guide organizes Red Hat's offerings into RHEL-based and OpenShift-based deployments, noting that both traditional virtual machines and modern containerized workloads can coexist and that platforms can overlap as requirements grow. It details image-mode RHEL (bootc), transactional updates with rollbacks, efficient over-the-air delivery, offline installation, and dependency pre-resolution for constrained or disconnected sites. RHEL is positioned for static workloads of one to five components, while Red Hat Device Edge adds MicroShift when lightweight Kubernetes is needed. Kubernetes becomes more appropriate for larger microservices or event-driven systems, automated recovery, dynamic workloads, cross-environment scaling, and rolling updates; the series next covers OpenShift topologies.
From token consumer to token provider: Building your org's AI API
Enterprise AI usage can become costly and fragmented when teams rely on separate provider accounts, personal keys, and independently operated inference stacks, leaving spending, workloads, and governance difficult to see. The post presents Model-as-a-Service (MaaS) in Red Hat OpenShift AI 3.4 as a shared service in which a platform team deploys approved models and exposes them through a standard, OpenAI-compatible API. Administrators can use the model catalog, distributed inference through llm-d on NVIDIA L40s, scoped API keys, rate limits, and dashboards for token use, latency, errors, GPU utilization, and queue length. Developers obtain self-service endpoints and can connect tools such as Claude Code, OpenCode, or Continue with existing clients. The architecture combines OpenShift, vLLM, an AI gateway, Prometheus, Grafana, and Jaeger to centralize inference, governance, and cost attribution across local and external model providers.
Deploy NeMo Guardrails on Red Hat OpenShift AI
This article shows how to deploy an evaluated NeMo Guardrails configuration on a Red Hat OpenShift AI cluster, continuing a series on local development and dataset-based evaluation. It requires OpenShift AI 3.5 or later and namespace write access, while OpenShift AI 3.2 or newer can be configured to use the 3.5 NeMo Guardrails image through TrustyAI service-operator changes. Deployment uses a ConfigMap for the configuration, a NemoGuardrails custom resource, and an authentication annotation that protects the server route; an empty rails.co file is sufficient when no custom algorithms are defined. The deployed server accepts v1/guardrail/checks requests with a bearer token, and the demonstrated classifier permits a benign prompt, blocks a prompt injection, and rejects an invalid token. The article notes that LLM proxying and LLM-based guardrailing are outside its scope.
Use a local and open source code assistant
Closed or cloud-based coding assistants can expose work to external services, so the article presents a local, open-source alternative built around OpenCode. OpenCode is an AI coding agent—not an LLM—that runs as a terminal interface, desktop application, or IDE extension and connects to models through the Model Context Protocol (MCP). The setup installs OpenCode, selects a local model runtime such as Ollama for AMD/NVIDIA GPUs or OpenVINO Model Server with INT4 Qwen3-8B for Intel integrated graphics, then configures opencode.jsonc and an editor plug-in. Examples cover Qwen3:14b with Ollama, Podman-based OpenVINO serving, and NetBeans integration; the resulting assistant can answer queries and edit files after permission is granted, although local inference may be slow and generated advice can be unreliable.
Personalize your product's text-to-speech voice for any language: Fine-tuning with Kubeflow Trainer on Red Hat OpenShift AI
Training an English-centered text-to-speech model for Turkish can produce fluent but hard-to-understand pronunciation for native speakers. This tutorial fine-tunes unsloth/orpheus-3b-0.1-pretrained with about 81,000 Turkish text-audio pairs on Red Hat OpenShift AI using a Kubeflow Trainer TrainJob distributed across two nodes with two A100 GPUs each. LoRA freezes the 3-billion-parameter base and trains 97.3M adapter parameters, while preprocessing converts text to Llama-3 tokenizer tokens and audio to three-layer SNAC codes, masking text labels during scoring. The merged model is evaluated against the base on ten Turkish sentences using Whisper transcription and WER/CER metrics. Mean WER falls from 2.44 to 0.35 and CER from 1.59 to 0.09, though the authors describe the small test as directional.
Optimize vLLM speculative decoding with FastMTP heads
Autoregressive LLM inference is memory-bandwidth bound because each token requires a full pass over billions of parameters, while speculative decoding can verify several drafted tokens in one pass. Speculators 0.6.0 adapts a single native MTP head with FastMTP-style recursive fine-tuning, addressing the mismatch between teacher-forced training and vLLM’s autoregressive reuse of the head. The training loop fuses each next-token embedding with the current hidden state, predicts the following token, feeds the resulting hidden state back, and weights three steps with normalized exponential decay using β = 0.6. Native mtp.* weights extraction, shared full-vocabulary embeddings and output head, and vLLM-compatible stitching support deployment; on Qwen3-Next-80B-A3B-Instruct evaluated on GSM8K, mean accepted length improved at positions 0–2 from 0.897, 0.719, 0.476 to 0.912, 0.776, 0.616.
Understanding W8A8 INT8 LLM quantization: Half the size, better performance, same accuracy
Serving Llama 3.1 8B in BF16 requires roughly 14.9 GB for weights, leaving little VRAM on a 16 GB GPU for KV cache and activations, which limits practical deployment. The workflow benchmarks the base model, applies INT8 W8A8 quantization with llm-compressor, then repeats accuracy and performance tests under identical hardware and load conditions. SmoothQuant modifies weights to reduce activation outliers, while GPTQ uses calibration data and error compensation before rounding weights to INT8; the experiment used 512 WikiText-2 samples and targeted deployment with vLLM. On the Llama 3.1 8B Instruct model, compression reduced storage from 14.9 GB to 8.0 GB, a 46% reduction, and accuracy was preserved across MMLU, ARC, HellaSwag, and IFeval; generation-speed results are reserved for part 2.
Red Hat Developer Hub software template authoring with rhdh-templates
Red Hat Developer Hub software template authors often face Nunjucks syntax, uncertain location.yaml placement, and errors discovered only after rendering in the developer portal. The rhdh-templates skill in the RHDH Users Skill Pack equips an AI assistant with RHDH and Backstage conventions, Nunjucks helpers, schema checks, reference templates, and local validation. After installation, it can scaffold repositories, guide interactive parameterization of existing codebases, add template parameters and Scaffolder steps, generate location.yaml, and check templates offline before merge. With an optional reachable RHDH instance, the skill can list active actions, read their schemas, and dry-run execution against the API; the walkthrough uses the Quarkus getting-started quick start and reports a clean validation run as the readiness signal.
Reference architecture for HA scanning with Red Hat Advanced Cluster Security for Kubernetes
Red Hat Advanced Cluster Security for Kubernetes exposes an image-scanning API through Central, but upgrades and restarts can interrupt CI/CD pipelines. The reference architecture uses two independent Central Service instances—primary and failover—with separate databases, scanner deployments, and namespaces, plus a client-side script that probes the primary’s health endpoint before each scan and falls back when needed. Operators install the failover instance, duplicate image registry integrations and build-time policies, configure CI authentication, and pause reconciliation on the failover Central custom resource while upgrading the primary, then resume it to upgrade the second instance. This avoids scan API unavailability during the staged upgrade process, but failover caches are cold, requiring temporarily longer scans and additional registry traffic. The instances do not synchronize databases, caches, configuration, or vulnerability updates, and the design does not provide general-purpose HA for secured cluster services or delegated scanning.
From incident to remediation: Building an AI-driven AIOps workflow with Red Hat Ansible Automation Platform
This tutorial builds an AIOps workflow for diagnosing and remediating a deliberately induced CPU incident in a product catalog application running on an AWS EC2 instance with RHEL. An Ansible investigation playbook collects CPU utilization, top processes, application status, and HTTP health evidence, which Claude Code analyzes through MCP to identify the Python process as the likely contributor and recommend the approved “AIOps - Remediate High CPU” Job Template. AI does not access the host directly: Ansible Automation Platform retains responsibility for credentials, inventory, execution, and remediation. The remediation changes the application from HTTP 503 and degraded to HTTP 200 and healthy, while a final health check confirms recovery and completes the detect-investigate-analyze-recommend-execute-verify sequence.
Build AI-assisted firewall workflows with Ansible Automation Platform
The workflow replaces a manual, fragmented process for turning a simple ServiceNow connectivity request into firewall changes. Event-Driven Ansible connects ServiceNow, GitHub, an LLM, and two Palo Alto Networks firewalls, checking drift, gathering topology data, and coordinating structured rule proposals, pull requests, and workflow records. Ansible Automation Platform keeps credentials away from the AI and deploys approved rules through a deterministic, idempotent path after engineer and CAB approvals. GitHub’s main branch records the last successful state, proposed changes are reviewed as Configuration as Code, and execution stops when live configuration differs from the intended state. The lab demonstrates partial coverage, no coverage, and an unresolvable request; failed deployments can roll back, while an undefined Palo Alto Networks App-ID stops the workflow for human intervention.
Configure admission fair sharing in Red Hat build of Kueue 1.4
Red Hat build of Kueue 1.4 provides admission fair sharing for shared CPU, GPU, and high-performance computing clusters where FIFO admission can starve later workloads. The guide configures the feature through the Kueue custom resource, using either default values—30-minute usage half-life, five-minute sampling interval, and resource weights of 1—or custom settings, then creates a ResourceFlavor, a ClusterQueue using UsageBasedAdmissionFairSharing, and two LocalQueues. Kueue tracks each LocalQueue's historical consumption, adds a temporary penalty when a workload is admitted, and compares usage adjusted by queue weight to choose the lowest-use pending workload when quota returns. In the example, lq-a receives the first job, lq-b is admitted after job1 is deleted, and usage decays or grows on sampling ticks; usage ordering takes precedence over priority classes, which only break ties.
Speeding up LLM inference with P-EAGLE in vLLM Speculators
P-EAGLE (Parallel EAGLE), implemented in Speculators v0.6.0, extends EAGLE-3 with parallel drafting so a draft model predicts multiple future tokens in one forward pass rather than sequentially. Speculative decoding pairs this small draft model with a verifier model, which checks the proposed sequence in parallel and accepts only its correct prefix while preserving the verifier’s output distribution. The post adds Conditional Drop-token (COD) sampling, learnable mask embeddings and hidden states, and a custom flex-attention mask to support multi-depth prediction within the EAGLE-3 framework. The post presents an end-to-end workflow covering data preparation, vLLM hidden-state extraction, training and deployment, including a Qwen3-8B example that takes about 50 minutes on four H100s. A published Qwen3-a8B draft model reports an average length of 3.044 across the listed datasets.
Evaluating LLM guardrail configs locally with EvalHub
This installment evaluates local NeMo Guardrails configurations against large-scale risk datasets using EvalHub, following earlier manual testing. EvalHub provides a unified interface for AI evaluation, with provider integrations, reusable benchmarks, Hugging Face dataset inputs, and both local and Kubernetes deployment options. The benchmark configuration maps dataset prompt and label columns to block and pass labels, while balanced safe and risky examples prevent an always-block guardrail from appearing perfect. On the prompt-injection benchmark, the nuanced regex configuration achieved 51.1% accuracy with 13.7 ms median latency; DeBERTa reached 88.93% accuracy with 45.2 ms median latency. The conclusion favors layered guardrails: targeted regex checks can provide speed, with DeBERTa handling other cases, and EvalHub supplies reproducible measurements of the latency–efficacy trade-off.
Topics they write about
All topics →Weekly digest
The best engineering reads, once a week.