How to Partition AMD GPUs for Shared AI Compute
A five-step how-to for deciding when an AMD Instinct GPU should stay dedicated and when SR-IOV virtual functions, VMs, or sidecar isolation can safely share it.
Loading…
Topic
OpenShift
21 posts about OpenShift. Every summary links to the original.
- 21 posts
- 1 company
- 2 explainers
See the idea
OpenShift explainers
Argo CD: The Reconciliation Loop Between Git and Your Cluster
Argo CD is a GitOps controller that treats Git as the desired state and continuously reconciles it against what actually runs in each cluster. It maps the parts of that loop and why pull-based agents change its scale.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Scale OpenShift Dev Spaces in air-gapped environments
Scaling OpenShift Dev Spaces in air-gapped environments requires addressing four friction points: third-party monitoring agents, rootless container runtimes, custom install namespaces, and admission webhooks. Dynatrace OneAgent sidecar conflicts are resolved by labeling all Dev Spaces namespaces with dynatrace.com/inject: “false” to prevent init container injection. Rootless Podman inside workspace pods needs user namespace UID/GID ranges respecting a 65,536 UID ceiling; UDI 3.22 requires a manual script with a corrected range, while UDI 3.26 handles mapping automatically. Stale admission webhooks from a previous operator install namespace must be manually deleted before reinstalling into a different namespace to avoid conflicting definitions. A readiness checklist includes mirroring operator catalogs, deploying an internal Open VSX server with PostgreSQL, configuring PKI trust via ConfigMap, and disabling the default devfile registry.
Red Hat OpenShift Virtualization memory management overview
The post overviews recent memory management evolutions in Red Hat OpenShift Virtualization, spanning from the Linux kernel to the orchestration planes. Historically, the platform uses a 1:1 memory request mapping, but data shows most VMs use only about 20% of allocated RAM, creating a delta that scales infrastructure with workload definitions rather than demand. Node-level optimizations include free page reporting (FPR), which asynchronously recovers pages freed by the guest OS without forcing allocation like ballooning, and KSM zero pages for guest-agnostic deduplication of zero-filled memory. At the cluster level, Multi-Gen LRU integrated with control groups enables swapping to fast secondary NVMe storage, and a PSI-aware descheduler proactively migrates VMs to relieve memory pressure across nodes. OpenShift Virtualization 4.22 provides an autopilot feature that automates configuration for generally available memory optimizations, while developer preview capabilities require manual opt-in activation by default.
Sharding etcd on hosted control planes and what the numbers actually say
Sharding etcd on hosted control planes splits the key-value store into multiple instances by resource kind, a tech preview feature tested with a main shard for cluster state and a dedicated events shard on Red Hat OpenShift 4.21 with HyperShift. Under event pressure from synthetic storms and realistic crashlooping pods, the sharded configuration showed up to 96% fewer slow applies, faster compaction and raft snapshot durations, and 60% more WAL fsyncs and 90% more backend commits combined. The sharded arm never failed the event storm workload, while the standard arm occasionally did. Pod-ready latencies and workload finish times remained comparable across both configurations. The trade-offs are roughly 1.8 to 2.3 times the etcd container memory and a compaction regression under large custom resource quota pressure that is still under investigation.
How IdeaBot drives AI innovation workflows on OpenShift
Red Hat’s emerging technologies team built IdeaBot, an AI-enabled SaaS platform on OpenShift that automates innovation workflows from initial concept to a standardized idea report. The platform guides users through four stages—discovery, research, refinement, and synthesis—using conversational interviews, automated research across internal and external sources, a curation board with confidence scoring, and a synthesis view that compiles deliverables like executive summaries and market analysis. Security enforces session isolation in dedicated pods, credential isolation through an auth proxy and token service that injects short-lived tokens, and stateless compute with PostgreSQL persistence and a warm pod pool to reduce cold starts. Development uses EARS syntax and Gherkin feature files to drive behavior-driven development, enabling automated test generation and reproducible builds. Red Hat plans to extend IdeaBot toward fully autonomous, reproducible agentic SDLC pipelines using factory patterns.
Why your non-root container dropped its capabilities (and how to fix it)
A platform engineer traces why a non-root OpenShift container configured with CAP_NET_RAW still fails ping with Operation not permitted. The root cause is that SCC is only an admission-time gate that does not enforce runtime privileges. CRI-O translates the PodSpec into an OCI spec with the capability in the Bounding, Effective, and Permitted sets but deliberately omits ambient capabilities. During the runtime’s UID transition from root to the target non-root user, the kernel clears both Permitted and Effective sets, leaving an empty CapEff. The only mechanism that survives a UID transition for non-root processes is ambient capabilities, which CRI-O intentionally avoids to preserve the Kubernetes assumption that switching to a non-root user drops capabilities. To resolve this, file capabilities can be applied directly to the binary via setcap, avoiding the need for ambient capabilities.
Kube AuthKit: Unified Kubernetes and OpenShift auth in Python
Kube AuthKit is a lightweight Python library that unifies Kubernetes and OpenShift authentication behind a single API, eliminating conditional code paths across local, pod, notebook, and CI/CD environments. It supports four strategies—kubeconfig, in-cluster, OIDC, and OpenShift OAuth—auto-detected by an AuthFactory that probes environment variables and in-cluster markers in a defined order. The library handles token lifecycle transparently, with in-memory storage by default and optional OS keyring persistence for local development to avoid repeated logins. It provides two OIDC flows: device code for headless environments and authorization code with PKCE for interactive applications, both with automatic token refresh. Kube AuthKit returns a standard Kubernetes Python client ApiClient, ensuring full compatibility with the official library, and is actively maintained by the Open Data Hub community for OpenShift AI environments.
Smarter GPU sharing: How Red Hat build of Kueue works with dynamic resource allocation
The Kubernetes device plugin model treats GPUs as anonymous integers, making fair sharing impossible because a full 80 GB GPU and a 5 GB partition both consume ‘1’ from the same quota. Dynamic resource allocation (DRA), generally available in Kubernetes 1.34, replaces this by having drivers advertise real device attributes like memory and compute capability, but DRA alone does not address quota management or preemption. Red Hat build of Kueue 1.4 on Red Hat OpenShift integrates with DRA to enforce quota, borrowing, and preemption for both whole GPUs and partitionable devices using device-count or counter-based memory accounting. For partitionable GPUs, Kueue reads counter data from DRA drivers and charges GPU memory against ClusterQueue quota instead of device count, enabling accurate fair sharing between training and inference workloads. Future developments include extended resources for simpler adoption without ResourceClaimTemplates, consumable capacity for dynamic time-slicing, and scheduler-library integration to verify scheduling feasibility.
GPU virtualization at scale: AMD MI300X SR-IOV on Red Hat OpenStack Services on OpenShift
Enterprise AI environments are moving from dedicated GPU servers to shared accelerator infrastructure, requiring higher GPU utilization with strong isolation. AMD Instinct MI300X GPUs support hardware-assisted SR-IOV in CPX mode, each exposing eight virtual functions with separate PCI and VRAM allocation. When integrated with Red Hat OpenStack Services on OpenShift, the control plane runs on OpenShift while RHEL compute nodes use EDPM, with Nova scheduling VFs via VFIO PCI passthrough. Validation on an 8-GPU system created 64 VFs, demonstrated 64 VMs with one VF each, and tested up to 32 VFs per VM. BabelStream benchmarks showed approximately 96-99% of bare-metal memory bandwidth, while the testing also identified limitations in GPU peer-to-peer DMA and runtime VF lifecycle.
Upgrading to Red Hat JBoss Web Server 7: Key changes & impacts
Red Hat JBoss Web Server 7, based on Apache Tomcat 11 and Jakarta EE 11, requires Java 17 as the new minimum and drops support for Java 11. Jakarta EE 11 does not require another import rename but adds a RecordELResolver for records and an OptionalELResolver for optionals. Tomcat 11 changes parameter parsing to throw exceptions on malformed requests, reduces maxParameterCount to 1,000 from 10,000, removes FailedRequestFilter and HTTP/2 push, and retains quotes in cookie values per RFC 6265. The Java SecurityManager is removed, UBI 8 images are discontinued for UBI 9, Prometheus is no longer bundled, and Red Hat recommends container isolation as replacement. Teams must confirm Java 17 readiness, check for SecurityManager usage, update their base image pipeline, and note that Spring Boot 4 on JWS 7 is unsupported.
Managing edge solutions with Red Hat: A layered approach
Managing edge infrastructure at scale across hundreds of locations demands consistent management rather than disparate tools. Red Hat’s layered approach organizes tools into three compartments: a universal foundation, OS-level fleet management for RHEL, and Kubernetes lifecycle management for OpenShift. The universal layer includes Ansible for agentless automation, Quay for container image registry with geo-replication, Lightspeed for proactive drift detection and compliance, and Trusted Software Supply Chain for end-to-end security. For RHEL-based edges, Edge Manager provides SaaS fleet management with OTA updates while Satellite offers on-premise content management for air-gapped environments. For OpenShift-based edges, Advanced Cluster Management handles cluster lifecycle and zero-touch provisioning while Advanced Cluster Security enforces container security across the fleet.
Build OpenJDK container images locally using the standalone S2I tool
Red Hat provides the Red Hat build of OpenJDK for running Java applications on Red Hat OpenShift 4. Developers can use the standalone Source-to-Image (S2I) CLI to build and test OpenJDK container images locally before deploying to a cluster. S2I produces reproducible images by injecting source code into a builder image and running its assemble script. The toolkit supports flags such as –loglevel for detailed output, –environment-file to pass custom JVM settings like G1GC and MaxRAMPercentage, and –as-dockerfile to output a Dockerfile. After building, the image can be tested locally with docker run, then uploaded to the cluster for deployment.
Comparison of Cluster Autoscaler and KEDA MachineSet Autoscaling on Red Hat OpenShift
This article compares two autoscaling approaches for Red Hat OpenShift compute infrastructure: the built-in Cluster Autoscaler and the KEDA-driven MachineSet Autoscaler. The key difference is the trigger mechanism: Cluster Autoscaler reacts to Pending pods, while KEDA scales based on external metrics like PromQL queries. KEDA triggers faster, scaling within about 15 minutes versus 24 minutes for Cluster Autoscaler, because it acts on metric thresholds before pods become unschedulable. Cluster Autoscaler offers built-in scale-down safety and zero configuration, while KEDA provides fine-grained, metric-driven control but requires the Custom Metrics Autoscaler operator. The choice depends on workload predictability, with Cluster Autoscaler suited for general-purpose workloads and KEDA for dedicated pools with predictable metric patterns. The two autoscalers cannot coexist on the same MachineSet, as they would fight over replica counts.
Scaling localnet user-defined networks (CUDNs) in Red Hat OpenShift to 3,500 networks
Migrating from legacy virtualization with flat VLAN-based tenant networks to Red Hat OpenShift Virtualization often requires re-architecting network topology, creating friction. Red Hat OpenShift supports localnet user-defined networks (CUDNs) to preserve existing VLAN infrastructure while modernizing. Red Hat tested localnet CUDNs at scale, successfully managing up to 3,500 localnet CUDNs hosting over 3,500 active virtual machines. OVN-Kubernetes achieved 100% SSH connectivity, near-line-speed data-path performance over 100GbE interfaces without Geneve tunnel overhead, and stable control plane resource usage. Resource peaks reached approximately 5.5 CPU cores per worker node and 10 CPU cores on control plane nodes during bulk creation, then returned to low baselines.
Red Hat OpenShift networking: Default OVN-Kubernetes vs. Cilium operator
The article compares Red Hat OpenShift’s default OVN-Kubernetes CNI with the Cilium operator for teams managing microservices, AI workloads, and declarative multi-cluster infrastructure with Cluster API (CAPI). It describes OVN-Kubernetes as natively managed by the OpenShift Cluster Network Operator, with Open vSwitch foundations, hardware offloading, Multus and OpenShift Virtualization compatibility, and OVN Interconnect for large-scale deployments. Cilium uses kernel-level eBPF programs and offers identity-based policies, sidecarless service mesh capabilities, and Hubble Layer 7 observability, while CAPI manifests and MachineSets support declarative worker-node provisioning. The source recommends Cilium for new clusters when eBPF capabilities and CAPI integration justify added operational complexity, while positioning OVN-Kubernetes as the simpler choice for integrated lifecycle management and hardware acceleration.
Matching OpenShift edge topologies to your physical footprint
Red Hat OpenShift topologies are mapped to edge-site constraints such as hardware footprint, availability, rack space, power, and workload needs. It is presented as a micro cloud that brings full Kubernetes functionality to server-class edge hardware, with optional virtualization and software-defined storage. Cluster capabilities can be minimized with baselineCapabilitySet: none and selectively enabled later, although added capabilities cannot be removed without reinstalling. Options span single-node OpenShift, which requires 4 vCPUs, 16 GB RAM, and 120 GB storage, two-node designs using fencing or an arbiter, and three-node compact clusters for resilient high availability. Each topology retains shared management and security practices, so selection depends on the physical footprint, failure tolerance, and scale required at the site.
Build a unified CI/CD control plane with Red Hat Developer Hub
Red Hat Developer Hub brings OpenShift, Argo CD, Tekton, and Quay workflows into a unified developer portal, addressing context switching across separate consoles. The guide targets platform engineers and covers installing the Operator, creating GitHub and cluster credentials, configuring application and dynamic-plugin settings, defining RBAC policies, and deploying a Backstage custom resource. It uses a Kubernetes service-account token for access to pods, deployments, and pipeline runs, while Quay integration exposes image tags, vulnerabilities, and manifest details. After granting read access in monitored namespaces, the resulting portal can display pipeline runs, deployment health, image information, and Argo CD status in one view. The source says the setup is a proof-of-concept foundation, with a follow-up planned for software templates and developer self-service.
Reference architecture for HA scanning with Red Hat Advanced Cluster Security for Kubernetes
Red Hat Advanced Cluster Security for Kubernetes exposes an image-scanning API through Central, but upgrades and restarts can interrupt CI/CD pipelines. The reference architecture uses two independent Central Service instances—primary and failover—with separate databases, scanner deployments, and namespaces, plus a client-side script that probes the primary’s health endpoint before each scan and falls back when needed. Operators install the failover instance, duplicate image registry integrations and build-time policies, configure CI authentication, and pause reconciliation on the failover Central custom resource while upgrading the primary, then resume it to upgrade the second instance. This avoids scan API unavailability during the staged upgrade process, but failover caches are cold, requiring temporarily longer scans and additional registry traffic. The instances do not synchronize databases, caches, configuration, or vulnerability updates, and the design does not provide general-purpose HA for secured cluster services or delegated scanning.
Configure admission fair sharing in Red Hat build of Kueue 1.4
Red Hat build of Kueue 1.4 provides admission fair sharing for shared CPU, GPU, and high-performance computing clusters where FIFO admission can starve later workloads. The guide configures the feature through the Kueue custom resource, using either default values—30-minute usage half-life, five-minute sampling interval, and resource weights of 1—or custom settings, then creates a ResourceFlavor, a ClusterQueue using UsageBasedAdmissionFairSharing, and two LocalQueues. Kueue tracks each LocalQueue's historical consumption, adds a temporary penalty when a workload is admitted, and compares usage adjusted by queue weight to choose the lowest-use pending workload when quota returns. In the example, lq-a receives the first job, lq-b is admitted after job1 is deleted, and usage decays or grows on sampling ticks; usage ordering takes precedence over priority classes, which only break ties.
Extend Layer 2 networks into Red Hat OpenShift Virtualization with BGP and EVPN
The article describes how to extend Layer 2 networks from a legacy data-center fabric into Red Hat OpenShift Virtualization using BGP and EVPN. It uses a three-node OpenShift lab with secondary NICs on a provider network, static IPv4 addressing, a VTEP CR for 172.19.0.0/24, FRRouting peering, and a Layer 2 primary CUDN with VNI 100. The configuration preserves imported VMs’ IP and MAC identities while maintaining isolation for tenant networks that may have overlapping address spaces. In the example, an external FRR router learns the VM MAC through EVPN and reaches 22.100.0.3; after the VM moves from node 3 to node 1, the MAC remains unchanged and the advertised VTEP changes from 172.19.0.102 to 172.19.0.100. The stated result is phased migration with Layer 2 adjacency to legacy application tiers and end-to-end tenant segmentation.
Red Hat OpenShift autoscaling using MachineSet autoscaler with KEDA
The demonstration compares OpenShift MachineSet Autoscaler with KEDA against the previously used Cluster Autoscaler, using metrics to scale a MachineSet before pods remain pending. After installing Red Hat’s Custom Metrics Autoscaler operator (KEDA 2.19), granting permissions for MachineSet scaling and Thanos/Prometheus access, it creates a ScaledObject with a Prometheus trigger targeting demo-p4p95-worker-eastus3. Because a MachineSet has no pods, the HPA path cannot use metricType: Value, so AverageValue is paired with a PromQL query that multiplies CPU request utilization by the current node count. With nine one-CPU test pods, the MachineSet scaled from one to two to three nodes as the metric crossed 75%, and all pods reached Running; two additional Standard_D4s_v3 VMs were provisioned in zone 3.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.