Loading…
Topic
144 posts about Kubernetes. Every summary links to the original.
See the idea
Bare metal gives engineers direct control over GPU, network, and virtualization layers, but it shifts elasticity and lifecycle work onto the team. This trade-off map shows when that bargain pays off.
Argo CD is a GitOps controller that treats Git as the desired state and continuously reconciles it against what actually runs in each cluster. It maps the parts of that loop and why pull-based agents change its scale.
Binary Authorization blocks container images that lack the attestations a cluster policy requires. This comparison sets a strict blocking posture beside an audit-only one, and shows how to choose between them.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Red Hat OpenShift topologies are mapped to edge-site constraints such as hardware footprint, availability, rack space, power, and workload needs. It is presented as a micro cloud that brings full Kubernetes functionality to server-class edge hardware, with optional virtualization and software-defined storage. Cluster capabilities can be minimized with baselineCapabilitySet: none and selectively enabled later, although added capabilities cannot be removed without reinstalling. Options span single-node OpenShift, which requires 4 vCPUs, 16 GB RAM, and 120 GB storage, two-node designs using fencing or an arbiter, and three-node compact clusters for resilient high availability. Each topology retains shared management and security practices, so selection depends on the physical footprint, failure tolerance, and scale required at the site.
AI infrastructure costs are often billed to a platform team as one lump sum, leaving Finance without a departmental breakdown and teams without feedback on their usage. The post presents a working demo on Red Hat OpenShift AI that attributes requests and token consumption through department-specific MaaS API keys, shared model servers, Prometheus metrics, Perses dashboards, and MLflow traces. In its example, three departments generate 79,420 inference requests and consume 634 million tokens against a $14,200 quarterly bill, while Marketing’s 32% GPU utilization reveals roughly $34,000 in annual waste. The system combines aggregate usage filters with request-level details such as prompts, token counts, latency, and calculated cost, and the post argues that showback helps identify inefficient allocations and model mismatches rather than police teams.
The guide explains how MLOps frameworks close the gap between notebook experimentation and reliable, scalable machine-learning production by applying automation, version control, and continuous delivery across the lifecycle. It describes experiment tracking, model registries, workflow orchestration, feature stores, deployment and serving, and monitoring as the capabilities needed to make training reproducible, prevent training-serving skew, automate pipelines, and detect degradation after release. MLflow is presented as a modular open-source option with Tracking, Model Registry, Models, and Projects, supporting run history, lifecycle stages, standardized packaging, and reproducible execution across environments. Kubeflow suits Kubernetes-standardized organizations and large distributed GPU workloads, while Metaflow, cloud-native services, and unified platforms such as Databricks are positioned as alternatives according to developer experience, cloud alignment, or operational preferences.
Equinix replaced its self-managed Kubernetes environment on Amazon EKS to address operational sprawl caused by independently managed clusters and inconsistent infrastructure practices. The new shared services architecture assigns platform ownership to a cloud operations team while application teams focus on business services. It uses separate Workloads, Platform, and Network accounts: workloads run across two Availability Zones with ALB, Gateway API, Cilium, and Hubble, while shared data services, GitHub Runners, and platform services reside in the Platform account. The Network account provides cross-account and multi-Region connectivity through AWS Transit Gateway, Route 53 Resolver, and Direct Connect. Equinix reports 40% lower operational overhead, 4x higher deployment frequency, and adoption of a unified architecture across multiple business organizations.
Red Hat Developer Hub brings OpenShift, Argo CD, Tekton, and Quay workflows into a unified developer portal, addressing context switching across separate consoles. The guide targets platform engineers and covers installing the Operator, creating GitHub and cluster credentials, configuring application and dynamic-plugin settings, defining RBAC policies, and deploying a Backstage custom resource. It uses a Kubernetes service-account token for access to pods, deployments, and pipeline runs, while Quay integration exposes image tags, vulnerabilities, and manifest details. After granting read access in monitored namespaces, the resulting portal can display pipeline runs, deployment health, image information, and Argo CD status in one view. The source says the setup is a proof-of-concept foundation, with a follow-up planned for software templates and developer self-service.
AI agents running on laptops or clusters may inherit broad filesystem, network, and credential access, making faulty permissions or prompt injection potential security risks. The guide uses Red Hat OpenShift to build layered controls around an OpenClaw agent: namespace isolation, default-deny network policy, ResourceQuota, and optional Kata Containers contain workloads and limit escape or resource abuse. SPIFFE and SPIRE issue rotating SVIDs based on workload attributes such as namespace and ServiceAccount, allowing services to verify an agent without shared secrets. Governance adds OPA Gatekeeper admission policies, an MCP Gateway with risk-tiered tool authorization, NeMo Guardrails for prompt and completion filtering, and MLflow traces for auditability. The recommended rollout starts with namespace isolation and quotas, then adds sandboxing, identity, and the more involved governance controls.
NVIDIA FLARE addresses the operational challenge of running federated learning across sites that use different infrastructure, including Docker, Kubernetes, and Slurm. Its two-layer architecture keeps long-running server and client parent processes available for authentication and coordination while launching transient job workers through each site’s native runtime. Jobs declare portable GPU, CPU, and host-memory requirements, which local launchers translate into Docker containers, Kubernetes pods, or Slurm allocations; studies add scoped membership and site-owned mappings for datasets, images, secrets, and scheduling policy. The result is a heterogeneous federation where organizations retain local resource authority, can allocate compute on demand, and run multiple studies without requiring one shared platform.
KubeVirt, VSOCK, and eBPF address different layers, so none replaces the others. KubeVirt adds virtual-machine lifecycle management to Kubernetes, running QEMU in a launcher Pod while KVM uses hardware virtualization features to execute guest code; it does not replace either component. VSOCK provides host–guest communication through a CID-and-port socket address, independently of guest IP networking, but attaching virtio-vsock does not define permitted applications or provide authentication and authorization. For host-enforced control, BPF LSM can use the socket_listen hook to deny unapproved host-side VSOCK listeners before creation, whereas ordinary XDP and TC filters on VM network interfaces do not see VSOCK traffic; this still leaves existing listeners, per-guest permissions, kernel attack surface, policy lifetime, and other virtual devices to address.
BoxCompute chose KubeVirt for its VM sandboxes because workloads need a real Linux machine with a guest kernel, systemd, and machine-level workflows that process-oriented environments do not fully provide. KubeVirt adds VM resource types and controllers to Kubernetes, while launcher Pods run QEMU/KVM and the guest on a worker. The arrangement keeps placement, lifecycle, permissions, storage, and cleanup within the Kubernetes model, avoiding a separate VM management platform, although resource accounting and readiness still require sandbox-specific work. The post stresses that full virtualization does not automatically provide security: the platform still needs deliberate controls for host access, guest traffic, credentials, persistence, and cleanup after failures.
Choosing an edge platform requires balancing hardware footprint, workload needs, high availability, and operational complexity; the options range from RHEL with Podman or KVM to OpenShift clusters. The guide organizes Red Hat's offerings into RHEL-based and OpenShift-based deployments, noting that both traditional virtual machines and modern containerized workloads can coexist and that platforms can overlap as requirements grow. It details image-mode RHEL (bootc), transactional updates with rollbacks, efficient over-the-air delivery, offline installation, and dependency pre-resolution for constrained or disconnected sites. RHEL is positioned for static workloads of one to five components, while Red Hat Device Edge adds MicroShift when lightweight Kubernetes is needed. Kubernetes becomes more appropriate for larger microservices or event-driven systems, automated recovery, dynamic workloads, cross-environment scaling, and rolling updates; the series next covers OpenShift topologies.
Pinterest describes building a VLM serving stack on NVIDIA Blackwell B200 GPUs and NVIDIA Dynamo for multimodal products including Pinterest Assistant, hybrid search, multimodal reranking, content understanding, signal generation, and content safety. Unlike text-only workloads, requests may contain thousands of images, costly and variable prefill, long multi-turn histories, and large KV caches, making preprocessing, routing, memory, and latency central serving concerns. The implementation combines Dynamo's distributed orchestration, multimodal frontend and vLLM support, KV-aware routing, disaggregated encoder/prefill/decode serving, and LMCache-based KV offloading with vLLM, an OpenAI Chat Completions interface, model-aware routing, EKS, and custom deployment components. Production-oriented AIPerf DAG benchmarks model QPS patterns, multi-turn conversations, many-image requests, and agentic DAGs; the team concludes that payload design, cache-aware routing, offload tiers, and a shared platform are essential to meeting product targets and accelerating new launches.
Together GPU Clusters is introducing preemptible compute in public preview for Kubernetes clusters in all regions, offering interruptible GPU capacity at a flat 50% of the on-demand rate. Preemptible nodes use the same NVIDIA accelerated compute, can be reclaimed when unused capacity is needed, and receive a maximum five-minute drain window for checkpointing and graceful exit. Clusters retain the requested preemptible target and automatically refill as capacity returns, while usage is metered every one to two minutes and billed sub-hourly. The feature suits resumable, retryable, or requeueable work such as experiments, batch inference, and temporary bursts, but critical services, uncheckpointed multi-day runs, and strict-SLO serving should remain on standard nodes; users must schedule eligible pods with the together.ai/compute-class=preemptible label.
Uber’s failover redesign exposed a limitation in its long-standing single-controller model for scaling stateless workloads. To reuse idle capacity during regional failover, the platform needed multiple orchestrators to influence replica counts without overloading UDC, which remained on the normal deployment lifecycle path. The solution introduced the ServiceScale CRD and Service Scale Controller, allowing separate actors to record steady-state and temporary failover intent in Kubernetes and letting SSC reconcile the combined decision. Production work then exposed risks from informer caches that lagged reality and near-simultaneous multi-writer updates, including ReplicaSet metadata-spec drift that disrupted proportional scaling and could leave workloads stuck. Uber added fleet-wide detection, an automated UDC healer, a long-term scaling-path fix, and extensive kind integration tests, staging, and canaries; the year-long rollout avoided customer-impacting outages while establishing a more extensible control plane.
Red Hat Advanced Cluster Security for Kubernetes exposes an image-scanning API through Central, but upgrades and restarts can interrupt CI/CD pipelines. The reference architecture uses two independent Central Service instances—primary and failover—with separate databases, scanner deployments, and namespaces, plus a client-side script that probes the primary’s health endpoint before each scan and falls back when needed. Operators install the failover instance, duplicate image registry integrations and build-time policies, configure CI authentication, and pause reconciliation on the failover Central custom resource while upgrading the primary, then resume it to upgrade the second instance. This avoids scan API unavailability during the staged upgrade process, but failover caches are cold, requiring temporarily longer scans and additional registry traffic. The instances do not synchronize databases, caches, configuration, or vulnerability updates, and the design does not provide general-purpose HA for secured cluster services or delegated scanning.
Red Hat build of Kueue 1.4 provides admission fair sharing for shared CPU, GPU, and high-performance computing clusters where FIFO admission can starve later workloads. The guide configures the feature through the Kueue custom resource, using either default values—30-minute usage half-life, five-minute sampling interval, and resource weights of 1—or custom settings, then creates a ResourceFlavor, a ClusterQueue using UsageBasedAdmissionFairSharing, and two LocalQueues. Kueue tracks each LocalQueue's historical consumption, adds a temporary penalty when a workload is admitted, and compares usage adjusted by queue weight to choose the lowest-use pending workload when quota returns. In the example, lq-a receives the first job, lq-b is admitted after job1 is deleted, and usage decays or grows on sampling ticks; usage ordering takes precedence over priority classes, which only break ties.
Federated data and AI platforms let users move among portals, datasets, notebooks, and assistants across clusters, but single sign-on alone does not carry trusted identity across control-plane and data-plane boundaries. The post presents a central identity gateway that owns platform sessions through OIDC, a shared session store, stateless data-plane gateways, and an identity-validation API. Regional gateways validate sessions through /gateway/userinfo, enforce local policy, and inject standardized identity headers instead of forwarding raw tokens or independently managing refresh and logout. This separates session ownership from request enforcement, supports consistent logout, and lets assistants pass delegated user identity to backend tools without broad service credentials. At NVIDIA, the approach reduced repeated login events by 55% across internal developer platforms spanning Kubernetes clusters in AWS and OCI, while establishing a foundation for unified platform shells and AI assistants.
The article describes how to extend Layer 2 networks from a legacy data-center fabric into Red Hat OpenShift Virtualization using BGP and EVPN. It uses a three-node OpenShift lab with secondary NICs on a provider network, static IPv4 addressing, a VTEP CR for 172.19.0.0/24, FRRouting peering, and a Layer 2 primary CUDN with VNI 100. The configuration preserves imported VMs’ IP and MAC identities while maintaining isolation for tenant networks that may have overlapping address spaces. In the example, an external FRR router learns the VM MAC through EVPN and reaches 22.100.0.3; after the VM moves from node 3 to node 1, the MAC remains unchanged and the advertised VTEP changes from 172.19.0.102 to 172.19.0.100. The stated result is phased migration with Layer 2 adjacency to legacy application tiers and end-to-end tenant segmentation.
Managing distributed on-premises infrastructure across hundreds of sites creates inconsistency, manual lifecycle bottlenecks, fragmented visibility, and scalability problems, especially when hardware, operating system, Kubernetes, and application operations span thousands of servers. The proposed design centralizes orchestration in AWS while keeping execution on site, using hybrid connectivity, AWS Lambda, AWS Step Functions, Amazon EventBridge, DynamoDB, AWS Systems Manager, and supporting services. Redfish APIs provide vendor-agnostic bare-metal control for BIOS, firmware, power, and health operations, while EKS Anywhere deploys and operates management and workload clusters on local hardware; orders and inventory records track workflows and resource relationships. DynamoDB Streams also automate DNS, certificates, AWS access, and storage integrations, while centralized observability aggregates server, cluster, and application signals; the post presents architectural patterns and defers detailed implementation workflows and templates to later parts.
This guide shows how to move a streaming retrieval-augmented generation (RAG) workflow from a monolithic Ray Data script to modular AI pipelines on OpenShift AI. It addresses failure recovery and run traceability by separating parsing and chunking, ingestion, model download, and deployment into five reusable components. Parsed and chunked JSONL is stored in S3-compatible storage so KFP can rerun ingestion without repeating document processing, and each run records parameters, timestamps, logs, and status. The data chain uses Docling’s HybridChunker and Ray map_batches with ActorPoolStrategy, while the model chain downloads a Hugging Face model and deploys it through vLLM and KServe in parallel. For embedding ingestion, the workflow supports local CPU processing with granite-embedding-125m-english or a dedicated GPU vLLM service for larger collections and offers notebooks for deployment and query validation.
Lyft migrated its fleet of hundreds of critical streaming jobs from an internally developed Flink Kubernetes operator to the open-source Apache Flink Kubernetes operator. The legacy system imposed maintenance burden, feature gaps, outdated dependencies, fragile savepoint handling, and a rigid memory model that complicated varied workloads, including Apache Beam applications. Lyft used an incremental migration designed to preserve deployment patterns and team workflows while addressing CRD translation, state-machine behavior, and operational compatibility. The new platform added explicit last-state upgrades, autoscaling, automatic memory tuning, and safer upgrade mechanisms, though autotuning required trade-offs with in-place autoscaling and node capacity planning. After the multi-quarter transition, maintenance burden was minimized, the autoscaler right-sized a fleet previously overprovisioned by a few million dollars annually, and Lyft could contribute fixes upstream instead of maintaining its own operator.
The best engineering reads, once a week.
