--- title: "Argo CD: The Reconciliation Loop Between Git and Your Cluster" description: "Argo CD is a GitOps controller that treats Git as the desired state and continuously reconciles it against what actually runs in each cluster. It maps the parts of that loop and why pull-based agents change its scale." --- # Argo CD: The Reconciliation Loop Between Git and Your Cluster **Type:** [What is it?](https://yomu.fyi/visual-explainers/category/what-is-it.md) ![Diagram defining Argo CD: a Git repository of desired manifests on the left, a central control plane in the middle, and managed clusters on the right, joined by a numbered reconciliation loop showing diff, sync, and health status.](https://yomufyi.s3.us-east-1.amazonaws.com/visual-explainers/20d08712-672c-48b9-a989-9078cbdc756a/image-080031d9-7c3b-4c82-9d67-a0c4436e0eba.webp) > Argo CD is a GitOps controller that treats Git as the desired state and continuously reconciles it against what actually runs in each cluster. It maps the parts of that loop and why pull-based agents change its scale. ## Explanation Argo CD is a GitOps controller. A Git repository holds the desired state as manifests, the cluster holds the observed state, and a reconciliation loop continuously compares the two and syncs the difference. In the classic model a central hub reaches into every cluster; in the pull model a lightweight agent on each cluster fetches its own configuration, which removes inbound administrative ports and cuts cross-cluster traffic. Understanding those parts, desired state, observed state, the diff, and who initiates the connection, is what makes scaling, security, and failure behavior predictable. ## FAQ ### What actually breaks first when Argo CD scales out? The hub's Kubernetes API limits tend to bind first, followed by informer processing slowdowns that cause watch restarts and event writer retries under load. ApplicationSet controller RAM consumption is another observed pressure point. These showed up in a 42-machine, 500-cluster test, and the findings are feeding back into future OpenShift GitOps releases. ### Does a Synced status mean the deployment is actually healthy? Sync tells you the cluster matches Git, not that the running system is correct, so verification is a separate layer. Readiness checks that run real workloads and attribute failures to specific nodes, and disruption-aware node updates that respect budgets and drain before applying changes, are the kinds of checks that close that gap. Treat sync as the start of validation rather than the end of it. **Tags:** [Architecture](https://yomu.fyi/topic/architecture.md), [Argo CD](https://yomu.fyi/topic/argocd.md), [Deployment](https://yomu.fyi/topic/deployment.md), [GitOps](https://yomu.fyi/topic/gitops.md), [Kubernetes](https://yomu.fyi/topic/kubernetes.md), [OpenShift](https://yomu.fyi/topic/openshift.md) ## Related reading ### [Scaling GitOps: The "pull" architecture for global fleets](https://yomu.fyi/post/scaling-gitops-the-pull-architecture-for-global-fleets.md) - Company: [Red Hat](https://yomu.fyi/company/red-hat.md) - Author: Jann Fischer, Pradeep Surisetty, Sho Weimer, Jan Hutař, Alex Krzos, Mike Ng - Published: Sep 24, 2026 - Type: Benchmark Red Hat conducted a performance and scale test of the argocd-agent pull architecture for GitOps, using 42 physical machines, 500 spoke clusters, and software including OpenShift 4.21 and OpenShift GitOps 1.20. The pull model decentralizes reconciliation to lightweight agents on each managed cluster, which pull configurations from a central hub, improving security by eliminating inbound administrative ports and reducing cross-cluster network traffic by up to 80%. In the bootstrap scenario, generating and syncing 31,200 applications across 480 clusters completed in 13 minutes 3 seconds, while the sync-to-commit scenario, which includes a polling delay, took 13 minutes. The test revealed bottlenecks including Kubernetes API limits on the hub, informer processing slowdowns causing watch restarts, event writer retry issues under load, and ApplicationSet controller RAM consumption. The team also deleted all 31,200 applications in 4 minutes 53 seconds using cascading deletion, and the insights are informing future OpenShift GitOps releases. ### [Validate GPU Cluster Readiness Before AI Workloads Land](https://yomu.fyi/post/validate-gpu-cluster-readiness-before-ai-workloads-land.md) - Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md) - Author: Michelle Horton - Published: Sep 23, 2026 - Type: Explainer GPU clusters can pass standard health checks yet fail under real AI training workloads due to a single degraded GPU, network link, or misrouted traffic. NVIDIA Cluster Readiness Engine (NVCRE) is an open source Kubernetes controller that runs real distributed workloads across topology-aware node groups before production jobs land, measuring results and reporting which nodes failed each test. Its layered API uses Custom Resource Definitions for Certification, Workflow, and Job resources, attributing each failure to a specific node and category such as NCCL bandwidth or training goodput. The WorkloadRun API handles platform detection, NCCL settings, shared-memory volumes, and gang scheduling, so operators do not need to write those manifests by hand. NVCRE integrates with NVSentinel for passive telemetry and is available under Apache 2.0, with a roadmap for inference and automated lifecycle validation. ### [Manage Kubernetes Node Fleets with NodeWright](https://yomu.fyi/post/manage-kubernetes-node-fleets-with-nodewright.md) - Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md) - Author: Michelle Horton - Published: Sep 23, 2026 - Type: Problem & solution Managing Kubernetes node fleets, especially GPU nodes, is difficult because host-level changes can disrupt long-running training jobs, and traditional tools are not cluster-aware. NodeWright, an open-source Kubernetes-native package manager from NVIDIA, solves this by declaratively configuring and updating node operating systems while respecting PodDisruptionBudgets and non-interruptible workload labels. The operator orchestrates a sequence of cordon, wait, drain, apply, interrupt, and uncordon on each node, using container image packages that carry modifications such as kernel parameters, security agents, and verification scripts. It supports progressive rollouts across a fleet via DeploymentPolicy compartments with configurable disruption budgets and strategies like batch, canary, and rolling. NodeWright integrates with NVIDIA AI Cluster Runtime for validated configurations and is part of the DSX OS open-source portfolio, available under Apache 2.0. ### [Build OpenJDK container images locally using the standalone S2I tool](https://yomu.fyi/post/build-openjdk-container-images-locally-using-the-standalone-s2i-tool.md) - Company: [Red Hat](https://yomu.fyi/company/red-hat.md) - Author: Francisco De Melo Junior - Published: Sep 23, 2026 - Type: Tutorial Red Hat provides the Red Hat build of OpenJDK for running Java applications on Red Hat OpenShift 4. Developers can use the standalone Source-to-Image (S2I) CLI to build and test OpenJDK container images locally before deploying to a cluster. S2I produces reproducible images by injecting source code into a builder image and running its assemble script. The toolkit supports flags such as –loglevel for detailed output, –environment-file to pass custom JVM settings like G1GC and MaxRAMPercentage, and –as-dockerfile to output a Dockerfile. After building, the image can be tested locally with docker run, then uploaded to the cluster for deployment. ### [Rollouts and Security Review](https://yomu.fyi/post/rollouts-and-security-review.md) - Company: [Cursor](https://yomu.fyi/company/cursor.md) - Published: Sep 23, 2026 - Type: Announcement Cursor launched two bots for the last mile of shipping code: Rollouts and Security Review. Rollouts watches every change as it deploys and reports health per environment, with verdicts of verified healthy, regression detected, or inconclusive. It reads the diff and systems a pull request touches, writes a monitoring plan as a PR comment, and tracks deploy events per environment. Security Review reads every pull request in the context of the codebase and posts one review comment reporting exploitable bugs, including injection, authentication bypasses, and secrets. Both bots are available today on Teams and Enterprise plans, with usage credits for the next 10 days.