Loading…
Topic
150 posts about CI/CD. Every summary links to the original.
See the idea
A five-step workflow for turning agent output into versioned artifacts: scope a repo per agent, branch instead of copying, run in an isolated sandbox, gate changes with push-triggered evals, then deploy or roll back.
Buildkite splits CI into a hosted control plane and agents you run on your own machines. This comparison weighs that self-hosted agent model against fully hosted runners so you can pick the right fleet.
Anka is a Docker-like macOS virtualization layer for disposable CI environments. This architecture shows how images, Mac hosts, snapshots, build agents, and artifacts connect.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Deploying AI coding assistants in regulated or sovereign environments presents risks around data privacy, hallucinated dependencies, and auditability. This tutorial outlines a self-hosted architecture deploying StarCoder2-7B as an NVIDIA NIM endpoint served entirely from local GPUs. NVIDIA NeMo Guardrails sits as an OpenAI-compatible proxy between developer IDEs and the model to intercept and reject requests for restricted or human-only code paths. Downstream CI gates catch package hallucinations and verify changes, while commit trailers and a Prometheus and Grafana metrics loop monitor defect escape rates. Because the model remains distinct from the external policy and validation layers, teams can incrementally adopt components or upgrade models without rearchitecting the pipeline.
Dependabot often floods repository maintainers with individual daily pull requests for single patch bumps, wasting review and continuous integration resources. Microsoft's GCToolkit project mitigated this issue by updating its dependabot.yml configuration to group dependency updates using wildcard patterns and slowing the schedule interval to monthly. The project also expanded coverage to include Maven alongside GitHub Actions so all relevant package ecosystems receive managed updates. Because Dependabot processes security alerts independently of regular version schedules and enforces a default three-day package cooldown, critical vulnerability fixes remain fast while routine maintenance noise drops significantly.
Traditional developer tools like Vim, Emacs, and IDEs earn trust through predictability, bounded roles, and muscle memory developed over time. In contrast, agentic coding tools operate across the entire software development lifecycle using ambiguous natural language, rapidly generating vast amounts of code that developers do not inherently trust. This influx shifts development bottlenecks from code authoring to code review, validation, and infrastructure cost management. Tooling additions such as automated code reviewers, context managers, and memory systems cannot repair broken software development lifecycles unless organizations actively adapt their culture and workflows. Teams can establish trust in agentic environments by providing explicit contextual specifications, reusing verified components to avoid duplicate implementations, and restricting non-deterministic artificial intelligence to appropriate problem domains.
Managing Infrastructure as Code across distributed repositories presents significant security and operational challenges. Pinterest developed the Resource Provisioner Pipeline (RPP), a proprietary Terraform execution engine, to securely deploy and manage AWS infrastructure across hundreds of workspaces. The pipeline utilizes centralized GitHub composite actions triggered by pull request events and enforces dual human approvals. To maintain least privilege, RPP uses GitHub OpenID Connect authentication to assume a central identity before validating repository paths and down-scoping to specific team execution roles. Automated checks include backend state validation, linting, plan generation, Semgrep static analysis, and optional LocalStack testing before deliberate comment-driven deployments execute.
Glaspoort replaced a sprawl of outdated, one-off BI reports with a custom application that combines Databricks analytical tools with Lakebase, a serverless Postgres OLTP database. The application keeps curated lakehouse data in a read-only schema while application state occupies a separate schema, and supports development, acceptance, and production environments. Its CI/CD pattern treats database branches like code: long-lived environments branch directly from production, each pull request receives an ephemeral production-shaped branch, migrations are replayed, and a staging app image runs the full test suite against it. This topology makes periodic resets cheap, avoids deleting dependent branches, and preserves fresh test data; the team chose stacked promotion for velocity, retaining a separate crisis pipeline for urgent fixes.
Recent industry updates focus on AI agent security, container maintenance, and developer workflows within Docker ecosystems. The OpenAI and Hugging Face incident involved 17,600 attacker actions, demonstrating that AI agent security requires automated system controls to constrain, observe, and govern agents rather than relying on human review. In software maintenance, Docker Extended Lifecycle Support (ELS) addresses escalating supply-chain risks and MinIO end of life by maintaining patches, building software from source, and enforcing local developer policies. For continuous integration, Docker Sandboxes enable isolated AI agents to run Testcontainers tests, implement code fixes, and submit draft pull requests within GitHub Actions. Additionally, Docker Verified Publisher applications have transitioned to a self-serve process accessible directly through Docker Hub.
The guide explains how to add a Heroku staging environment so changes are tested in a near-identical, isolated replica before production. It uses a pipeline with production and staging apps, disables production automatic deployments, connects staging to GitHub, and enables automatic builds, with Review Apps offered for pull requests. The setup applies Twelve-Factor practices by separating config vars and databases, recommending migrations into a fresh Postgres database or warning that pg:copy overwrites staging data and can create compliance risks. After validation, slug promotion sends the exact compiled staging asset to production without recompiling, while numbered releases support rollback; database changes must still be reversed separately.
Monolithic system prompts become difficult to reason about, test, and change as teams add safety policies, domain rules, formatting requirements, and escalation behaviors. The proposed approach treats prompts as software artifacts: modular skill files are composed with includes, variables, and macros, then transpiled into deterministic artifacts. Build-time validation checks missing imports, undefined variables, and circular dependencies, while dependency graphs and CI/CD golden-file comparisons catch drift before deployment. At runtime, progressive disclosure keeps a stable control plane loaded while agents retrieve task-specific skills, reducing context exhaustion and noise. Agents can propose new skill modules or import changes through pull requests, but human review and evaluation remain part of the merge process.
Slack developed Shipyard to modernize its EC2 infrastructure by replacing continuously updated, mutable instances with immutable deployable artifacts. The platform adopts a layered image architecture starting with a standardized base image called slack-zero, which is built using AWS Image Builder and consumed by downstream service-specific machine images. Rather than running periodic Chef jobs in the background, configuration management is restricted to image baking and initial instance provisioning phases. Fleet management is handled through the Peekaboo inventory system, metric-driven progressive rollouts via Gondola, and automated instance rotation managed by a lifecycle component known as the Reaper. Developers validate cookbook updates prior to merging using Ship Quick, which provisions real infrastructure across dedicated worker fleets to verify provisioning workflows.
An Android monorepo’s CI pipeline averaged about 35 minutes in its build phase, with a P90 of roughly 44 minutes, as repeated dependency downloads, environment-driven invalidations, variant bloat, and machine limits increased work. The team reworked caching with branch- and architecture-aware keys, release/develop fallbacks, cached SDK and dependency data, and a Gradle Remote Cache, while filtering repositories and restricting CI to required variants. It also tuned 32GB runners to a 16GB heap and 12GB Metaspace, disabled configuration-cache saves in ephemeral CI, upgraded Kotlin, AGP 8.10.1, and Gradle 8.11.1, and moved critical workflows to Apple Silicon. Gradle tasks fell from 3399 to 2745, while remote-cache hit rates reached P50 44% and P90 56%. After the Apple Silicon refresh, total CI time reached P50 10 and P90 11 minutes, average credits per build dropped from 55 to 43, and a comparable rollout window saved about 5800 credits.
Maintaining documentation across separate repositories often leads to severe lag because technical writers must reverse-engineer shipped features weeks after release. To address this in the Aspire project, the team implemented an automated pipeline using GitHub Agentic Workflows to bridge the product and documentation repositories. When product pull requests merge, a bash step maps milestones to docs release branches before an LLM agent evaluates the diff, drafts documentation updates, and emits structured pull request intents. A dedicated safe-outputs handler materializes these drafts via a scoped GitHub App and assigns the original code reviewers to verify accuracy. Across 396 product pull requests, the system generated 82 documentation pull requests that all merged with a median turnaround time of 44.8 hours.
huggingface_hub, the Python client underlying libraries such as transformers, datasets, diffusers, and sentence-transformers, moved from releases every four to six weeks to a weekly cadence. A single manually triggered GitHub Actions workflow now computes versions, manages branches, tags and publishes both huggingface_hub and the hf CLI, creates downstream RC test branches, and handles post-release tasks. An open-weights model drafts structured release notes and Slack announcements using PR metadata and documentation diffs, while OpenCode, HF Inference Providers, PyPI Trusted Publishing, and pinned, checksum-verified tooling provide the stack. Deterministic manifest validation checks missing or extra PR references, then a human reviews and publishes the drafts. The reported result is weekly shipping at about $0.25 per full release, with more consistent notes, earlier downstream breakage detection, and shorter contributor feedback loops.
DataOps is presented as a collaborative practice that applies DevOps principles—continuous integration, automated testing, and rapid delivery—to the full data lifecycle, from ingestion and transformation through trusted data-product delivery. It treats data as a continuously delivered product, with technical and non-technical participants sharing an operational cadence to improve quality and shorten time-to-insight. The recommended operating model standardizes source onboarding, enforces schemas at ingestion, uses idempotent jobs, version-controls transformations, and promotes them through tested staging and production environments, with the Bronze, Silver, and Gold layers serving as quality gates. Automated unit, data-contract, and regression tests, alongside observability, anomaly detection, SLAs, lineage, and documented rollback procedures, support reliable delivery and faster incident diagnosis. Organizations with mature DataOps practices report up to 99% fewer data-downtime incidents and 30–50% less reactive maintenance.
Grab is migrating from heavy base images to Distroless images to reduce security risks, despite runtime failures caused by missing shared objects, utilities, or expected directory structures. Its medium tests run services in Docker, leave internal dependencies unmocked through Testcontainers, mock external dependencies with a network proxy, and execute in CI to catch Dockerfile regressions before deployment. For services without this setup, an agentic workflow detects candidates, scaffolds tests, fixes repository-specific failures, raises GitLab merge requests, and monitors CI while processing batches in parallel. Claude Code uses MCP integrations for GitLab, Glean, and Sourcegraph, with guardrails limiting changes to test files and CI configuration and humans reviewing every draft merge request. In roughly 1.5 months, the agent raised more than 100 medium-test merge requests and reduced work for 400 services from 400–1,200 engineer-days to approximately 0.1 days per service; Distroless adoption also grew substantially.
Modern data pipelines move raw data from source systems through transformation and storage to downstream consumers, with batch, streaming, or hybrid operation selected according to latency needs. The guidance places service-level agreements first, defining acceptable latency, uptime, and error rates before architecture choices, and compares ETL, ELT, and zero-ETL patterns. It recommends immutable raw landing zones, CDC for transactional sources, schema registries, event replay, incremental loads, and idempotent writes using techniques such as MERGE operations. Performance and reliability practices include partitioning, clustering, data-quality checks, observability, automated CI/CD testing, and cost controls such as autoscaling and storage tiering. Adoption guidance advises piloting a representative pipeline, measuring telemetry and cost, then iterating through regular architecture reviews.
Sentry has released the beta version of Sentry Snapshots to catch visual regressions on pull requests across any platform with a frontend. The tool captures screenshots of an application across various viewports, themes, and languages, compares them against a baseline, and blocks pull requests when diffs are detected. Teams can attach custom context metadata, including test file paths, to help developers and AI agents locate test sources and generate additional test cases. By leveraging the sentry-mcp integration, REST API, and sentry-cli, workflows can retrieve base images, diff masks, and run local diff verification before pushing commits. Originally created at Emerge Tools for mobile applications, this iteration expands automated visual testing capabilities to broader frontend codebases.
Trackio's GitHub Actions CI was limited by generic hosted machines, maintenance-related delays, and lack of readily available GPU runners, while the project needed CPU checks and CUDA-based tests. The setup keeps GitHub Actions as the CI coordinator and uses a dispatcher Space plus the huggingface/jobs-actions bridge to launch ephemeral GitHub runners inside Hugging Face Jobs, selected through labels such as hf-jobs-cpu-upgrade and hf-jobs-t4-small. The guide covers duplicating and configuring the dispatcher, creating a GitHub App, supplying runner credentials and an HF token, changing runs-on, choosing Docker images, and validating runs and logs; Playwright and NVIDIA CUDA images are used for CPU and GPU examples. In Trackio, the CPU job took 1m10s versus a 1m40s GitHub ubuntu-latest baseline, while the GPU check passed in 45s and cost less than a cent at the t4-small rate; logs were available through both systems.
This installment revisits Evolutionary Database Design and argues that Databricks Lakebase removes the infrastructure constraints that kept several of its database-change practices aspirational. Lakebase is a managed Postgres database with compute separated from shared durable storage, while copy-on-write branches create a new pointer and divergence marker in roughly one second without copying parent data. That enables per-developer, per-PR, and per-experiment databases, real Postgres test branches instead of mocks or in-memory substitutes, and CI isolation at pull-request granularity. The updated playbook adds idempotent migrations, destructive testing, and database-level A/B prototyping, with Unity Catalog governance inherited by branches and agents receiving branches rather than production access. Jen’s example corrupts production-shaped data to test an inventory-code migration and compares column and lookup-table designs, rejecting the latter because its common read path requires a join.
Spotify experienced rapid codebase growth alongside widespread adoption of AI coding tools, resulting in a 76% increase in pull request frequency. To manage fleet-wide codebase maintenance and complex refactoring, the company combined its Fleetshift automation platform with Honk, a background coding agent powered by Claude. Honk runs inside Kubernetes pods with access to CI build environments and integrates with Backstage via Model Context Protocol tools to inspect component metadata. Standardized service architectures and active lint guardrails enable the agent to maintain consistency and auto-correct invalid design patterns during execution. By automating routine migrations, Spotify reduced fleet-wide Java backend updates from multi-week cross-team efforts down to three days handled by a single engineer.
The guide presents data warehouse design as a business-aligned process for building, modernizing, or scaling analytics infrastructure, rather than merely storing data. It recommends defining analytics use cases and stakeholder needs first, then organizing a three-tier architecture of source, storage, and semantic output layers; cloud designs can decouple compute and storage and use open formats. A Bronze–Silver–Gold medallion flow preserves raw lineage, applies cleansing and deduplication, and produces consumption-ready dimensional models, while retention and archival policies control sprawl. For modeling, it favors star schemas for user-facing BI, uses snowflake normalization when redundancy is material, and stresses explicit fact-table granularity, domain-owned data marts, and workload-specific refresh cadences. Governance and operations include Unity Catalog, access controls, masking, lineage, multi-region deployment, disaster recovery, and CI/CD, followed by phased rollout through high-value domains.
The best engineering reads, once a week.
