How Caddy Serves a Static Site: The Request Flow
See how Caddy handles a request for a static site, from DNS resolution through container routing to file_server, and how TLS and redirects fit into the flow.
Loading…
Topic
Docker
83 posts about Docker. Every summary links to the original.
- 83 posts
- 13 companies
- 3 explainers
See the idea
Docker explainers
Anka CI Architecture: From Layered Images to Disposable macOS Builds
Anka is a Docker-like macOS virtualization layer for disposable CI environments. This architecture shows how images, Mac hosts, snapshots, build agents, and artifacts connect.
How Alpine Linux Reduces Container Size
Alpine Linux shrinks container images by replacing glibc with musl and using a minimal base, but introduces compatibility trade-offs you must evaluate before switching.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Supabase Edge Runtime: Self-hosted Deno Functions
Supabase has open-sourced Edge Runtime, an MIT-licensed Rust web server built on Deno Runtime 1.32+ for self-hosting Deno Edge Functions. It serves TypeScript, JavaScript, and WASM functions, and existing Supabase Edge Functions can run without code changes; the release also adds npm modules and Node built-in APIs. The Supabase CLI uses it to serve all local functions with JWT verification, import maps, custom environment variables, hot reload, and defaults of 150 MB memory and 60 seconds execution time. Its architecture combines hyper, a Main Worker in a V8 isolate, and restricted User Workers that handle requests with controllable resource limits, while exposing APIs intended to improve development and deployment parity. A Fly.io demonstration shows deployment, with Digital Ocean and AWS also cited as provider options; the project is in beta with bugs and performance quirks still expected.
Supabase Beta February 2023
Supabase’s February 2023 beta update covers product changes across its dashboard, Auth, Postgres, Edge Functions, GraphQL, documentation, and community resources. GraphiQL is now built into the dashboard for exploring and querying pg_graphql APIs, while Docs search uses pgvector and OpenAI to support Supabase-specific questions. Supabase CLI 1.36+ can serve multiple Edge Functions simultaneously, and a multi-stage Postgres Dockerfile reduces the image from 1.3GB to 250MB, enabling faster boot time. The release also adds dashboard management for database roles, moves API documentation into the table editor with type-file generation, and provides a full OpenAPI 3.0 specification for Auth. Additional updates include inferred supabase-js response types, Deno 1.30.3 for Edge Functions, and Realtime behavior for deletes under RLS.
Building an In-Browser IDE the Hard Way
This tutorial builds a personal remote development environment: an in-browser IDE for Elixir/Phoenix work, deployed as a Fly Machine from a Dockerfile and managed with command-line tools. The image uses an official Elixir base, adds code-server, flyctl, build dependencies, and the Elixir Language Server extension, while a multi-stage build copies in Tired Proxy. A persistent volume holds working files, and the startup configuration can clone a repository, launch code-server behind HTTPS, and arrange for the machine to sleep after an hour without HTTP requests. From the IDE terminal, the walkthrough then creates and deploys a separate Phoenix hello-world application backed by SQLite, setting its secret and provisioning storage. The resulting setup can wake on browser access, but random HTTP traffic or an active code-server terminal may keep it running, and stopping or destroying the apps limits costs.
Spin Infrastructure Adventures: Containers, Systemd, and CGroups
Spin’s infrastructure team investigated instance instability after Kubernetes nodes experienced frequent failures, including roughly five nodes per day, and instances showed memory use that could exceed their assigned limits. Spin runs instances as Kubernetes pods, with systemd inside each container managing initialization and processes, and the team compared resource behavior across kubectl exec and Spin shell sessions. Stress tests showed that processes started through kubectl exec entered the kubepods cgroup hierarchy, while processes launched through Spin shell entered a hierarchy with an effectively unlimited memory limit, causing usage to appear on the node rather than the pod. Experiments comparing containerd, Docker, and Podman identified systemd cgroup delegation as the relevant difference: Podman isolated systemd and its child processes correctly, while the production setup used cgroupfs. The team identified switching COS to the systemd cgroup driver as the forthcoming fix, while noting that the bind-mounted cgroup filesystem enabled the escape.
Visualizing Supabase Data using Metabase
Organizations seeking to analyze database records can bypass code-heavy Python visualization libraries by connecting a Supabase backend directly to Metabase. Deploying the open-source Metabase Docker container exposes an initial setup interface on default port 3000, allowing administrators to input PostgreSQL connection credentials retrieved from Supabase database settings. Once connected, Metabase automatically surfaces table insights and automated x-rays across the public schema, exposing metrics such as inventory count ranges, price distributions, and column-specific statistics without requiring manual queries. For specialized analytics, users can execute custom SQL joins across relational tables like Product and Vendor to render tailored visual dashboard formats, including configurable bar charts mapped to designated axis fields.
Talk Nerdy to Me: 10 Tech Podcast Recommendations from Shopify Engineering
Shopify Engineering curated ten podcast recommendations from conversations with its technology employees, extending an earlier roundup of book recommendations. The list spans software engineering, front-end development, JavaScript, career growth, cybersecurity, data science, and broader technology topics, with selections ranging from quick, technical episodes to witty, conversational shows. Several entries emphasize specific formats or practices: Software Engineering Daily conducts researched interviews and focuses on open source, Soft Skills Engineering answers two listener-submitted questions in 30 minutes, and Darknet Diaries covers hacking, cybercrime, and data breaches. The recommendations also include podcasts valued for approachable explanations, collaborative software work, diverse guests, and perspectives on leadership and professional development, while some contributors identify particular episodes or hosts as reasons they keep listening.
Getting Started with Transformers on Habana Gaudi
Habana Labs and Hugging Face partnered to accelerate Transformer model training using Habana Gaudi accelerators, which offer up to 40% better price performance than recent GPU-based Amazon EC2 instances. Setting up the environment requires launching an Amazon EC2 dl1.24xlarge instance using the Habana Deep Learning Base AMI on Ubuntu 20.04. Inside an interactive PyTorch Docker container configured for Habana runtimes, users install the Optimum Habana package from source along with necessary dependencies. Fine-tuning the bert-large-uncased-whole-word-masking model on the GLUE MRPC task completes in 2 minutes and 12 seconds with an F1 score of 0.9181. Utilizing EC2 Spot Instances reduces the hourly compute cost by 70%, from $13.11 down to $3.93.
The Magic of Merlin: Shopify's New Machine Learning Platform
Shopify’s machine learning platform team redesigned its infrastructure to accommodate varied use cases, dependencies, data types, integrations, and production requirements. Merlin uses an open-source stack in which on-demand Merlin Workspaces run as short-lived Ray clusters on Kubernetes, while projects isolate code and dependencies in Docker-based environments. Its first iteration focuses on training and batch inference, with users moving from local or Jupyter-based prototypes to automated production workflows through the Merlin API, Airflow, or Oozie. Shopify validated the platform by migrating its product categorization workflows, using Ray Train for distributed TensorFlow training and Ray ActorPool for batch inference; planned extensions include online inference, model lifecycle management, and machine learning monitoring.
Case Study: Millisecond Latency using Hugging Face Infinity and modern CPUs
Deploying large Transformer models at scale often faces severe prediction latency bottlenecks, driving up infrastructure costs and limiting real-time production use cases. To address this challenge, Hugging Face evaluated Infinity, a containerized hardware-optimized inference solution paired with the Infinity Multiverse model optimization service. Testing covered 192 configurations on Amazon EC2 C6i instances powered by 3rd generation Intel Xeon Scalable processors across varying CPU cores, sequence lengths, and batch sizes. The benchmarks demonstrated that an Ice Lake-optimized DistilBERT container achieved up to 800% higher throughput than vanilla Transformers and delivered 1 to 4 millisecond end-to-end latencies for sequence lengths up to 64 tokens. Although Infinity was later discontinued in favor of Inference Endpoints and Optimum libraries, the results demonstrated substantial efficiency gains on modern CPU hardware.
Supabase Studio
Supabase has released Supabase Studio, making the web dashboard used on its hosted platform available for local development and self-hosted environments. The dashboard originally existed within a single internal monorepo alongside cloud infrastructure, but managing split repositories and code synchronization significantly slowed down production release cycles. To streamline development, the team unified the frontend into a single open-source codebase modeled after Sentry's shared architecture strategy. Built with Next.js, Tailwind CSS, Supabase UI, and MobX, the application works directly with existing deployments across Docker and the Supabase CLI. The open-source release focuses strictly on database management features, including SQL editors, role policies, and extensions, while excluding cloud project administration.
Docker without Docker
Fly.io explains how it accepts OCI container images while avoiding Docker at runtime, choosing Firecracker micro-VMs because Docker’s isolation is not strong enough for its high-density multitenant environment. It describes OCI images as ordered tarball layers, showing how registry authentication, manifests, architecture selection, blob downloads, and configuration data are enough to reconstruct a container filesystem and entrypoint. On the platform side, flyctl uses Docker’s client library to push images, while a custom Go repository server adds token-based authorization, repository rewriting, and handling for cross-repository blob mounts and registry state. Servers use containerd as a cache over an LVM2 thin pool, leases and copy-on-write snapshots to provision VM filesystems quickly, and a Rust init configures mounts, networking, DNS, SSH, and the entrypoint before Firecracker runs the workload.
Supabase Beta March 2021
Supabase announced a series of platform updates for March 2021, headlined by the launch of Supabase Storage and new local development capabilities. The new storage offering integrates an open-source Fastify and TypeScript API backed by S3, while direct database connections now bundle PgBouncer for connection pooling. Developers can run the platform locally via the Supabase CLI, utilize Docker tooling for self-hosting, and access the open-sourced Supabase UI component library. Community contributions brought OAuth scopes to Supabase Auth for requesting elevated third-party provider permissions, alongside newly published free and pro pricing tiers. In addition, development is underway on an Elixir-based workflow engine compatible with the Amazon States Language for automated event handling.
Supabase CLI
Supabase announces its CLI as the first step toward running Supabase locally, managing database migrations, generating types and API or validation schemas from a database, managing projects, and eventually pushing local changes to production. The local stack runs through Docker Compose, using tools described as 100% compatible with every Supabase project, and a React setup can initialize it with supabase init before supabase start. For schema changes, the post favors database diffing over accumulating manual SQL migrations and adds a CLI mode and Docker image for pgAdmin Schema Diff, which tracks view-owner changes needed for Row Level Security and handles domains. The supabase eject command outputs Docker Compose, Kong, and Postgres configuration for self-hosting. An update dated 15 August 2022 states that CLI V1 is generally available and that a Management API is in beta.
Launch week
Following its transition from Alpha to Beta, Supabase scheduled an intensive Launch Week to deliver multiple major features built over a three-month cycle. The event opened with a pricing model designed around predictable metrics like storage to prevent unexpected billing spikes from API requests. The team subsequently launched Supabase Storage, combining a Fastify Node.js API layer with Kong authentication and Postgres Row Level Security for policy evaluation. Additional launches delivered a CLI supporting database migrations and Docker self-hosting, alongside an open-source dashboard UI component library. Finally, Supabase added PgBouncer connection pooling for Postgres instances, previewed an Elixir workflow engine, and transitioned its primary domain to supabase.com.
My Journey to a serverless transformers pipeline on Google Cloud
The post documents Maxence Dominici’s effort to deploy a Transformers sentiment-analysis pipeline on Google Cloud for classifying customer reviews from Discord. After considering AI Platform Prediction and App Engine, he chose Cloud Run with a Docker image because custom checkpoint handling, dependencies, and resource requirements made the other options unsuitable. The implementation uses a PyTorch DistilBERT model fine-tuned on SST-2, Flask, Gunicorn, and a GET request protected by an optional API key; the shown handler retrieves a review but passes the hard-coded text “that was great” to the classifier. Cloud Run is configured with 4 GB of memory, one worker, and one thread to limit memory pressure and billing. Requests take under five seconds including model loading, while cold starts may add about ten seconds, and the author reports that the pretrained model’s accuracy was already impressive.
Supabase Beta February 2021
Supabase published its February 2021 beta update marking one year of development as an open-source Firebase alternative. The release introduces dashboard interface improvements across the Table view, Auth section, and SQL Editor, which now features autocomplete suggestions for tables and columns. Authentication updates allow user redirection to specific site routes upon invoking signIn and signUp, while a new database management interface lets users configure Postgres replication settings and events for the realtime API. Infrastructure additions include a new deployment region in South Africa and a Docker Compose configuration intended to power local development in the forthcoming CLI. Documentation expanded with resources for GoTrue and Google OAuth, alongside ongoing RFCs covering connection pooling and local workflows.
Keeping Developers Happy with a Fast CI
Shopify’s Test Infrastructure team tackled a 45-minute 95th-percentile CI time for its Core monolith, which had more than 170,000 tests and ran on every git push. They instrumented Buildkite commands and prioritized agent preparation, dependency building, and test execution using command frequency and duration. Disk I/O from writing large caches slowed Docker startup, so the team increased disk capacity and write speed, mounted most caches read-only, skipped unchanged database and asset work through hashes, parallelized setup, and expanded changed-file test mappings. These changes reduced container-start p95 from 90 to 25 seconds, raised builds avoiding the full suite from 45% to over 60%, improved test stability from 88% to 97%, and lowered overall CI p95 to 18 minutes.
One Small Step Closer to Containerising Service Binaries
Engineering teams at Grab initiated a transition to containerized microservices to standardize environments, enhance security, and decouple services from internal runtime tooling. During this migration, developers noticed that statically-linked Go service binaries were reaching bloated sizes over 100 MB. By analyzing the binaries using the open-source tool go-binsize-viz alongside the Go nm toolchain, the team visualized compiled symbols as interactive treemaps. This analysis revealed that 11 MB of unused message format symbols were being pulled in because a generic interface shared a directory with auto-generated streaming code. Restructuring the packages to isolate interfaces from generated code successfully decreased the binary size down to 78 MB.
Persistent Storage and Fast Remote Builds
Fly introduces persistent volumes for apps and a new remote-build path designed to make Docker image builds faster. Volumes are created through flyctl, backed by encrypted ext4 filesystems on Linux LVM2 thin pools, and attached one-to-one with app instances on compatible hosts. Volumes survive reboots and deployments and are described as performance-competitive with, usually slightly faster than, EBS, but raw-volume resilience remains the customer’s responsibility. Because volume count limits instance count and prevents an extra canary volume, volume-backed apps use rolling deployments; the post identifies replicated database clusters, caches, metrics, and restorable data as suitable cases. Remote builds now match jobs to automatically created builder apps with attached storage for cached layers and manifests; builders expose authenticated Docker servers, terminate after 10 minutes of inactivity, and are not restarted after successful exit.
Faster TensorFlow models in Hugging Face Transformers
Hugging Face updated Transformers to improve the computational performance and robustness of its TensorFlow models, including BERT, RoBERTa, ELECTRA, and MPNet. In version 4.2.0, the optimized BERT implementation runs up to approximately 10 percent faster than Google's official implementation and twice as fast as Transformers release 4.1.1 when tested on a V100 GPU. The release also improves TensorFlow SavedModel generation by allowing flexible sequence lengths across runs, exposing all model inputs, and grouping hidden states or attention outputs into single tensors. In addition, users can customize serving signatures by subclassing models and decorating custom methods with explicit input specifications. Models exported to the SavedModel format can be packaged into Docker containers and served via TensorFlow Serving using either HTTP REST or gRPC APIs for production inference.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.