Loading…
Fly.io
Global application platform enabling developers to deploy and scale applications with edge computing capabilities.
- 199 posts summarised
- latest
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
From their blog
Latest articles
32 Bit Real Estate
Fly.io’s globally distributed network needs routable IPv4 addresses because it assigns distinct public addresses to applications, including non-HTTP services such as UDP DNS servers and TCP game servers. The post explains how address control depends on Regional Internet Registry transfers, a BGP4 ASN, upstream peering, and advertising the same address from multiple locations for Anycast routing. IPv6 and TLS SNI can reduce address demand, but IPv6 connectivity remains unreliable for many users and SNI does not cover non-TLS applications; keeping separate provisioning paths is also considered undesirable. IPv4 blocks are therefore bundled with apps, while outright acquisition is preferred to leasing because addresses can remain assigned indefinitely, despite future IPv6 obsolescence and the risk that their value could eventually fall to zero.
How We Got to LiveView
Chris McCord recounts how Phoenix and LiveView emerged from attempts to make real-time features as simple as Rails CRUD development. He first built sync.rb, which used WebSockets to re-render server-side templates and push HTML updates without requiring programmers to write JavaScript, but Ruby’s concurrency model and EventMachine proved unreliable for the task. He then turned to Elixir and the Erlang VM, whose lightweight message-passing processes and fault tolerance supported Phoenix Channels and a reported load test of 2 million concurrent WebSocket connections on one server. LiveView unifies client and server state in a stateful process that synchronizes only necessary HTML changes. The account also describes binary uploads, WebSocket-based navigation via live_redirect and live_patch, and HEEx, an HTML-aware Elixir template syntax with compile-time validation.
Fly.io Is Hiring Full-Stack Developers
Fly.io is seeking early-to-mid-level full-stack developers to work on customer-facing parts of its global application-hosting product. The role sits in a CLI-first organization whose UX stack combines Rails, GraphQL, and Go, with more Elixir and Phoenix planned, while the broader platform uses Rust and Go. Candidates are not expected to be Rails or Elixir experts; the stated bar is the ability to learn unfamiliar code and produce work that holds up, including deploying an application end to end and building browser UI components. Hiring centers on untimed work-sample challenges based on the team’s actual work rather than resume screening, interview gauntlets, or gotchas. The remote full-time role offers US healthcare, flexible vacation, hardware and phone allowances, and $120k–$140k plus equity.
API Tokens: A Tedious Survey
The post surveys API token schemes in the context of designing fine-grained permissions for Fly.io’s control-plane API, which currently provides all-or-nothing root credentials. It compares simple random tokens, platform tokens, OAuth 2.0, and other formats and protocols against revocation, database access, and vulnerability concerns, while distinguishing end-user API security from inter-service authentication and federated identity. The discussion explains how stateless signed tokens trade database lookups for cache-consistency and revocation problems, then presents Macaroons as a scheme the team plans to adopt, while noting that their expressiveness may be excessive for typical CRUD applications. Biscuits extend that model with public-key signatures and Protocol Buffers-encoded Datalog programs, offering unusually explicit authorization rules but demanding a potentially wholesale commitment to token-based policy. The conclusion favors simple random tokens for many CRUD applications, chooses Macaroons for the planned system, and advises against JWT.
Better Business Intelligence in Elixir with Livebook
The post argues that Elixir developers already perform business intelligence when they use iex or a Rails console to query application data, calculate figures, and produce CSV rows for managers. It presents Livebook as a more capable alternative for young projects that need analysis without adopting a large, continuously running BI application such as Metabase. Because notebooks execute Elixir code and can communicate with running applications through BEAM/OTP clustering, they can reuse business logic, Ecto schemas, associations, and external integrations such as Stripe while avoiding direct database access. Livebook also supplies charts, graphs, tables, Markdown-based sharing, and reproducible reruns, with WireGuard enabling secure connections to apps on Fly.io. The resulting notebooks can answer onboarding, sales, and integration questions while preserving work that may later support an admin dashboard.
FYI: Phoenix drops webpack and npm for esbuild
Phoenix is replacing its use of Node, npm, and webpack with esbuild, a Go-written JavaScript bundler, in the upcoming Phoenix 1.6 release. The change responds to support burden and reliability problems: the post says npm, webpack, or brunch were involved in roughly 30% of Phoenix’s reported issues, while node-sass and npm v7 had recently broken phx.new. esbuild compiles to native code, handles common asset-pipeline tasks, and is intended to provide newcomers with a seamless setup and long-term users with more reproducible JavaScript and CSS builds. Teams doing substantial frontend JavaScript can still use customized webpack or other tooling, while esbuild remains an opt-in path for complex client-side work. Existing projects can use Phoenix 1.6, although the post notes possible zombie processes and says migration snags remain to be assessed.
Last Mile Redis
Last Mile Redis presents a globally distributed caching design aimed at keeping backend responses under the 100-millisecond threshold by placing Redis near users. It argues that regional datasets overlap less than expected, making one Redis server per region a practical alternative to centralized sharding. The design combines standalone regional caches with a Dallas primary Redis and writable replicas configured with replica-read-only no, allowing local writes while replication distributes global changes. A DEL issued at the primary can therefore purge bad cache data across regions, while timestamped keys and expiration handle ordinary freshness; the same mechanism can also broadcast content or fan out queue messages.
FYI: Livebook 0.2 with Kino
Livebook 0.2 was released with Kino, a client-side-focused feature for animating data changes, alongside new input support and the data_table feature. Elixir cells can read input values with IO.gets("input name: "), although those values arrive as strings and may require parsing or other processing. data_table presents data in a table with pagination and sorting, while Livebook is positioned for teaching Elixir, documenting libraries, experimenting with Nerves and embedded hardware, and connecting to a live production system. The post also notes that ElixirConfEU and ElixirConf US 2021 were announced as hybrid or mixed-format events, with ElixirConfEU scheduled for September 9–10 and ElixirConf US for October 12–15 across physical and online sessions.
Globally Distributed Postgres
Globally distributed Postgres describes a way to serve a conventional CRUD application across regions while keeping Postgres writes directed to one primary and reads near users. Fly’s proxy sends each request to the local application instance, where most requests succeed; when a write reaches a read-only replica, Rails catches PG::ReadOnlySqlTransaction and returns a fly-replay header naming PRIMARY_REGION. The proxy then retries the complete HTTP request inside its network, avoiding per-query cross-region database routing and allowing the application to retain its ordinary connection behavior. The approach can also support session-based read-your-own-writes and replica-freshness checks, but it is not suitable for every workload; the text contrasts it with geographically distributed databases such as CockroachDB.
Monitoring Elixir Apps on Fly.io With Prometheus and PromEx
Monitoring Elixir applications on Fly.io, the post presents a way to add production visibility without administering a Prometheus instance. Fly.io’s managed service scrapes an application’s metrics endpoint at a configured interval, with fly.toml specifying the port and path; Prometheus stores the resulting time-series snapshots and Grafana queries them with PromQL. PromEx supplies Elixir instrumentation through plugins for components such as the application, BEAM, Phoenix, Ecto, and Phoenix LiveView, while optionally uploading dashboards and annotating application lifecycle events through Grafana’s HTTP API. The example enables Fly.io metrics on port 4000 at /metrics and configures PromEx dashboards for a Todo List application. The stated result is an instrumented application with dashboards and annotations in minutes, while monitoring can also support rollback decisions, stress-test analysis, prioritization, and capacity planning.
Observing Elixir in Production
Observer, included with Elixir and Erlang’s BEAM, is a WxWidgets interface that connects in real time to running nodes, showing process trees and state while allowing processes to be killed. This guide uses Fly.io to connect a local Observer to production nodes through a release cookie, a WireGuard tunnel, and a helper script. The example is a multi-region Tic-Tac-Toe application in Frankfurt and Los Angeles; a local IEx session joins the remote cluster, and Observer can connect to either node. Inspecting a live game exposes its GenServer state in Erlang representation. Killing its process immediately starts a replacement with a different PID and leaves other games running, but recovery still requires the player to reload the page or restart game joining.
Livebook is a secret weapon for documentation
The post presents Livebook as an interactive way to document the central logic of an Elixir or Phoenix application, helping new developers understand and explore code alongside existing documentation and tests. Using a TicTac example, it documents and executes the GameState module and struct, then describes and demonstrates a Horde-managed GenServer that runs the game state. Livebook connects to an existing project through a Mix standalone runtime, providing Elixir code completion, Markdown explanations, execution, and access to operations such as Ecto queries, context calls, and starting GenServers. The resulting notebooks are plain Markdown files, can be stored in source control, and support experimentation, reviewable diffs, and collaborative editing. The setup described requires Elixir 1.12: install Livebook with mix escript.install hex livebook, start livebook server, and connect through the locally generated token URL.
Hooking Up Fly Metrics
Fly apps include built-in Prometheus instrumentation, exposing automatically collected metrics through a public Prometheus endpoint and allowing applications to export custom metrics. The walkthrough connects an organization’s Fly Prometheus URL to Grafana with a Fly token, then explains dashboard variables and PromQL queries using counters, gauges, histograms, labels, irate(), rate(), increase(), sum(), and regional grouping. Examples cover network traffic, TCP connections, HTTP response rates by region, and response-time heatmaps built from histogram buckets ranging from 5ms to 120s. Application instrumentation is shown for Go, Python Flask, and Node.js, with scraping configured in fly.toml through a metrics port and path. The post concludes that metrics are currently free, with some tier free forever, although a future pricing plan is expected for ambitious usage.
Fly's Prometheus Metrics
Fly describes the Prometheus-style metrics infrastructure behind its globally distributed micro-VM platform and explains why it favors metrics over traditional monitoring checks. The system collects counters and other measurements through HTTP /metrics endpoints, stores them in a clustered VictoriaMetrics (“Vicky”) deployment, and uses Telegraf, Vector, exporters, and Consul to connect sources and manage discovery. Within Firecracker instances, custom init, the Nomad driver, and a vsock carry node-exporter-type JSON statistics from guests for collection, while applications can expose their own exporters. Deployments can declare an exporter port and /metrics path in fly.toml, after which Fly routes those application metrics into Vicky alongside platform data. The resulting Prometheus API is available for Grafana queries, dashboards, and alerts, with labels such as app, region, host, and instance; the post argues that the same historical data supports anomaly detection, trending, and planning.
Building a Distributed Turn-Based Game System in Elixir
Mark Ericksen describes building a proof-of-concept distributed, clustered game system for turn-based multiplayer applications with Elixir and Phoenix LiveView. Rather than separate JavaScript and API layers or external state systems such as Kafka and Redis, the design uses server-rendered HTML over websockets, Phoenix.PubSub, and Horde’s distributed process registry. A player’s LiveView sends moves to a GameServer, which publishes updated game state so subscribed LiveViews re-render for every connected player; Tic-Tac-Toe supplies the model. Hosted on Fly.io, geographically close web servers connect over a private WireGuard network while a game’s single GameServer remains the source of truth, with libcluster handling cluster formation and healing. The resulting open-source proof of concept supports multiplayer games, four-letter game codes, on-demand creation, responsive updates, and globally clustered servers.
Docker without Docker
Fly.io explains how it accepts OCI container images while avoiding Docker at runtime, choosing Firecracker micro-VMs because Docker’s isolation is not strong enough for its high-density multitenant environment. It describes OCI images as ordered tarball layers, showing how registry authentication, manifests, architecture selection, blob downloads, and configuration data are enough to reconstruct a container filesystem and entrypoint. On the platform side, flyctl uses Docker’s client library to push images, while a custom Go repository server adds token-based authorization, repository rewriting, and handling for cross-repository blob mounts and registry state. Servers use containerd as a cache over an LVM2 thin pool, leases and copy-on-write snapshots to provision VM filesystems quickly, and a Rust init configures mounts, networking, DNS, SSH, and the entrypoint before Firecracker runs the workload.
Low Latency LiveView
LiveView sends HTML fragments over a WebSocket connection, allowing Phoenix applications to update the DOM in response to server or user activity. For interactions such as a like button, however, a server round trip can make feedback feel slow: a sub-100ms sequence may be acceptable, while many interactions benefit from responses under 40ms. The post presents client-side state prediction as one remedy, where the browser displays an expected value immediately and later reconciles it with the server. Its preferred “boring” alternative is reducing round-trip time by running application processes near users in multiple cities, connecting Phoenix instances over a private network and enabling clustering. A seven-region Counter deployment is shown with fly regions set and fly scale count 7, while Phoenix.PubSub and Phoenix.Presence handle clustering transparently.
Anycast the easy way
Anycast lets an operator announce one IP block from multiple locations so routers connect users to a “nearest” location, but a basic setup requires address space and BGP announcements. The described path involves registering with a Regional Internet Registry, acquiring IPv4 through a broker or obtaining an IPv6 block, and checking purchased addresses for transfer eligibility and abuse-list history. It specifies minimum announcement sizes of an IPv4 /24 or IPv6 /48, notes a roughly $25-per-IPv4-address market rate, and estimates cloud holdings using published ranges. An ASN, available free from RIPE according to the text, groups prefixes and enables independent announcements, while BGP details, traffic direction, and failure scenarios are identified as harder areas left for later.
The 5-hour CDN
The post presents a five-hour path to building a functional content delivery network as a distributed caching proxy rather than a Google-scale infrastructure project. It defines the architecture as NGINX instances deployed across cities, with DNS rules or Anycast directing users to regions and hash-based load balancing assigning URLs to cache shards. Health checks and multi-location monitoring detect failed or slow regions, while peer caches, origin-access protection, and request coalescing address redundancy, security, and cache-miss spikes. The design can improve cache ratios and reduce origin load, but cross-region proxying can add latency, and distributed invalidation remains unresolved; app-specific workloads may be better served by distributing the application itself. The conclusion is that existing NGINX features and modest scripting make a simple CDN practical, while advanced capabilities such as image optimization, WAF, rate limiting, and bot detection remain possible extensions.
SSH and User-mode IP WireGuard
Fly.io’s SSH feature addresses the need to access applications running in Firecracker micro-VMs without requiring users to build containers with an SSH service or install WireGuard. The first part uses 6PN, a private IPv6-based network routed across Fly’s WireGuard mesh, and Hallpass, a small Go SSH server bound to each instance’s 6PN address. Authentication relies on an organization’s root SSH certificate published in private DNS, while the API signs short-lived user certificates. To support users without WireGuard installed, flyctl combines wireguard-go’s userland protocol with gVisor’s user-mode TCP/IP stack, allowing ordinary Go networking code and DNS resolution to operate over the private network. The resulting certificate-based SSH path is built into flyctl and reaches internal DNS names such as dogmatic-potato-342.internal, although the user-mode implementation is substantially slower than kernel TCP/IP.
Topics they write about
All topics →Weekly digest
The best engineering reads, once a week.