---
title: "Latest reads"
description: "The engineering internet, summarised so you can actually read it."
---

# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [Observing Elixir in Production](https://yomu.fyi/post/observing-elixir-in-production.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Jun 14, 2021
- Type: Tutorial

Observer, included with Elixir and Erlang’s BEAM, is a WxWidgets interface that connects in real time to running nodes, showing process trees and state while allowing processes to be killed. This guide uses Fly.io to connect a local Observer to production nodes through a release cookie, a WireGuard tunnel, and a helper script. The example is a multi-region Tic-Tac-Toe application in Frankfurt and Los Angeles; a local IEx session joins the remote cluster, and Observer can connect to either node. Inspecting a live game exposes its GenServer state in Erlang representation. Killing its process immediately starts a replacement with a different PID and leaves other games running, but recovery still requires the player to reload the page or restart game joining.


### [Deleting the Undeletable: Managing PII at Shopify - Shopify](https://yomu.fyi/post/deleting-the-undeletable-managing-pii-at-shopify-shopify.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jun 10, 2021
- Type: Problem & solution

Shopify describes how its Privacy and Data Science & Engineering teams redesigned analytical event handling to make personal-information deletion workable at Shopify’s scale. The effort addressed a lack of guaranteed event schemas by introducing a human-readable JSON schematization system covering field types, version evolution, ownership, data controllers and subjects, and privacy instructions; more than 4,500 active schemas now govern events. Before warehouse storage, identifying data is either obfuscated into generalized values, such as masked IP addresses with city or country information, or tokenized into consistent random values whose mappings reside in a secured vault and vary by controller and subject. Removing a vault mapping makes warehouse copies of the token non-detokenizable, allowing deletion without warehouse operations; the production platform processes about 20 billion events daily, while tens to hundreds of millions of mappings are deleted daily.


### [Updating Illustrations at Scale](https://yomu.fyi/post/updating-illustrations-at-scale.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jun 3, 2021
- Type: Problem & solution

Shopify’s Polaris team needed to update an illustration style across a product whose assets had become fragmented across more than 6,000 repositories. Illustrators and developers could not reliably identify an image’s repository, code references, reuse, or file type, while GitHub API limits and search behavior prevented dependable large-scale discovery. An initial Node.js monorepo cloned repositories, used globby to find PNG, SVG, and WebP files, searched source lines, and served JSON results through a Preact interface, but rebuilding 24 repositories took more than 35 minutes and produced inaccurate data. The team replaced it with @shopify/get-repo-images, a Go executable that parallelizes file operations, bundles crawling and site generation, and exposes a searchable Next.js interface through NPM. Search and build time fell to eight minutes, with repository cloning as the remaining bottleneck, and the tool was open-sourced.


### [Few-shot learning in practice: GPT-Neo and the 🤗 Accelerated Inference API](https://yomu.fyi/post/few-shot-learning-in-practice-gpt-neo-and-the-accelerated-inference-ap.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Philipp Schmid
- Published: Jun 3, 2021
- Type: Tutorial

Limited labeled data often prevents machine learning models from achieving high performance during traditional fine-tuning. Few-shot learning addresses this hurdle in natural language processing by supplying a short task description and a few demonstration examples at inference time. EleutherAI developed GPT-Neo, an open transformer-based model trained on the Pile dataset, which can execute few-shot tasks via the Hugging Face Accelerated Inference API. Compared to larger models like GPT-3, the 2.7B parameter version of GPT-Neo requires three to four examples alongside tuned hyperparameters like temperature and stopping sequences to generate accurate completions. Because few-shot learning relies heavily on pre-trained associations, careful output monitoring and human feedback mechanisms remain necessary to prevent automated bias.


### [Supabase Beta May 2021](https://yomu.fyi/post/supabase-beta-may-2021.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Jun 2, 2021
- Type: Announcement

Supabase published its May 2021 beta updates covering authentication, regional infrastructure, and tooling. Developers can now implement Apple and Twitter authentication via Supabase Auth, addressing App Store requirements and expanding third-party login options. The platform introduced a Policy Editor with templates to configure Row Level Security for Storage, alongside CSV data retrieval capabilities. Infrastructure expanded with a new Tokyo region to support developers in Japan and China, while Supabase Grid was publicly released as part of an effort to open-source the dashboard. Additionally, community contributors advanced client libraries, completing postgrest-go and progressing development on Swift libraries including gotrue-swift, realtime-swift, and storage-swift.


### [Supabase Beta June 2021](https://yomu.fyi/post/supabase-beta-june-2021.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Jun 2, 2021
- Type: Announcement

Supabase announced a series of platform updates, documentation additions, and tooling enhancements for June 2021 ahead of its scheduled Launch Week. Developers can now provision a Postgres database directly through a new Vercel integration and enable authentication using Discord logins. Supabase Storage added support for upsert operations alongside public buckets that allow unauthenticated file access. The web dashboard introduced a restart button for debugging and a table policy editor with templates to configure Row Level Security. Additional backend improvements include timestamps for PostgREST error logs, an OpenAPI mode patch ignoring anonymous privileges for Prisma users, and an XSS fix in the Auth0 Next.js library.


### [Shipit! Presents: How We Write React Native Apps](https://yomu.fyi/post/shipit-presents-how-we-write-react-native-apps.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: May 27, 2021
- Type: Announcement

A May 19, 2021 Shipit! event brought together Colin Gray, Haris Mahmood, Guil Varandas, and Michelle Fernandez, developers setting Shopify’s React Native standards, to discuss writing performant apps. The Q&A covers accessibility and design guidance through Restyle and Polaris documentation; their responsive layouts avoid fixed sizing where possible and use breakpoints for fonts and spacing. They describe React Native state as the usual source of truth while native modules manage internal state and send updates through EventEmitter, with SQLite in Point of Sale as an exception that still has one source of truth. Jest handles unit and integration tests, Buildkite manages CI builds, nightly builds are automatically distributed to testers, and failing feature-branch tests block merges to main.


### [Livebook is a secret weapon for documentation](https://yomu.fyi/post/livebook-is-a-secret-weapon-for-documentation.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: May 26, 2021
- Type: Tutorial

The post presents Livebook as an interactive way to document the central logic of an Elixir or Phoenix application, helping new developers understand and explore code alongside existing documentation and tests. Using a TicTac example, it documents and executes the GameState module and struct, then describes and demonstrates a Horde-managed GenServer that runs the game state. Livebook connects to an existing project through a Mix standalone runtime, providing Elixir code completion, Markdown explanations, execution, and access to operations such as Ecto queries, context calls, and starting GenServers. The resulting notebooks are plain Markdown files, can be stored in source control, and support experimentation, reviewable diffs, and collaborative editing. The setup described requires Elixir 1.12: install Livebook with mix escript.install hex livebook, start livebook server, and connect through the locally generated token URL.


### [How Shopify Built An In-Context Analytics Experience](https://yomu.fyi/post/how-shopify-built-an-in-context-analytics-experience.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: May 25, 2021
- Type: Explainer

Shopify’s Analytics section gave merchants access to business data, but research and quantitative analysis found that they often moved between it and admin pages while making decisions. The team therefore designed an in-context experience for the Orders page, selecting metrics through workflow analysis, segmenting merchants by factors such as order volume and fulfillment process, and examining each metric’s usefulness, variability, and time horizon. Data scientists prototyped visualizations with real, segmented data and created a specification sheet so Web, Android, and iOS used consistent queries, data sources, fields, and aggregations; the project also tested batch models before streaming them. The launched experience surfaces order and fulfillment metrics for daily, seven-day, and thirty-day periods, with comparison indicators, and links to reports covering orders, returns, products, and fulfillment, shipping, and delivery times.


### [Using & Mixing Hugging Face Models with Gradio 2.0](https://yomu.fyi/post/using-mixing-hugging-face-models-with-gradio-2-0.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Abubakar Abid
- Published: May 25, 2021
- Type: Announcement

The Hugging Face Model Hub hosts more than 10,000 user-submitted machine learning models spanning natural language processing, image classification, and audio processing. Machine learning developers increasingly need to demo models to interdisciplinary collaborators or non-programmers to uncover biases and failure points. The Gradio 2.0 library enables users to create graphical user interfaces and load almost any Hugging Face model using a single line of code. By default, these interfaces query Hugging Face's hosted Inference API without requiring an API key, though users can also run computations locally via the transformers library. Gradio interfaces can be customized through the Interface class and composed to load multiple models in parallel or chain them in series.


### [Elixir at Ramp 🧪](https://yomu.fyi/post/elixir-at-ramp.md)
- Company: [Ramp](https://yomu.fyi/company/ramp.md)
- Author: Pablo Meier
- Published: May 24, 2021
- Type: Explainer

Ramp engineer Pablo describes why Elixir suited several critical services, while arguing that technology choice is only one part of engineering success. He contrasts BEAM, Erlang’s virtual machine, and OTP’s abstractions for reliable, concurrent systems with the adoption and developer-experience advantages of Elixir, including familiar syntax, Mix, iex, hot code reloading, doctests, macros, and sigils. Immutability and aliased module calls are presented as safeguards against hidden state changes and circular-import problems in growing codebases, though the post also notes that Erlang and Elixir lag behind bolted-on type checkers. Ramp uses Elixir for its real-time Authorization engine and Risk analysis platform; the Authorization service is described as exceptionally stable and flexible for rapid feature development. The company’s largest backend remains Python, and the conclusion prioritizes sleep, trust, ownership, and psychological safety over any specific stack.


### [Building a Hyper Self-Service, Distributed Tracing and Feedback System for Rule & Machine Learning (ML) Predictions](https://yomu.fyi/post/building-a-hyper-self-service-distributed-tracing-and-feedback-system.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Warren Zhou
- Published: May 24, 2021
- Type: Problem & solution

Grab's Trust, Identity, Safety, and Security team processes billions of daily rule and machine learning decisions for fraud detection, safety, and identity checks. Earlier logging approaches using plain text Kibana logs and the ActionTrace library lacked structured formats, dynamic entity customization, and fine-grained access controls. To resolve these limitations, the team built Archivist, a centralized tracing, statistics, and feedback system. Archivist ingests events through an SDK into Kafka streams, buffers and routes data into Elasticsearch indices and Amazon S3, and provides a role-based user portal. The platform handles 80 million daily logs across roughly 50 business scenarios, reducing scenario onboarding times from days to minutes.


### [Other Driven Developments](https://yomu.fyi/post/other-driven-developments.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: May 21, 2021
- Type: Explainer

The article argues that development models turn implicit lessons into shared, conscious practices as teams, codebases, and responsibilities grow. It presents Grep Driven Development, emphasizing expressive and consistent names so behavior remains searchable, refactorable, abstractable, and predictable, while warning that premature abstraction and locally “improved” naming can hinder later change. It also makes the case for Copy/Paste Driven Development as a deliberate, judgment-based tool, noting that duplication can be cheaper than the wrong abstraction and that abstractions with flags and conditionals may create their own costs. The article names Ego Driven Development and Stickler Driven Development among its four approaches, then concludes with Human Driven Development: choosing, bending, or breaking rules to make work easier for present and future teammates.


### [Hooking Up Fly Metrics](https://yomu.fyi/post/hooking-up-fly-metrics.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: May 20, 2021
- Type: Tutorial

Fly apps include built-in Prometheus instrumentation, exposing automatically collected metrics through a public Prometheus endpoint and allowing applications to export custom metrics. The walkthrough connects an organization’s Fly Prometheus URL to Grafana with a Fly token, then explains dashboard variables and PromQL queries using counters, gauges, histograms, labels, irate(), rate(), increase(), sum(), and regional grouping. Examples cover network traffic, TCP connections, HTTP response rates by region, and response-time heatmaps built from histogram buckets ranging from 5ms to 120s. Application instrumentation is shown for Go, Python Flask, and Node.js, with scraping configured in fly.toml through a metrics port and path. The post concludes that metrics are currently free, with some tier free forever, although a future pricing plan is expected for ambitious usage.


### [Fly's Prometheus Metrics](https://yomu.fyi/post/fly-s-prometheus-metrics.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: May 13, 2021
- Type: Explainer

Fly describes the Prometheus-style metrics infrastructure behind its globally distributed micro-VM platform and explains why it favors metrics over traditional monitoring checks. The system collects counters and other measurements through HTTP /metrics endpoints, stores them in a clustered VictoriaMetrics (“Vicky”) deployment, and uses Telegraf, Vector, exporters, and Consul to connect sources and manage discovery. Within Firecracker instances, custom init, the Nomad driver, and a vsock carry node-exporter-type JSON statistics from guests for collection, while applications can expose their own exporters. Deployments can declare an exporter port and /metrics path in fly.toml, after which Fly routes those application metrics into Vicky alongside platform data. The resulting Prometheus API is available for Grafana queries, dashboards, and alerts, with labels such as app, region, host, and instance; the post argues that the same historical data supports anomaly detection, trending, and planning.


### [Three Ways We Share Context at Shopify Engineering](https://yomu.fyi/post/three-ways-we-share-context-at-shopify-engineering.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: May 13, 2021
- Type: Explainer

Shopify Engineering treats context—the why, what, and how behind development philosophies, approaches, and choices—as essential to company, team, and individual growth. It describes the Development Handbook, which evolved from a 2015 Google Doc into a searchable internal website in 2018, with hundreds of developer-contributed topics maintained through GitHub pull requests, reviews, and Markdown. Dev Talks began as a weekly open stage and became an on-demand recording program after the 2020 shift to Digital by Design, with most presentations lasting 15 to 20 minutes. The Technical Leadership Team’s Shift podcast provides monthly, in-depth discussions of technical decisions, distributed in audio and video with transcripts. The programs continue to evolve while facing content-maintenance and participation challenges.


### [Our Journey to Continuous Delivery at Grab (Part 2)](https://yomu.fyi/post/our-journey-to-continuous-delivery-at-grab-part-2.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Sylvain Bougerel
- Published: May 10, 2021
- Type: Problem & solution

Conveyor, an in-house continuous delivery system at Grab, introduces hermetic deployments by tracking application code alongside static and dynamic configuration parameters. This hermeticity guarantees that production releases use combinations of versioned artifacts previously verified in staging, preventing rollback incompatibilities. Conveyor replaces single multi-environment pipelines with decoupled pipelines, while automating cluster locking, deployment slot scheduling, release note generation, and canary monitoring with automated rollbacks. These automations reduced production deployment failure rates from 1.5% to an average of 0.3% over a three-month period. Grab also doubled the volume of production changes between 2018 and 2020 while saving more than 5,000 man-days of engineering effort in 2020.


### [How I Define My Boundaries to Prevent Burnout](https://yomu.fyi/post/how-i-define-my-boundaries-to-prevent-burnout.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: May 6, 2021
- Type: Explainer

The post describes how an intense, single-tasking software leader developed a sustainable way to divide work, family, and recovery time. After 100-hour startup weeks caused scattered thinking, dulled analytical ability, wrist problems, and an inability to type, the author found that a focused 60-hour, 9-to-9 schedule delivered 80 to 90% productivity compared with 100 hours, then shifted to a 9-to-6 model in 2009. The approach separates concentrated work from non-work, schedules blocks for writing, planning, reviewing, and thinking, and uses email and Slack response patterns to communicate boundaries and expectations. It lets lower-priority tasks fall away instead of extending work indefinitely into personal hours. The author says this structure supports recovery, family presence, and prioritization, while emphasizing that each person should experiment with a sustainable model rather than treat nine-to-six as universal.


### [Supabase Beta April 2021](https://yomu.fyi/post/supabase-beta-april-2021.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: May 5, 2021
- Type: Announcement

Supabase focused on platform stability, security enhancements, and community support during its April 2021 beta update cycle. To address visual accessibility needs for developers with astigmatisms or those operating in brightly lit environments, the team delivered Light Mode within the dashboard settings. The release also introduced OpenAPI-powered Storage API documentation, community translations for the main repository spanning nineteen distinct languages, and an experimental synchronization server connecting Postgres with Stripe. Dashboard components are systematically migrating into an open-source UI library, establishing necessary foundations for local development tooling and self-hosted environments. Additionally, the team reported substantial contributor growth across its realtime server and UI repositories while continuing engineering on a Workflows engine aimed at a July release.


### [Building a Distributed Turn-Based Game System in Elixir](https://yomu.fyi/post/building-a-distributed-turn-based-game-system-in-elixir.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Apr 29, 2021
- Type: Tutorial

Mark Ericksen describes building a proof-of-concept distributed, clustered game system for turn-based multiplayer applications with Elixir and Phoenix LiveView. Rather than separate JavaScript and API layers or external state systems such as Kafka and Redis, the design uses server-rendered HTML over websockets, Phoenix.PubSub, and Horde’s distributed process registry. A player’s LiveView sends moves to a GameServer, which publishes updated game state so subscribed LiveViews re-render for every connected player; Tic-Tac-Toe supplies the model. Hosted on Fly.io, geographically close web servers connect over a private WireGuard network while a game’s single GameServer remains the source of truth, with libcluster handling cluster formation and healing. The resulting open-source proof of concept supports multiplayer games, four-letter game codes, on-demand creation, responsive updates, and globally clustered servers.


[Newer posts](https://yomu.fyi/posts/page/221.md) · [Older posts](https://yomu.fyi/posts/page/223.md)