---
title: "Latest reads"
description: "The engineering internet, summarised so you can actually read it."
---

# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [How I Define My Boundaries to Prevent Burnout](https://yomu.fyi/post/how-i-define-my-boundaries-to-prevent-burnout.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: May 6, 2021
- Type: Explainer

The post describes how an intense, single-tasking software leader developed a sustainable way to divide work, family, and recovery time. After 100-hour startup weeks caused scattered thinking, dulled analytical ability, wrist problems, and an inability to type, the author found that a focused 60-hour, 9-to-9 schedule delivered 80 to 90% productivity compared with 100 hours, then shifted to a 9-to-6 model in 2009. The approach separates concentrated work from non-work, schedules blocks for writing, planning, reviewing, and thinking, and uses email and Slack response patterns to communicate boundaries and expectations. It lets lower-priority tasks fall away instead of extending work indefinitely into personal hours. The author says this structure supports recovery, family presence, and prioritization, while emphasizing that each person should experiment with a sustainable model rather than treat nine-to-six as universal.


### [Supabase Beta April 2021](https://yomu.fyi/post/supabase-beta-april-2021.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: May 5, 2021
- Type: Announcement

Supabase focused on platform stability, security enhancements, and community support during its April 2021 beta update cycle. To address visual accessibility needs for developers with astigmatisms or those operating in brightly lit environments, the team delivered Light Mode within the dashboard settings. The release also introduced OpenAPI-powered Storage API documentation, community translations for the main repository spanning nineteen distinct languages, and an experimental synchronization server connecting Postgres with Stripe. Dashboard components are systematically migrating into an open-source UI library, establishing necessary foundations for local development tooling and self-hosted environments. Additionally, the team reported substantial contributor growth across its realtime server and UI repositories while continuing engineering on a Workflows engine aimed at a July release.


### [Building a Distributed Turn-Based Game System in Elixir](https://yomu.fyi/post/building-a-distributed-turn-based-game-system-in-elixir.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Apr 29, 2021
- Type: Tutorial

Mark Ericksen describes building a proof-of-concept distributed, clustered game system for turn-based multiplayer applications with Elixir and Phoenix LiveView. Rather than separate JavaScript and API layers or external state systems such as Kafka and Redis, the design uses server-rendered HTML over websockets, Phoenix.PubSub, and Horde’s distributed process registry. A player’s LiveView sends moves to a GameServer, which publishes updated game state so subscribed LiveViews re-render for every connected player; Tic-Tac-Toe supplies the model. Hosted on Fly.io, geographically close web servers connect over a private WireGuard network while a game’s single GameServer remains the source of truth, with libcluster handling cluster formation and healing. The resulting open-source proof of concept supports multiplayer games, four-letter game codes, on-demand creation, responsive updates, and globally clustered servers.


### [A Five-Step Guide for Conducting Exploratory Data Analysis](https://yomu.fyi/post/a-five-step-guide-for-conducting-exploratory-data-analysis.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 28, 2021
- Type: Tutorial

Exploratory data analysis (EDA) is presented as a practical way to describe a dataset, uncover patterns such as missing data, correlations, and outliers, and generate hypotheses that inform business questions and later data products. The guide recommends first clarifying the intended task with leadership and product, then checking dataset size, sample rows, and the unique row identifier; its synthetic merchant-behaviour example has 1,500 samples, 13 features, and one row per shop per day. It then advises examining missingness by feature, investigating why values are absent before deleting or imputing them, and classifying features as continuous, discrete, or categorical to guide analysis. Visualizing distributions and relationships helps reveal changes across samples and time, while box plots and percentile thresholds support systematic outlier review. The process is framed as a starting point whose findings can support regression, classification, or KPI-dashboard decisions.


### [Dynamic ProxySQL Query Rules](https://yomu.fyi/post/dynamic-proxysql-query-rules.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 20, 2021
- Type: Problem & solution

Shopify uses ProxySQL to manage application connections to MySQL databases, but applying query rules across thousands of ProxySQL instances is difficult, especially during incidents. ProxySQL rules can reroute, rewrite, or reject regex-matched queries, yet incorrect patterns may cause unintended rejection, rewriting, data corruption, or downtime. To make rule changes safer, Shopify added dry-run support to a ProxySQL fork, using the mysql\_query\_rules table’s active and log fields; dry-run rules log matched queries without executing their actions. A sidecar polls a global key-value store, synchronizes dynamic rules to each ProxySQL instance, caches rules, supports optional TTLs, and uses exponential backoff for broad logging patterns. The system favors consistency during synchronization failures, while static startup rules cover requirements for full availability.


### [Scaling-up BERT Inference on CPU (Part 1)](https://yomu.fyi/post/scaling-up-bert-inference-on-cpu-part-1.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Morgan Funtowicz
- Published: Apr 20, 2021
- Type: Benchmark

Scaling BERT-like model inference on CPUs presents practical deployment challenges as transformer architectures proliferate in production environments. Using a reproducible Hydra-based benchmarking framework, inference workloads were evaluated on an Amazon Web Services c5.metal instance featuring an Intel Xeon Platinum 8275 processor with AVX512 and VNNI support. Out-of-the-box evaluations showed PyTorch 1.8.1 delivering lower latency than Google TensorFlow 2.4.0, likely due to PyTorch relying on OpenMP and Intel MKL/oneDNN rather than Eigen. Implementing multiple inference streams by pinning parallel model instances to dedicated CPU cores yielded near-linear throughput scaling. Matching instance core allocations to specific problem sizes also demonstrated that smaller workloads can achieve optimal latency on fewer cores, enabling up to a sixfold reduction in cloud compute costs.


### [How We Improved Agent Chat Efficiency with Machine Learning](https://yomu.fyi/post/how-we-improved-agent-chat-efficiency-with-machine-learning.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Suman Anand
- Published: Apr 19, 2021
- Type: Problem & solution

Agent typing time represented a large portion of Grab's chat support journey, and 85% of messages were still free typed because agents customized static templates to fit their personal style. To accelerate typing across multilingual markets without robotic templates, Grab built SmartChat, a machine learning feature that provides contextual sentence completion. The team opted for a lightweight seq2seq architecture using single-layered GRU encoder-decoders in TensorFlow instead of bulky attention models to keep model latency under 100ms. The user interface was implemented in React using a content-editable div with inline typeahead suggestions activated via keyboard shortcuts.


### [Management of Native Code and React Native at Shopify](https://yomu.fyi/post/management-of-native-code-and-react-native-at-shopify.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 16, 2021
- Type: Problem & solution

Shopify’s Point of Sale team describes how its React Native strategy changed after limitations appeared in complex UI and background synchronization. The team kept complicated views in React Native, improving rendering with why-did-you-render, React DevTools Profiler, Application Performance Index measurements, memoization, lazy renders, and Reanimated 2 for drag-and-drop interactions. For background jobs, it built a native job manager with Kotlin Multiplatform Modules, sharing synchronization logic in common/ while isolating platform-specific file, database, networking, preferences, and threading code through libraries or wrappers and exposing it through React Native native modules. The resulting implementation shared approximately 95% of its code and reduced a medium-sized store’s initial sync from at least 30 seconds to 2–3 seconds, while the article notes KMM’s learning curve and recommends a separate library project for clearer boundaries and isolated sandbox testing.


### [Introducing 🤗 Accelerate](https://yomu.fyi/post/introducing-accelerate.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Sylvain Gugger
- Published: Apr 16, 2021
- Type: Announcement

🤗 Accelerate enables PyTorch developers to execute raw training loops across CPUs, multi-GPU setups, and TPUs with minimal boilerplate modifications. Standard distributed training typically demands manual device assignments, DistributedSampler configurations, and divergent code paths that break single-device portability. By wrapping models, optimizers, and dataloaders through a unified prepare call, Accelerate automatically manages hardware placement, mixed precision operations, and sampler batch slicing without requiring custom distributed samplers. Distributed evaluation is streamlined using a gather utility that consolidates prediction tensors across active worker processes. Furthermore, a dedicated command-line interface provides an interactive questionnaire to store runtime configuration defaults and orchestrate local or AWS SageMaker runs.


### [How Shopify Dynamically Routes Storefront Traffic](https://yomu.fyi/post/how-shopify-dynamically-routes-storefront-traffic.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 9, 2021
- Type: Explainer

Shopify’s Storefront Renderer team rewrote the storefront implementation beginning in 2019 and needed a way to shift traffic between the legacy and new versions while migrating features. Using nginx with OpenResty, Lua modules, a control plane, and the spy chatbot, they moved routing rules from a Lua file into JSON payloads that could be created, updated, enabled, disabled, and deleted without a CI or deployment cycle. Rules selected shops and request subsets through filters, then controlled rendering and verification rates, including forward verification, reverse verification, and self-verification in the nearest region. During nginx request processing, matching occurs in the rewrite phase; verification is scheduled with timers in the log phase so duplicate requests run asynchronously after the client response path. The system supported gradual rollouts, parity checks, and resilience testing under Redis failures, while the team planned to simplify routing after the new storefront served almost all traffic.


### [Docker without Docker](https://yomu.fyi/post/docker-without-docker.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Apr 8, 2021
- Type: Explainer

Fly.io explains how it accepts OCI container images while avoiding Docker at runtime, choosing Firecracker micro-VMs because Docker’s isolation is not strong enough for its high-density multitenant environment. It describes OCI images as ordered tarball layers, showing how registry authentication, manifests, architecture selection, blob downloads, and configuration data are enough to reconstruct a container filesystem and entrypoint. On the platform side, flyctl uses Docker’s client library to push images, while a custom Go repository server adds token-based authorization, repository rewriting, and handling for cross-repository blob mounts and registry state. Servers use containerd as a cache over an LVM2 thin pool, leases and copy-on-write snapshots to provision VM filesystems quickly, and a Rust init configures mounts, networking, DNS, SSH, and the entrypoint before Firecracker runs the workload.


### [Distributed Training: Train BART/T5 for Summarization using 🤗 Transformers and Amazon SageMaker](https://yomu.fyi/post/distributed-training-train-bart-t5-for-summarization-using-transformer.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Philipp Schmid
- Published: Apr 8, 2021
- Type: Tutorial

Hugging Face and Amazon SageMaker introduced optimized Deep Learning Containers alongside a SageMaker Python SDK estimator to streamline distributed transformer model training. This workflow demonstrates fine-tuning the 400-million-parameter facebook/bart-large-cnn model for dialogue summarization on the 16,000-conversation samsum dataset. Using SageMaker Data Parallelism enabled in the distribution configuration, training executes across two ml.p3dn.24xlarge compute instances equipped with sixteen total GPUs and a cumulative batch size of 64. The HuggingFace estimator pulls the run\_summarization.py script directly from GitHub, enables mixed precision with fp16, and tracks evaluation performance through ROUGE scores. Following completion, the unzipped model artifacts and a generated model card are pushed to the Hugging Face Hub using the huggingface\_hub library for hosted inference.


### [Supabase Beta March 2021](https://yomu.fyi/post/supabase-beta-march-2021.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Apr 6, 2021
- Type: Announcement

Supabase announced a series of platform updates for March 2021, headlined by the launch of Supabase Storage and new local development capabilities. The new storage offering integrates an open-source Fastify and TypeScript API backed by S3, while direct database connections now bundle PgBouncer for connection pooling. Developers can run the platform locally via the Supabase CLI, utilize Docker tooling for self-hosting, and access the open-sourced Supabase UI component library. Community contributions brought OAuth scopes to Supabase Auth for requesting elevated third-party provider permissions, alongside newly published free and pro pricing tiers. In addition, development is underway on an Elixir-based workflow engine compatible with the Amazon States Language for automated event handling.


### [Low Latency LiveView](https://yomu.fyi/post/low-latency-liveview.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Apr 5, 2021
- Type: Explainer

LiveView sends HTML fragments over a WebSocket connection, allowing Phoenix applications to update the DOM in response to server or user activity. For interactions such as a like button, however, a server round trip can make feedback feel slow: a sub-100ms sequence may be acceptable, while many interactions benefit from responses under 40ms. The post presents client-side state prediction as one remedy, where the browser displays an expected value immediately and later reconciles it with the server. Its preferred “boring” alternative is reducing round-trip time by running application processes near users in multiple cities, connecting Phoenix instances over a private network and enabling clustering. A seven-region Counter deployment is shown with fly regions set and fly scale count 7, while Phoenix.PubSub and Phoenix.Presence handle clustering transparently.


### [Supabase Dot Com](https://yomu.fyi/post/supabase-dot-com.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Apr 2, 2021
- Type: Announcement

Supabase announced a migration of its primary website from supabase.io to supabase.com over the upcoming week. User API endpoints will remain unchanged on supabase.co, requiring no configuration adjustments for active applications. The company originally selected supabase.io as a meme-inspired placeholder after releasing a Postgres Realtime engine on GitHub in 2019 and failing to find an available dot com name. Following advice from Y Combinator to secure a dot com address, the founders bypassed domain brokers charging percentage fees and negotiated directly with the domain owner, whose business address was located one hundred meters from the founder's home. The migration updates public website destinations while preserving existing developer infrastructure unaffected.


### [PgBouncer is now available in Supabase](https://yomu.fyi/post/pgbouncer-is-now-available-in-supabase.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Angelico de los Reyes
- Published: Apr 2, 2021
- Type: Announcement

Serverless JavaScript frameworks and developer tools frequently establish separate database connections during bursty traffic, rapidly exhausting PostgreSQL connection limits in the absence of traditional middleware. To resolve connection surges, Supabase integrated the open-source connection pooler PgBouncer directly onto the PostgreSQL server across all newly created projects. Rather than increasing the total number of connections PostgreSQL can open, PgBouncer recycles open connections and queues excess requests until active connections become available. Developers can manage pooling through the dashboard across Session, Transaction, and Statement modes, with Transaction mode recommended for serverless functions despite disabling session features like prepared statements. The default pool size is initially configured to 15 connections.


### [Workflows are coming to Supabase](https://yomu.fyi/post/workflows-are-coming-to-supabase.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Francesco Ceccon
- Published: Apr 2, 2021
- Type: Announcement

Supabase is developing Workflows, an Elixir-based orchestration engine designed to coordinate complex serverless tasks and event-driven logic. Standard serverless and database-triggered functions often struggle with delayed execution and queuing without external cron processes. To solve this with native Postgres integration, the new engine adopts the open-source Amazon States Language specification to orchestrate functions across platforms including AWS, GCP, Azure, OpenFaaS, and Postgres itself. Execution states, jobs, queues, and logs are deeply integrated with Postgres using the Oban job processing library, though transient in-memory workers are also supported. The engine responds directly to HTTP calls or database change events delivered through Supabase Realtime.


### [Supabase Launches NFT Marketplace](https://yomu.fyi/post/supabase-launches-nft-marketplace.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Ant Wilson
- Published: Apr 1, 2021
- Type: Announcement

Non-fungible tokens frequently suffer from the copy-paste problem, where public URLs allow unauthorized users to download underlying media directly from web hosts or IPFS. To address this issue, BuyMeth proposes combining public blurhash thumbnails with full image files encrypted under the active owner's cryptographic key and hosted on IPFS. Completed sales trigger Metamask to re-encrypt the file with the buyer's public key, followed by a one-week escrow challenge window where automated verification matches decrypted image hashes against public thumbnails. Ongoing royalty distributions to previous owners provide a financial mechanism to disincentivize leaking unencrypted original image files. The announced platform serves as an April Fools joke rather than an active Supabase product release.


### [Building Smarter Search Products: 3 Steps for Evaluating Search Algorithms](https://yomu.fyi/post/building-smarter-search-products-3-steps-for-evaluating-search-algorit.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Apr 1, 2021
- Type: Tutorial

Shopify’s Help Center serves more than 2 million users monthly, making search relevance important across setup, marketing, and troubleshooting queries. The post presents a three-step framework for evaluating new search algorithms against an existing system, combining data collection, offline metrics, and online A/B testing. It contrasts Vanilla Pagerank, which boosts articles by total views across searches, with Query-specific Pagerank, which uses click frequencies from historic searches containing the search term. Kafka event streams are modeled into search facts for near-real-time monitoring and experiment assignment, while Support-team annotations provide relevance labels for offline evaluation, with dataset staleness requiring recurring updates. In the reported A/B test, Query-specific Pagerank users clicked more, relied less on follow-up searches and later pages, and reached a lower average clicked rank; the algorithm replaced Vanilla Pagerank.


### [Supabase CLI](https://yomu.fyi/post/supabase-cli.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Bobbie Soedirgo
- Published: Mar 31, 2021
- Type: Announcement

Supabase announces its CLI as the first step toward running Supabase locally, managing database migrations, generating types and API or validation schemas from a database, managing projects, and eventually pushing local changes to production. The local stack runs through Docker Compose, using tools described as 100% compatible with every Supabase project, and a React setup can initialize it with supabase init before supabase start. For schema changes, the post favors database diffing over accumulating manual SQL migrations and adds a CLI mode and Docker image for pgAdmin Schema Diff, which tracks view-owner changes needed for Row Level Security and handles domains. The supabase eject command outputs Docker Compose, Kong, and Postgres configuration for self-hosting. An update dated 15 August 2022 states that CLI V1 is generally available and that a Management API is in beta.


[Newer posts](https://yomu.fyi/posts/page/222.md) · [Older posts](https://yomu.fyi/posts/page/224.md)