---
title: "Latest reads"
description: "The engineering internet, summarised so you can actually read it."
---

# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [Postgres WASM by Snaplet and Supabase](https://yomu.fyi/post/postgres-wasm-by-snaplet-and-supabase.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: burggraf
- Published: Oct 3, 2022
- Type: Explainer

Supabase and Snaplet open-sourced postgres-wasm, a PostgreSQL server that runs inside a browser and supports browser persistence, pg\_dump restoration, and logical replication with a remote database. Rather than compiling PostgreSQL directly to WebAssembly, the project packages a stripped-down Linux image with PostgreSQL in a Buildroot-based virtual machine, then runs it through v86 in the browser. PostgreSQL 14 startup failures were resolved by disabling v86 JIT compilation, while image optimization used a compressed 9P filesystem, asynchronous HTTPS loading, and page\_poison=on to produce an initial state of about 12MB. Websocket proxying provides outbound connectivity and a forked proxy with nginx tunnels incoming connections to PostgreSQL. The project remains experimental because the download is currently about 30MB, but the post identifies offline data, testing, development environments, snapshots, and support as potential uses.


### [A Software Engineer’s Guide to Giving Feedback that Actually Matters](https://yomu.fyi/post/a-software-engineer-s-guide-to-giving-feedback-that-actually-matters.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 30, 2022
- Type: Tutorial

The post presents feedback as a professional tool that can either support growth and collaboration or fuel ego-driven competition. It focuses first on pull request reviews, where engineers should prioritize critical code and logic over stylistic nitpicks, use linters or pre-commit hooks for uniform formatting, and frame comments to preserve a two-way educational dynamic. Recommended practices include discussing team review processes, explaining feedback goals, drafting messages before sending, reinforcing strengths, asking questions, separating behavior from identity, and making comments specific and timely. For broader workplace issues, the post advises asking whether feedback must be said, by the speaker, and immediately, then using considerate timing and SMART framing to improve the chance that it will be heard and produce change.


### [A Flexible Framework for Effective Pair Programming](https://yomu.fyi/post/a-flexible-framework-for-effective-pair-programming.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 27, 2022
- Type: Tutorial

Pair programming is presented as a mentoring and onboarding tool for early-career developers in Shopify’s Dev Degree program, where interns use it to build technical and communication skills. The author created a flexible framework to address progress tracking, session goals, a safe space, and communication, then refined it through an interactive RenderATL workshop with more than 30 attendees. The framework lets pairs choose roles, structure, agenda, environment, and communication practices; one described style assigns a driver to type and explain thought processes while a navigator reviews, observes, and suggests edge cases or refactorings, with frequent role switches. It also warns against hostile, passive-aggressive, and overly absolute language, and the pilot received positive intern feedback for supporting learning goals in a flexible format while continuing to evolve.


### [Lessons From Building iOS Widgets](https://yomu.fyi/post/lessons-from-building-ios-widgets.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 22, 2022
- Type: Explainer

Shopify’s widget team describes a third-generation redesign for iOS and Android after the original iOS widgets offered only two metrics and consumed substantial space. The widgets target merchants who need fresh shop analytics, using native SwiftUI on iOS because WidgetKit extensions have memory constraints, serialize SwiftUI view hierarchies to disk, and lacked official React Native support. Independent widget-side fetching uses shared session-management code and tokens; repeated update callbacks caused duplicate network requests and token-refresh races, so the team introduced a short-lived, widget-specific cache. Replacing existing widgets also exposed migration limits: changing widget kind identifiers could blank or remove installed widgets, leading Shopify to deprecate old widgets while releasing new ones and guide merchants through the transition. The team planned further customization and integration with iOS 16 features, including watch and lock-screen complications and live activities.


### [Lessons From Building Android Widgets](https://yomu.fyi/post/lessons-from-building-android-widgets.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 22, 2022
- Type: Explainer

Shopify revisited its Android widgets alongside an iOS widget effort, aiming to give merchants faster access to frequently changing business metrics. The team chose native Android development because React Native lacked official RemoteViews support, while widget configuration data was persisted with shared preferences and used to build requests. Because Android could invoke onUpdate() multiple times per cycle, they added a one-minute, short-lived cache keyed by widgetID and mindful of configuration such as locale. This reduced redundant network calls, system load, and incorrect analytics, while the widget rollout also exposed migration limitations between old fixed-size widgets and new space-responsive ones, leading Shopify to deprecate the old widgets rather than transform them.


### [Introducing LiteFS](https://yomu.fyi/post/introducing-litefs.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Sep 21, 2022
- Type: Announcement

LiteFS is an open-source distributed file system for SQLite that addresses the durability, replication, availability, and latency limitations of deploying SQLite on a single server. LiteFS interposes a FUSE pass-through filesystem between an application and its database, identifies transactions from SQLite’s journal files, packages page changes into LTX files, and ships them to replicas without requiring application-code changes. A Consul-backed, time-based lease elects a primary writer, while local replicas serve reads; checksum state embedded in each LTX file helps detect out-of-order application and preserve byte-for-byte consistency. Because replication is asynchronous, the design trades some durability guarantees for performance and operational simplicity, while defensive checksum checks address potential split-brain failure modes. The project is described as beta-ready, with WAL-mode support, write forwarding, S3 replication, and encryption listed as planned features.


### [What is a Full Stack Data Scientist?](https://yomu.fyi/post/what-is-a-full-stack-data-scientist.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 15, 2022
- Type: Explainer

Shopify defines full stack data science as an ownership-oriented, end-to-end approach to solving business problems across the data science lifecycle. Rather than specializing in one stage, practitioners participate in discovery and analysis, acquisition, data modeling, pipeline development, and production delivery. The role combines business acumen, communication, Python and SQL programming, exploratory data analysis, data engineering, and machine learning, while treating machine learning as a tool rather than a default and starting simple before adding complexity. Shopify describes the model as generalist with T-shaped development: broad capability is paired with deeper expertise in a specific area, enabling individual strengths within teams to address complex challenges.


### [Scale-to-Zero Minecraft server with Terraform and Fly Machines](https://yomu.fyi/post/scale-to-zero-minecraft-server-with-terraform-and-fly-machines.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Sep 14, 2022
- Type: Tutorial

This tutorial presents a scale-to-zero Minecraft Java server built with Terraform and Fly Machines. It addresses the cost and resource burden of keeping a multiplayer server running when nobody is playing, while allowing it to restart when a player connects. The configuration provisions a Fly.io app, a Firecracker-based machine using Geoff Bourne’s itzg/minecraft-server Docker image, a 15GB persistent Fly Volume, an IPv4 address, and exposed TCP port 25565. Environment variables enable Autostop with estimated and initial timeouts of 120 seconds, while the machine receives four shared vCPUs and 8GB of RAM and mounts world data at /data. After terraform apply, the machine shuts down after the timeout and automatically starts again when Minecraft reconnects, though any TCP traffic can wake it and incur costs.


### [Managing React Form State Using the React-Form Library](https://yomu.fyi/post/managing-react-form-state-using-the-react-form-library.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 14, 2022
- Type: Tutorial

Shopify’s tutorial shows how to manage a React product form with the open-source react-form library from its Quilt repository. It replaces component-level useState and earlier input and submit handlers with the useForm and useField hooks, which provide form and field state, event handlers, submission, reset, and dirty tracking. The example connects each TextField to its corresponding fields entry, using its value and onChange properties so typed input is captured and displayed. It then wires the hook’s submit function, resets the form after data is handled, and disables Save until either field changes. Finally, the notEmpty validation hook requires a title and displays a required-field error when a submission contains only a description.


### [Leveraging Go Worker Pools to Scale Server-side Data Sharing](https://yomu.fyi/post/leveraging-go-worker-pools-to-scale-server-side-data-sharing.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 9, 2022
- Type: Problem & solution

Shopify’s Server Pixels service validates, processes, augments, and produces more than one billion customer events daily, but rising traffic caused Kafka input-topic consumption lag when downstream processing slowed. Its original design could spawn an unlimited number of goroutines while processing or producing event batches, consuming CPU and memory and threatening the five-minute event-processing SLO. The service was redesigned around fixed-size Go worker pools: workers receive Job values through channels, process them via reusable work functions, and use wait groups and context cancellation for orderly shutdown. Load testing showed 15 workers in both processor and producer raised throughput to 12.9 thousand events per second per pod, while 50 workers reached 19.3 thousand; subsequent improvements reached 21 thousand, a 170% increase over 7.75 thousand, and the service handled a maximum of 46 thousand events per second during BFCM 2021.


### [Choosing a Postgres Primary Key](https://yomu.fyi/post/choosing-a-postgres-primary-key.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: victor
- Published: Sep 8, 2022
- Type: Explainer

Choosing a Postgres primary key involves balancing simplicity, storage, sortability, standards compliance, information leakage, and collision avoidance. The discussion starts with natural and surrogate keys, then compares integer and biginteger identity columns with serial sequences, noting that sequences can have gaps and exposed numeric IDs may reveal information. It then examines pseudorandom and secure random numeric IDs before turning to UUIDs, including UUIDv4 and time-based alternatives, whose generation methods and index behavior differ. Postgres integration, size queries, and the pg\_idkit library provide practical ways to generate and evaluate identifiers; the suggested rules of thumb favor integers for simplicity, UUIDv4 for collision avoidance, xid for time-based sort friendliness, and UUIDv6 or UUIDv7 for a more standards-oriented choice.


### [Automatic rule backtesting with large quantities of data](https://yomu.fyi/post/automatic-rule-backtesting-with-large-quantities-of-data.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Chao Wang
- Published: Sep 8, 2022
- Type: Problem & solution

Evaluating new or modified risk rules previously required Grab analysts to run slow offline Presto queries, manually construct payloads, or run rules in shadow mode for days. To standardize and accelerate this workflow, Grab developed an automated backtesting system powered by an AWS EMR Spark pipeline. Historical events are continuously ingested via Kafka and a Kubernetes stream pipeline into S3 using Snappy-compressed Parquet. Users configure replay intervals and rule definitions directly in the rule engine UI, which triggers asynchronous Spark jobs through Amazon SQS and Lambda. This automated simulation replaces multi-week shadow mode runs and generates downloadable aggregation metrics on transactions, user counts, and treatment outcomes.


### [Supabase Beta August 2022](https://yomu.fyi/post/supabase-beta-august-2022.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: ant\_wilson
- Published: Sep 7, 2022
- Type: Announcement

Supabase’s August 2022 Beta Update recaps Launch Week 5 releases, the Supa Hackathon, platform changes, migration resources, and community contributions. The release set includes Supabase CLI v1 with TypeScript type generation, the Management API Beta for programmatic project and organization control, and the supabase-js v2 release candidate with type support, new authentication methods, an async Auth overhaul, and Edge Functions improvements. Supabase also announced SOC2 compliance and made Realtime Presence and Broadcast available to all projects, alongside Vault, Auth UI, dashboard permissions, JSON schema validation, pg\_graphql v0.4.0, and MFA early access. From September 11, HTTP API requests will redirect to HTTPS, and clients no longer need to send anon or service keys in the apiKey header; client-library users require no change.


### [How the SQLite Virtual Machine Works](https://yomu.fyi/post/how-the-sqlite-virtual-machine-works.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Sep 7, 2022
- Type: Explainer

SQLite processes SQL through a parse-optimize-execute pipeline that turns query text into executable virtual-machine instructions. It tokenizes characters, parses them into an AST such as a Select struct, and uses table statistics to choose between a full table scan and index-based access. The source describes sqlite\_stat1's coarse index and duplicate-value counts alongside sqlite\_stat4's distributed samples, then traces bytecode that opens the persons table, compares favorite\_color with "blue", and returns matching columns. Registers hold intermediate values, cursors iterate through tables, and sqlite3\_step() resumes execution after each returned row; the VM has 186 opcodes as of writing, while bind parameters allow prepared statements to skip repeated parsing and optimization.


### [Four Approaches to Debugging Server-side WebAssembly](https://yomu.fyi/post/four-approaches-to-debugging-server-side-webassembly.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 7, 2022
- Type: Explainer

Server-side WebAssembly offers Shopify Functions low cold-start latency and robust security, but debugging remains difficult because step-debugger support is less refined. The post compares four approaches: compiling to a native architecture, using LLDB or GDB with Wasmtime, using a dedicated WebAssembly debugger, and using browser developer tools. Native debugging generally provides the best variable rendering and a familiar workflow, though it may require stubs for external calls and cannot reproduce WebAssembly-specific behavior. Wasmtime keeps execution closer to the target environment, while dedicated and browser-based tools offer WebAssembly-specific capabilities but have missing features or lower-fidelity variable views. The recommended choice depends on whether the issue reproduces natively, and browser-based tooling may become more accessible as it improves.


### [Focus On Behavior, Not State, for a More Maintainable Codebase](https://yomu.fyi/post/focus-on-behavior-not-state-for-a-more-maintainable-codebase.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 1, 2022
- Type: Explainer

Shopify’s work adding pre-orders and “try before you buy” to its Purchase Options APIs exposed code that coupled selling plans too tightly to subscriptions, making future purchase models harder to add. The team reframed the domain around orders and behaviors, treating subscriptions as recurring orders and distinguishing recurring or deferred delivery and billing where those properties mattered. It introduced a wrapper around selling-plan Active Record objects, replaced those objects outside their defining component, and exposed methods such as whether deliveries are recurring while keeping policy details encapsulated. Because platform behavior and merchant reporting need different concepts, the team also added a translation table populated at order creation, with calculated or merchant-provided labels and optional trait records; the maintenance cost isolates reporting semantics from platform and data-team logic.


### [RailsConf 2022: 10 Shopify Tech Talks You Might Have Missed](https://yomu.fyi/post/railsconf-2022-10-shopify-tech-talks-you-might-have-missed.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Aug 30, 2022
- Type: Announcement

Ruby Central has posted videos from RailsConf talks and workshops online, following the 2022 in-person conference where several Shopify Ruby on Rails team members attended and around a dozen gave talks. The collection features ten Shopify sessions on Rails’s state, Ruby’s YARV virtual machine, cache serialization, algorithmic complexity, historical Ruby gems, supply-chain attacks, team trust, GraphQL delivery, and regular-expression denial-of-service attacks. Several talks use concrete technical examples: MessagePack replaced Marshal in the core monolith’s cache, OpenTelemetry is used to diagnose performance issues, and one tutorial demonstrates GraphQL delivery through WebSockets, Webhooks, WebAssembly code, and Bulk Operations. The sessions also connect implementation details with broader engineering practices, including investment in open-source teams, playful problem solving, pair programming, Rubocop, compilers, and examining older code to inform current decisions.


### [When Culture and Code Reviews Collide, Communication is Key](https://yomu.fyi/post/when-culture-and-code-reviews-collide-communication-is-key.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Aug 26, 2022
- Type: Explainer

The post examines how cultural differences in politeness, directness, and face-saving can distort written code reviews, causing useful feedback to be softened, misread, or ignored. It contrasts a passive-aggressive Shopify pull-request exchange about missing permalink tests with a more curious and explicit version that states why a separate test would be worthwhile. The discussion distinguishes assuming positive intent from asserting it: context can reduce bias, but it must not erase the impact of embarrassing or harmful communication. It recommends treating reviews as collaborative invitations, asking questions, explaining reasoning, acknowledging reviewers as contributors, and looking past stylistic language to shared goals. The conclusion is that clearer communication and shared ownership can make disagreement productive and improve both learning and the codebase.


### [Launch Week 5 Hackathon Winners](https://yomu.fyi/post/launch-week-5-hackathon-winners.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: tyler\_shukert
- Published: Aug 25, 2022
- Type: Announcement

Supabase announces the winners of its Launch Week 5 Hackathon, which received 46 project submissions and was described as the most popular hackathon yet. The awards recognized Supabase Cache Helpers as Best Overall for a caching utility that turns Supabase queries into cache keys and provides methods to retrieve and mutate database data, while Super UI was runner-up with a customizable React component library. Other category winners included Hop, a Tinder-like job board; Otentik Authenticator, a desktop app for managing two-step verification tokens; That Hot Dog Game, which uses facial recognition to detect eating; Quid Faciam Hodie?, a video journaling app; and RepoWatch, which shows Supabase GitHub organization updates in realtime. Winning team members receive limited-edition golden Supabase caps, runners-up receive silver caps, and all submissions are available on madewithsupabase.com.


### [Shopify Data’s Guide To Opportunity Sizing](https://yomu.fyi/post/shopify-data-s-guide-to-opportunity-sizing.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Aug 25, 2022
- Type: Explainer

Opportunity sizing gives businesses a structured way to estimate an initiative’s potential impact and compare it with the cost of pursuing alternatives. The framework recommends expressing an estimate as an expected change in a metric, with a range, timeframe, and documented assumptions; an annualized view makes initiatives easier to compare. Directional t-shirt sizing uses rough benchmarks for early ideation and sanity checks, while bottom-up sizing uses observed performance from a comparable product or system, such as email click-through rates, to produce a more accurate estimate. Top-down sizing is intended for new initiatives without comparable systems and starts with broader market or audience information before narrowing the estimate. Shopify recommends connecting initiative-level changes to wider business goals, testing estimates against actual results after launch, and refining the process over time.


[Newer posts](https://yomu.fyi/posts/page/203.md) · [Older posts](https://yomu.fyi/posts/page/205.md)