Loading…
Browse by type
Explainer reads
Explainer posts on Yomu, summarised newest first.
- 1,318 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Explainer posts
React Native Skia: A Year in Review and a Look Ahead
React Native Skia’s first year produced an initial alpha and 89 additional releases, evolving a React Renderer and Skia integration for declarative 2D graphics across iOS, Android, the Web, and Node.js testing. The project changed direction after identifying two bottlenecks: dependence on the JavaScript thread and the overhead of thousands of JSI allocations in large drawings. Skia DOM addresses them with a declarative drawing state that recomputes only necessary changes and executes drawings on the UI thread, with C++ implementations on iOS and Android and a JavaScript implementation on React Native Web. CanvasKit compatibility, Expo integration, Metal on iOS, and OpenGL on Android and the Web support the cross-platform design, while extension points enable modules such as Skottie and camera-frame effects. The library’s tests compare rendered examples across platforms, and planned work includes JavaScript-worklet and Reanimated animation integration plus advanced text layouts through Skia’s paragraph module.
Optimizing Ruby’s Memory Layout: Variable Width Allocation
CRuby’s original garbage collector stores most Ruby objects in fixed 40-byte RVALUE slots, leaving differently sized data partly unused or forcing additional allocations through malloc. Shopify’s Variable Width Allocation project adds dynamic-sized allocation APIs and five fixed-size pools—40, 80, 160, 320, and 640 bytes—while increasing heap pages from 16 KB to 64 KB. The design targets cache locality, malloc overhead, memory utilization, and future garbage-collector experimentation; a new GC.stat_heap API reports size-pool statistics, and an escape hatch can disable the feature at build time on Ruby 3.2. Benchmarks on development Ruby 3.2.0 showed gains in most workloads, including 9.8% for psych-load, 5.3% for hexapdf, and 2.1% for railsbench, while mail was 1.7% slower.
Year in Review 2022: Tenderlove's Ruby and Rails Reflections and Predictions
Aaron Patterson’s 2022 review surveys developments in Ruby and Rails, pairing runtime performance work with tools intended to improve developer productivity. YJIT’s compiler was rewritten in Rust, enabling ARM and Apple Silicon support while producing faster machine code; Variable Width Allocation adjusts object slot sizes to keep instance variables embedded and improve data locality. Ruby 3.2’s Object Shapes use property order as a class-independent inline-cache key, while Ruby LSP adds editor services and Error Highlight adds column-level diagnostics that now appear on Rails error pages. Patterson predicts—more accurately, hopes—that languages, frameworks, and development environments will become more interconnected, providing better suggestions, context, and assistance so developers can spend less time coding and more time solving problems.
What's new in Postgres 15?
PostgreSQL 15, released as a stable version in October 2022, adds security, performance, SQL, replication, logging, compression, and string-processing changes. CREATE permission is revoked from all users except the database owner, views can use caller permissions with security_invoker=on, and sorting is reported up to four times faster for in-memory and on-disk workloads; window functions also improve. The release introduces MERGE, supports two-phase commits and selective data in logical replication, allows conflicting transactions to be skipped or subscriptions disabled, and enables parallel workers for SELECT DISTINCT. It also adds JSON logging, Zstandard and ZL4 compression for WAL and pg_basebackup, and POSIX regexp functions such as regexp_substr() and regexp_count(). The post recommends upgrading for bug fixes and stability, while noting that migrated databases may need manual permission changes and upgrades should be checked for rare manual actions.
Who We Hire at Supabase
Supabase describes the hiring traits and operating model it believes support its growth, arguing that people matter more than methodology. It looks for extreme asynchronous effectiveness, egoless execution, continuous incremental improvement, a bias toward immediate action, ambitious problem-solving, and a founder mindset; seniority does not exempt anyone from independent-contributor work, self-management, or maintenance. The 60-person team spans 25 countries, uses meetings as a last resort, and gives autonomous product teams access to shared metrics, user feedback, and equity through an ESOP. Teams are organised around products such as Postgres, Auth, Storage, Realtime, and Functions, while cross-cutting groups consult rather than merely receive delegated tasks. Three-month product cycles set ambitious goals, and Kaizen Weeks focused on QA, documentation, and performance help address technical debt and incorporate improvements into ongoing workflows.
Planning in Bets: Risk Mitigation at Scale
Preparing for BFCM and other high-traffic events, Shopify describes risk mitigation for a globally distributed, complex, interconnected infrastructure platform that peaked at 75.98M requests per minute, or 1.27M per second. At this scale, identifying and mitigating every possible failure is impossible, so the process starts with “what could go wrong” exercises, then uses voting, expert review of likelihood and severity, and assigned ownership to prioritize action. Decision makers are empowered to weigh risks and rewards, while summarized findings, including key risks and single points of failure, are shared to maintain alignment and awareness. The post frames engineering choices as probabilistic bets: decision quality should be judged separately from outcomes, and uncertainty cannot be eliminated, only addressed through thoughtful prioritization and communication.
How We Built Fly Postgres
Fly.io describes how it built Fly Postgres as an automated database app rather than a managed database service. After initially resisting durable storage and database work, the company added attached disk storage and designed a deployable Postgres application with convenience commands, using Postgres streaming replication, Stolon, and Consul to support high availability. A two-node cluster can designate a writable leader and asynchronous standby, while Stolon’s sentinels, keepers, and proxies coordinate state, elect a replacement leader, and route connections; Consul introduces an additional failure mode. Backups use point-in-time, block-level snapshots of the volume, a platform capability that can also support other file-system databases. The post concludes that automated Postgres serves many apps, while managed database partnerships may suit users seeking a simpler experience.
SQL or NoSQL? Why not use both (with PostgreSQL)?
The post compares SQL and NoSQL storage for an application that records daily food, water, exercise, weight, and notes, focusing on the tension between easy document writes and later analysis. It proposes using PostgreSQL as a hybrid: keep core fields such as date, user_id, weight, and notes in relational columns, while storing food_log, water_log, and exercise_log as JSONB fields. For analysis, PostgreSQL's jsonb_array_elements expands arrays into rows; ->> extracts text, -> returns JSON, and numeric values such as calories must be cast to INTEGER before aggregation, with a common table expression used to work around the restriction on set-returning functions inside aggregate calls. The examples show daily calorie totals and a search for garlic bread returning 200 calories, supporting the conclusion that JSONB can simplify storage while SQL preserves querying and analysis capabilities.
Implementing Server-Driven UI Architecture on the Shop App
The Shop Store team implemented a server-driven UI architecture for the Shop App’s Store Screen to personalize sections and layouts for different merchants. Previously, the client rendered a fixed layout, limiting customization, experiment timing, and the speed of fixes after weekly app releases. The new design uses template processing, orchestration, data loading, and a shop-server GraphQL layer, with ProductsSection and CollectionsSection types and configurable GridLayout or ShelfLayout options. On the client, ServerDrivenStoreScreen and StoreSectionContainer render the sections returned by the server, while default layouts allow older app versions to handle unfamiliar section or layout types. The architecture enables backend-controlled experiments and section changes without requiring an app release, while giving the product team a system intended to grow with the store experience.
What We Learned from Open-Sourcing FlashList
FlashList is Shopify’s React Native list library, created after Shopify’s migration from native technology exposed poor list performance in React Native. The post explains how a four-person team prototyped a library that combined FlatList’s familiar API with RecyclerListView’s performance, while addressing blank spacing and low frame rates on some devices. It also describes a launch planned six months ahead around a landing page, documentation, blog storytelling, social media, conference exposure, and feedback from external users. The post says FlashList was used in production on some Shopify app screens, and the launch achieved 1,000 GitHub stars in less than 24 hours. The broader conclusion is that successful open-source projects require product thinking, user empathy, community building, and a long-term authority-building strategy in addition to code.
Caching Without Marshal Part 1: Marshal from the Inside Out
Rails caching commonly relies on Ruby’s Marshal serialization, which can encode complex objects but also embeds their class names in cache entries. Shopify describes an incident in which a beta-flag refactor changed classes while old and new code overlapped during deployment; old code then failed on cached instances containing unfamiliar class names and methods, despite passing CI. The article examines Marshal through Ruby’s marshal.c implementation, covering atomic and composite types, instance variables, object references, circularity via TYPE_LINK, and core-type subclasses via TYPE_UCLASS. It presents MessagePack as a more compact, stricter, and controllable alternative intended to make caching safer by default, while deferring the cache migration and implementation details to the series’ next part.
Apollo Cache is Your Friend, If You Get To Know It
Shopify’s migration from Apollo GraphQL client 2 to client 3 prompted a closer examination of the Apollo InMemoryCache after past bugs were linked to misunderstanding or misuse. The cache stores an in-memory representation of queried data for the current browser session, and its lifecycle covers fetching, normalization, updating and merging, then garbage collection and eviction. Fetch policies determine whether data comes from the cache, the network, or both; under the default cache-first policy, incomplete data triggers a network request. Normalization breaks responses into objects, assigns cache identifiers usually from __typename and id or configured key fields, and stores them in a flattened structure. Automatic UI updates depend on matching identifiers and suitable query or mutation responses, while objects made unreachable by changed identifiers can remain until garbage collection removes them.
A love letter to React
React’s 2013 release had a lasting impact on frontend development and influenced Phoenix LiveView through its reactive component model, JSX colocation, and efficiency. Chris McCord explains how a component’s state and render function create a simple programming model, then shows LiveView adapting that model to a stateful server process with HTML-aware function components and slots. The discussion argues that tightly coupled markup and code should live together, while extensible components preserve HTML structure and support customization without bespoke templates. For rendering efficiency, React computes virtual DOM diffs, whereas LiveView tracks static and dynamic HEEx template parts, sends minimal diffs over the network, and applies client-side patches with morphdom. The conclusion credits React with shaping Phoenix’s direction and argues that backend frameworks should provide HTML-aware component systems.
The Management Poles of Developer Infrastructure Teams
Developer infrastructure managers must balance three poles: management support, system and domain expertise, and organization-wide road maps, with the tension intensified by teams that often lack dedicated product managers and support critical systems for hundreds or thousands of users with six to eight developers. Domain-focused teams build deep knowledge needed for maintenance, migrations, incident investigation, on-call coverage, and developer-experience improvements, but their local road maps can diverge from the department’s highest-impact opportunities. The proposed responses are temporary individual assignments, having a whole team contribute to another team’s goals, and forming a cross-functional tiger team; each shifts trade-offs among focus, maintenance, expertise, motivation, and management oversight. A month-long tiger team built the first Spin proof of concept, which led to a dedicated team, but no structure eliminates the tensions, so choices must be judged according to context.
Hubble: Our Tool for Encapsulating and Extending Security Tools
Shopify’s Trust team faced rising complexity in securing a distributed device fleet, including many tools, provisioning and patching work, varied networks, and the loss of in-person remediation. Hubble was built as a unified layer over mobile device management and other security systems, ingesting and standardizing their data while sending commands back with granular access controls and centralized auditing. IT staff use it for inventory, device management, and security, while employees can view device health and compliance, receive remediation guidance, and manage test-device or beta participation. The platform also serves as a standardized source for automation, and the post presents encapsulation and cross-team investment as ways to reduce administrative overhead, support proactive security, and limit risk.
Finding Relationships Between Ruby’s Top 100 Packages and Their Dependencies
RubyGems’ phased MFA rollout required owners of gems with at least 180 million downloads to use MFA, but raised a supply-chain question: could a popular gem depend on a less-downloaded gem that remained an account-takeover target? The investigation loaded rubygems.org data dumps to identify 112 gems above the threshold, queried the API for direct dependencies, and used Bundler with Gemfile.lock files to resolve transitive dependencies. It found 13 big gems with small direct dependencies and 24 with small dependencies overall, attributing the mismatch mainly to newer dependencies and gems shipped with Ruby, such as racc and rexml. Graph visualizations, breadth-first traversal, and a custom depth-first search then mapped dependency structure and paths; the post leaves open whether to do nothing or enforce MFA early for the 24 technically insecure gems.
The Engineering Story Behind Flex Comp
Shopify’s Flex Comp lets employees allocate total rewards among base salary, Restricted Stock Units (RSUs), and Stock Options, with allocation changes during windows and no one-year equity cliffs. Building the system required a prioritized backlog, cross-functional teams, early API contracts, and privacy controls around its code and data. For sensitive analytics events, the team encrypted entire Kafka payloads with RSA, using 4096-bit keys for ordinary events and 6144-bit keys for larger payloads, while later identifying hybrid encryption as a better design. On the frontend, typed hooks, mocked data, and artificial delays allowed business logic and interface work to proceed in parallel while contracts evolved. Version one shipped to Shopify’s employees, with documented technical and design debt and future decisions retained for subsequent iterations.
Safely Adding NOT NULL Columns to Your Database Tables
Shopify’s Database Migrations team investigated whether Large Hadron Migrator (LHM) can safely add NOT NULL columns to existing MySQL tables with minimal downtime. LHM creates a shadow table, installs triggers for INSERT, UPDATE, and DELETE operations, and copies records in batches before renaming the tables. The experiments varied whether the new column had a DEFAULT value or UNIQUE index and whether MySQL used strict or non-strict mode, evaluating trigger compatibility and record counts. A NOT NULL column with a DEFAULT was backward compatible, but adding a UNIQUE index could cause data loss; without a DEFAULT, strict mode broke compatibility while non-strict mode used implicit defaults, which may be undesirable.
Postgres WASM by Snaplet and Supabase
Supabase and Snaplet open-sourced postgres-wasm, a PostgreSQL server that runs inside a browser and supports browser persistence, pg_dump restoration, and logical replication with a remote database. Rather than compiling PostgreSQL directly to WebAssembly, the project packages a stripped-down Linux image with PostgreSQL in a Buildroot-based virtual machine, then runs it through v86 in the browser. PostgreSQL 14 startup failures were resolved by disabling v86 JIT compilation, while image optimization used a compressed 9P filesystem, asynchronous HTTPS loading, and page_poison=on to produce an initial state of about 12MB. Websocket proxying provides outbound connectivity and a forked proxy with nginx tunnels incoming connections to PostgreSQL. The project remains experimental because the download is currently about 30MB, but the post identifies offline data, testing, development environments, snapshots, and support as potential uses.
Lessons From Building iOS Widgets
Shopify’s widget team describes a third-generation redesign for iOS and Android after the original iOS widgets offered only two metrics and consumed substantial space. The widgets target merchants who need fresh shop analytics, using native SwiftUI on iOS because WidgetKit extensions have memory constraints, serialize SwiftUI view hierarchies to disk, and lacked official React Native support. Independent widget-side fetching uses shared session-management code and tokens; repeated update callbacks caused duplicate network requests and token-refresh races, so the team introduced a short-lived, widget-specific cache. Replacing existing widgets also exposed migration limits: changing widget kind identifiers could blank or remove installed widgets, leading Shopify to deprecate old widgets while releasing new ones and guide merchants through the transition. The team planned further customization and integration with iOS 16 features, including watch and lock-screen complications and live activities.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.