---
title: "Latest reads"
description: "The engineering internet, summarised so you can actually read it."
---

# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [How to Make Dashboards Using a Product Thinking Approach](https://yomu.fyi/post/how-to-make-dashboards-using-a-product-thinking-approach.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jan 28, 2021
- Type: Tutorial

The guide presents dashboards as automatically updated collections of visualisations or metrics for monitoring business questions and argues they should be built with a product-thinking approach. It first recommends deciding whether a dashboard is appropriate, then defining one clear goal, a specific audience, and metrics that reflect their needs before sketching the layout with stakeholders. Implementation guidance covers readable and efficient queries, version control, peer review, visual hierarchy, restrained content, business and data context, refresh frequency, technical reviews, and user sign-off. After launch, teams should market the dashboard, make it discoverable, monitor usage and outcomes, iterate when it falls short, and assign ownership for maintenance. The intended outcome is a dashboard that stays useful and delivers business impact by keeping users front and centre.


### [Managing Google Cloud Platform Project-Wide SSH Keys](https://yomu.fyi/post/managing-google-cloud-platform-project-wide-ssh-keys.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jan 27, 2021
- Type: Problem & solution

Google Cloud project-wide SSH keys can remain in project metadata indefinitely after a user connects to a VM, potentially allowing matching private-key holders to access Linux VMs with root access unless project-wide keys are blocked. Shopify developed SSH-Pruner because OS Login did not work with Google Kubernetes Engine, while deleting all keys also removed Google’s keys, disrupted scripts, and created slow metadata updates. The new Go application uses Google Cloud APIs to find organization projects with Compute Engine enabled, reads the “ssh-keys” metadata, parses entries, retains unexpired keys, writes pruned common instance metadata, and records readable logs; Google-managed “sshKeys” entries are skipped. SSH-Pruner was already running as a cron job on some projects, with a slow rollout planned to limit workflow disruption.


### [Serving Driver-partners Data at Scale Using Mirror Cache](https://yomu.fyi/post/serving-driver-partners-data-at-scale-using-mirror-cache.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Indrajit Sarkar
- Published: Jan 26, 2021
- Type: Problem & solution

Grab's Drivers Data service handles up to 10,000 requests per second during peak hours to supply driver information across backend microservices. The original setup used MySQL with Redis and standalone in-memory local caches, but yielded a low 25% local cache hit rate due to traffic patterns characterized by high burst frequency for individual drivers alongside redundant database calls across nodes. To solve this, the team developed Mirror Cache, an in-memory caching system that pairs Dgraph's Ristretto library with an asynchronous gRPC replication layer to mirror updates across cluster nodes. The replicator batches updates within the same AWS availability zone and forwards data to single nodes across zones to minimize transfer overhead. Production deployment increased the in-memory cache hit rate to approximately 75% and reduced direct MySQL queries by 5%.


### [The January 2021 Fly ChangeLog](https://yomu.fyi/post/the-january-2021-fly-changelog.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Jan 26, 2021
- Type: Announcement

The January 2021 Fly ChangeLog covers platform and flyctl changes from December 2020 through January 2021, focusing on private networking, VPN access, scaling, and infrastructure work. 6PN networking adds private communication and DNS-based app discovery over Fly's WireGuard mesh, while IPv6 WireGuard Peering provides tunnels into an organization; the new wireguard command creates, lists, and removes tunnels. Fly also made instance-count scaling the default model, separating it from autoscaling through scale and autoscale commands. Backend updates switched image pulling to containerd, improving subsequent boot times for larger images, added app lifecycle events to application logs, and made volume initialization lazy with encryption enabled by default. The listed flyctl releases also add volume, memory-scaling, region, completion, and WireGuard updates, alongside bug fixes.


### [Faster TensorFlow models in Hugging Face Transformers](https://yomu.fyi/post/faster-tensorflow-models-in-hugging-face-transformers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Julien Plu
- Published: Jan 26, 2021
- Type: Tutorial

Hugging Face updated Transformers to improve the computational performance and robustness of its TensorFlow models, including BERT, RoBERTa, ELECTRA, and MPNet. In version 4.2.0, the optimized BERT implementation runs up to approximately 10 percent faster than Google's official implementation and twice as fast as Transformers release 4.1.1 when tested on a V100 GPU. The release also improves TensorFlow SavedModel generation by allowing flexible sequence lengths across runs, exposing all model inputs, and grouping hidden states or attention outputs into single tensors. In addition, users can customize serving signatures by subclassing models and decorating custom methods with explicit input specifications. Models exported to the SavedModel format can be packaged into Docker containers and served via TensorFlow Serving using either HTTP REST or gRPC APIs for production inference.


### [Using GraphQL for High-Performing Mobile Applications](https://yomu.fyi/post/using-graphql-for-high-performing-mobile-applications.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jan 22, 2021
- Type: Explainer

GraphQL is presented as a way to address mobile developers’ REST API pain points, especially repeated JSON-to-model mappings and failures when server fields or types change. It provides a strongly typed, self-documented schema contract, while introspection enables tooling to generate client code and surface deprecations at compile time. The post describes mobile practices including using reusable UI-oriented fragments, aliases for readable data models, and directives that conditionally include or skip fields; Shopify uses the open-source Syrup tool to generate strongly typed Swift and Kotlin code from application queries. This flexibility shifts responsibility toward query design: Shopify calculates field costs, limits calls through a replenishing per-user cost bucket, and requires clients to consider query complexity for responsiveness and server resources. GraphQL is positioned as language-independent and suited to bandwidth, latency, and UX-sensitive clients.


### [Fit More and Train Faster With ZeRO via DeepSpeed and FairScale](https://yomu.fyi/post/fit-more-and-train-faster-with-zero-via-deepspeed-and-fairscale.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Stas Bekman
- Published: Jan 19, 2021
- Type: Benchmark

Machine learning model sizes frequently outpace consumer GPU memory, preventing practitioners from loading or fine-tuning large architectures on standard hardware. To address these resource constraints, the Hugging Face Trainer introduced experimental support for Zero Redundancy Optimizer features from DeepSpeed and FairScale starting in transformers v4.2.0. In dual-GPU benchmarks with a t5-large model, FairScale and DeepSpeed integrations expanded allowable batch sizes and shortened training durations compared to baseline DistributedDataParallel runs. Furthermore, DeepSpeed permitted single-GPU fine-tuning of a t5-3b model on a 24GB RTX-3090 card at batch size 20, whereas the baseline execution crashed immediately with an out-of-memory error. These optimizations manage memory allocation internally to curb memory fragmentation while requiring changes only to training scripts rather than model code.


### [The GrabMart Journey](https://yomu.fyi/post/the-grabmart-journey.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Clarisse Peralta
- Published: Jan 18, 2021
- Type: Problem & solution

During the COVID-19 pandemic, Grab accelerated the rollout of its GrabMart on-demand grocery delivery service to meet surging regional demand. Early pilot testing within the existing GrabFood interface revealed that grocery shopping fundamentally diverges from meal ordering, as users search at an item level rather than browsing merchants. To support grocery workflows, the team restructured the user experience with item and store categories, one-click add-to-cart functionality, and scheduled delivery options. Engineering teams simultaneously updated core infrastructure across order management, driver allocation systems, and merchant Partner APIs to handle specialized mart bookings. Rollouts were validated across two main phases using the internal Grab Early Access program to catch issues before regional release.


### [Fly In 2020 - A year in features (and articles)](https://yomu.fyi/post/fly-in-2020-a-year-in-features-and-articles.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Jan 18, 2021
- Type: Announcement

Fly’s 2020 retrospective covers the launch and expansion of its Global Application Platform, alongside features and articles released throughout the year. The platform added Turboku for moving Heroku apps to the edge, Cloud Native Buildpack support, Custom Domains with a GraphQL management API, scaling controls, Deno support, pausing and resuming, persistent volumes, and builtin Flyctl builders for Node, Ruby, Deno, Go, and static websites. Later additions included a new logging system, 6PN Private Networking, WireGuard peering, graceful VM shutdown options, and revised VM sizes, while examples demonstrated services such as Redis, MinIO, Gogs, and MQTT using persistent volumes. The article also records supporting work including GoStatic, JSON output in Flyctl, Firecracker-based workload isolation, and Heroku database usage.


### [How we sped up transformer inference 100x for 🤗 API customers](https://yomu.fyi/post/how-we-sped-up-transformer-inference-100x-for-api-customers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Nicolas Patry
- Published: Jan 18, 2021
- Type: Problem & solution

Deploying large transformer models in production poses severe latency and scaling hurdles for machine learning engineers. Hugging Face achieved a cumulative 100x inference acceleration on its Accelerated Inference API by pairing library-level algorithmic adjustments with low-level hardware compilation. The initial 10x improvement reduces forward-pass computations, such as restricting attention calculations to the latest token in GPT generation tasks, while utilizing Rust-based tokenizers and caching mechanisms. The subsequent 10x speedup requires custom static graph compilation targeted to selected CPU or GPU hardware profiles, applying unused flow removal, instruction-specific layer fusion, and precision quantization via ONNX Runtime. These coordinated techniques enable real-time prediction latency across varied model sizes and demand profiles without sacrificing task accuracy.


### [Building clusters with serf, my new favorite thing](https://yomu.fyi/post/building-clusters-with-serf-my-new-favorite-thing.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Jan 14, 2021
- Type: Problem & solution

The post builds a geographically distributed Headless Chromium cluster on Fly.io and uses it to examine the limits of DNS-based service discovery. It first deploys Chromium in a Singapore Firecracker VM, secures Chrome Debug Protocol access by enabling Fly's private 6PN network and connecting through WireGuard, then scales instances across Singapore, Sydney, Paris, and Chile. Because DNS provides only snapshots and no convenient change notifications, each node runs a leaderless Serf agent bound to its private IPv6 address, tags members by role and region, and joins peers discovered through internal DNS. Serf then supplies an up-to-date membership map, RTT and Vivaldi-based nearness information, plus events and queries; the post concludes that it is simpler than Consul or Zookeeper and more capable than DNS for this setup.


### [Trident - Real-time Event Processing at Scale](https://yomu.fyi/post/trident-real-time-event-processing-at-scale.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Jie Zhang
- Published: Jan 13, 2021
- Type: Problem & solution

Trident serves as Grab's internal real-time event-processing and workflow automation engine, driving user campaigns, rewards, and notifications across multiple business lines. To handle peak loads exceeding 2,000 events per second without duplicate execution, the system consumes decoupled Kafka streams and enforces exactly-once semantics using Redis and MySQL deduplication checks. Processing efficiency relies on server autoscaling aligned with Kafka partition counts, combined with dynamic goroutine allocation per consumer. To minimize rule evaluation overhead, Trident indexes active campaigns into an in-memory hash map by event type, cutting processing time by at least 90%. Furthermore, condition evaluation is optimized through lazy loading and a weighted sorting algorithm that checks low-cost in-memory data prior to executing expensive database queries or external service calls.


### [Apache Beam for Search: Getting Started by Hacking Time](https://yomu.fyi/post/apache-beam-for-search-getting-started-by-hacking-time.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Jan 8, 2021
- Type: Tutorial

Search relevance can use clickstream signals such as clicks and purchases to promote popular results and demote results users do not favor, but batch processing can delay those adjustments by days. The Discovery team at Shopify uses Apache Beam, a unified batch and stream processing system, to support real-time and historical search-behavior workflows, including boosting, popularity aggregates, offline evaluation sets, and reinforcement learning tasks. The article’s central lesson is that Beam requires careful handling of event time, processing time, delayed events, out-of-order data, and watermarks, especially when combining streams. A Kafka source can use Kafka processing time, client create time with a five-minute out-of-order allowance, or a custom timestamp policy that reads the SearchQueryEvent.searchTimestamp field; Beam windows add further buffering and lateness controls. Accurate timestamping is presented as the first milestone for replaying clickstream data reliably before tackling more complex operations such as streaming joins.


### [Get fly with your Fly command line](https://yomu.fyi/post/get-fly-with-your-fly-command-line.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Jan 5, 2021
- Type: Tutorial

Fly’s command-line tool, available as both flyctl and fly on most platforms, includes shortcuts and options that reduce directory changes and browser navigation. App selection can use -a appname instead of relying on the current directory’s fly.toml; fly deploy can also build from a specified child directory, such as ./mygreatchildapp. When a fly.toml is present, the command asks for confirmation that the supplied app name should be used. Multiple deployment variants can be kept in separately named configuration files and selected with -c or --config, including when deploying from a monorepo tree. Browser-oriented commands include fly open for an app’s .fly.dev URL, fly dash metrics for dashboard metrics, and fly docs for the documentation’s top page.


### [Supabase Beta December 2020](https://yomu.fyi/post/supabase-beta-december-2020.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Jan 2, 2021
- Type: Announcement

Supabase transitioned to Beta following ten months of development concentrated on performance, security, and reliability. The platform introduced inline table documentation updates, storing descriptions directly as PostgreSQL comments and exposing them through OpenAPI specifications. Developers can now utilize a table view featuring immediate realtime update reflection alongside pagination for handling large datasets. Performance measurements show a 26 percent increase to 1460 requests per second in regions supporting AWS Graviton instances. Furthermore, the update introduces a new region in Sao Paulo, editing support for native Postgres arrays in the grid editor and side panel, and broader compatibility for custom database types.


### [Graceful VM exits, some dials](https://yomu.fyi/post/graceful-vm-exits-some-dials.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Dec 29, 2020
- Type: Announcement

Fly.io introduces configurable VM shutdown behavior for applications whose clients, connections, or transactions need more than the default five-second termination window. After sending SIGINT, Fly normally waits five seconds before forcefully terminating a VM; the new kill\_timeout setting extends that wait to as much as five minutes on Shared CPU VMs or 24 hours on dedicated CPU VMs, while kill\_signal selects among several signals. The post demonstrates HAProxy’s soft-stop mode with SIGUSR1 to drain existing TCP connections and Postgres’s smart shutdown with SIGTERM to let open transactions commit before termination. It also describes infrastructure considerations: deploys and other releases can disrupt in-flight connections, so Fly drains connections where possible and offers ways to reduce reliance on shared HTTP and TLS handlers, plus stateless UDP and experimental stateless TCP load balancing.


### [Building a Monorepo with Yarn 2](https://yomu.fyi/post/building-a-monorepo-with-yarn-2.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Danielle Adams
- Published: Dec 23, 2020
- Type: Tutorial

The post presents Yarn 2 workspaces as a way to manage dependencies for a JavaScript or Node.js monorepo containing separate server and UI applications. It configures the root package.json with workspace directories, imports the workspace-tools plugin, pins the project to a checked-in Yarn binary with yarn set version berry, and enables the dependency cache through .yarn/cache and Plug’n’Play. The walkthrough adds Express, TypeScript, and @types/express to a server workspace, creates a Next.js UI workspace, and uses workspace scripts to build both applications, while yarn dlx initializes TypeScript without a global install. It also shows the Git files to retain, generated JavaScript to ignore, and a Heroku start script that launches the server workspace; the stated benefit is faster installs through a committed cache and zero-install deployment.


### [IPv6 WireGuard Peering](https://yomu.fyi/post/ipv6-wireguard-peering.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Dec 23, 2020
- Type: Announcement

Fly.io presents WireGuard Peering as a way to connect applications running on its platform privately to AWS, GCP, or other networks, while preserving automatic communication among apps in an organization. Each organization receives a private IPv6 network called a 6PN, with addresses encoding organization, host, and instance identifiers; BPF enforces isolation, and internal DNS resolves app, regional, and peer names. Because WireGuard’s cryptokey routing does not allow overlapping Allowed IPs, Fly swaps the address’s network and host chunks before and after routing, giving the mesh non-overlapping prefixes without changing IPv6 checksums. Users create regional peer configurations with flyctl wireguard create, receiving /120 delegations and organization-specific DNS. The post says this works for database gateways such as an AWS RDS setup, direct app-instance access, and arbitrary services, with connectivity remaining direct between nodes.


### [Pharos - Searching Nearby Drivers on Road Network at Scale](https://yomu.fyi/post/pharos-searching-nearby-drivers-on-road-network-at-scale.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Hao Wu
- Published: Dec 22, 2020
- Type: Problem & solution

Ride-hailing allocation requires identifying nearby drivers based on actual routing distance and estimated time of arrival rather than straight-line haversine distance. Grab developed Pharos, a distributed in-memory spatial microservice that evaluates road networks to match fast-moving drivers with pick-up requests. The system partitions OpenStreetMap road graphs by city and vehicle vertical while indexing driver positions and edge-based nodes using Adaptive Radix Trees. To locate candidates, Pharos projects driver coordinates onto road segments as phantom nodes and runs Incremental Network Expansion to compute isochrone reaches. Read-write concurrency is managed by switching tree root snapshots, allowing concurrent reads while updates produce new roots for subsequent lookups.


### [How Shopify Uses WebAssembly Outside of the Browser](https://yomu.fyi/post/how-shopify-uses-webassembly-outside-of-the-browser.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 18, 2020
- Type: Explainer

Shopify describes using WebAssembly (Wasm) outside the browser to run untrusted Partner code on its own infrastructure, reducing partners’ need to operate scalable web services and avoiding network latency for time-sensitive customizations. Wasm provides a sandboxed, stack-based environment with explicit host imports, protected call stacks, and runtime type checking, while targeting near-native performance and multiple source languages. Shopify wraps Fastly’s open-source Lucet runtime and compiler in a Rust web service called the Wasm Engine, which manages module I/O and storage and applies results during runtime processes such as web requests. In a test executing 100,000 modules per minute for roughly five minutes, Lucet execution hovered around 100 μs and total p99 execution time was about 4 ms. Shopify selected AssemblyScript, with TypeScript-like syntax, for first-class developer support and is extending its tooling, SDKs, monitoring, and language support.


[Newer posts](https://yomu.fyi/posts/page/225.md) · [Older posts](https://yomu.fyi/posts/page/227.md)