---
title: "Latest reads"
description: "The engineering internet, summarised so you can actually read it."
---

# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [We cut bandwidth prices. Go nuts.](https://yomu.fyi/post/we-cut-bandwidth-prices-go-nuts.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Dec 3, 2020
- Type: Announcement

Fly.io announces lower outbound data-transfer pricing for apps running in its Firecracker VMs: $0.02 per GB from North America and Europe, and $0.04 per GB almost everywhere else. The December 1, 2020 change follows dramatically increased volume, reducing bandwidth costs by 75% from the March launch prices of $0.085 per GB in North America and Europe and $0.14 in Asia Pacific. Rather than use graduated tiers or regional rates, Fly.io uses one blended price for all customers, accepting that some traffic is more expensive and some use cases may pay too much. A 70% target margin underpins the model, while a 160GB monthly free allowance covers low-volume projects; India is an exception at $0.12 per GB, and large-transfer customers can request lower prices if they commit to substantial monthly usage.


### [Capacity Planning at Scale](https://yomu.fyi/post/capacity-planning-at-scale.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 3, 2020
- Type: Explainer

Shopify’s BFCM preparation combines traffic forecasting, resource planning, and platform-wide scalability testing to reduce the risk of outages during its busiest sales period. The engineering team worked with data scientists to model expected traffic, reviewed historical numbers, added safety margins, and created a master Google Cloud resource plan covering CPUs, storage, machine types, locations, and project-level quotas. Because 2020 brought unusual uncertainty from COVID-19-driven changes in online commerce, Shopify modeled multiple demand scenarios and chose the more aggressive growth cases, while retaining buffers for changes and regional failover. It also ran the Oktoberfest scale-up on Google Kubernetes Engine, using internal load-testing tooling powered by go-lua and embedded Lua to simulate storefront browsing, cart additions, checkout, and mock payment processing. The exercise exposed bottlenecks, which teams fixed before repeating the test; the successful rerun increased confidence for BFCM and reinforced a longer-term aim of more automated, dynamic, and resilient systems.


### [Monitoro Built a Web Crawler Handling Millions of API Requests](https://yomu.fyi/post/monitoro-built-a-web-crawler-handling-millions-of-api-requests.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Published: Dec 2, 2020
- Type: Announcement

Supabase showcases numerous customer case studies illustrating how companies across various industries utilize its platform to build products, migrate infrastructure, and improve operational metrics. Several organizations report substantial cost and efficiency improvements, such as Shotgun achieving an 83% data infrastructure cost reduction and Good Tape saving 60% after moving to managed Postgres and authentication. In terms of developer velocity, Kayhan Space experienced an 8x boost in speed, while Quilia reduced development time by 75% through Data API and Row Level Security features. Other highlighted migrations include Mobbin moving 200,000 users from Firebase, Bree transitioning from Fauna, and Firecrawl switching from Pinecone to Supabase Vector for PostgreSQL vector embeddings.


### [TAYFA Built a No-Code Website Builder in Seven Days](https://yomu.fyi/post/tayfa-built-a-no-code-website-builder-in-seven-days.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Published: Dec 2, 2020
- Type: Explainer

Supabase compiles case studies detailing how organizations across industries build applications, scale infrastructure, and migrate databases. Organizations adopt the platform for capabilities including managed Postgres, Authentication, Row Level Security, Data APIs, and Supabase Vector for PostgreSQL vector embeddings. Reported operational outcomes include Shotgun reducing data infrastructure costs by 83 percent, Quilia cutting development time by 75 percent, and Good Tape achieving a 60 percent cost reduction. Several teams completed platform migrations to Supabase, including Mobbin migrating 200,000 users from Firebase, Bree moving from Fauna, and Firecrawl transitioning from Pinecone. Furthermore, organizations like Replenysh implemented one-time passwords within 24 hours, while Xendit shipped a production solution in under one week.


### [Xendit Built a Counter-Fraud Watchlist for the Fintech Industry](https://yomu.fyi/post/xendit-built-a-counter-fraud-watchlist-for-the-fintech-industry.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Published: Dec 2, 2020
- Type: Problem & solution

Payment processor Xendit needed a fast, cost-effective counter-fraud watchlist system to screen transactions against international sanctions lists in real time without relying on expensive providers like Worldcheck or Refinitiv. Leveraging their existing familiarity with Postgres, the team loaded parsed sanctions lists from the United Nations and the Indonesian government into Supabase. They implemented full-text search using the Postgres Trigram extension, enabling relevance scoring across queried records. A dedicated database function handles the search queries directly via Xendit's Python clients, allowing the entire solution to reach production in under one week. The deployment has operated reliably in production for nine months without issues, with potential future iterations planned around machine learning techniques.


### [Supabase Partners With Strive School To Help Teach Open Source](https://yomu.fyi/post/supabase-partners-with-strive-school-to-help-teach-open-source.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Rory Wilding
- Published: Dec 2, 2020
- Type: Announcement

Supabase has partnered with fellow Y Combinator Summer 20 alumni Strive School to teach upcoming software engineers open-source development practices. Strive School operates an eight-month training program in Europe covering full-stack web and artificial intelligence engineering, with students paying tuition only after securing employment. Through this collaboration, Supabase aims to improve relational database instruction using Postgres while producing exclusive educational content for the program. In addition to creating learning materials, Supabase is hiring a Strive School graduate as an intern to integrate new engineering talent into its team. Supabase is also introducing a Student Developer Pack offering two years of free platform credits, Slack community access, founder office hours, and merchandise.


### [Pummelling the Platform–Performance Testing Shopify](https://yomu.fyi/post/pummelling-the-platform-performance-testing-shopify.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 2, 2020
- Type: Explainer

Shopify prepares for high-traffic sales events such as BFCM by repeatedly simulating customer traffic on representative test shops, aiming to expose bottlenecks before merchants experience them. Its Platform Conditioning team supports service-owning teams with approachable performance-testing tools and processes, distinguishing load tests for known traffic levels from stress tests that push services toward failure. A Go load generator runs Lua flows through lightweight VMs, models realistic traffic shapes and costly endpoints, and can issue tens of millions of requests per minute or more. Tests are initiated through Slack using the internal ChatOps tool Spy, which returns links to dashboards, logs, and results. Cronograma provides formal experiments with hypotheses, observations, conclusions, alert detection, Slack timelines, and comparable metrics such as response times, 5xx errors, and requests per minute; the tools have led to performance improvements and give Shopify confidence in future sales events.


### [Supabase Alpha November 2020](https://yomu.fyi/post/supabase-alpha-november-2020.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Dec 1, 2020
- Type: Announcement

After nine months of development, Supabase introduced several platform updates as it progressed closer to its beta release. The dashboard now supports manual user creation alongside administrative actions such as deleting users and sending password reset emails or magic links. Database management gained direct browser support through a full Monaco SQL editor and new foreign key controls within the table editor. Platform reliability and security were addressed through a DigitalXRAID audit, default email confirmations on signups, and a dedicated status page tracking uptime and latency. Additional progress included community integration work with Vercel, Stripe, and Redwood, as well as an updated benchmarking suite for realistic server workloads.


### [Vouching for Docker Images](https://yomu.fyi/post/vouching-for-docker-images.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 1, 2020
- Type: Explainer

Kubernetes does not enable Docker image signature verification by default, leaving organizations to decide whether images are trusted before they run in a cluster. The post explains code signing through hashes and public/private keys, then describes Binary Authorization as a combination of metadata storage, an enforcer, and a signing service. Using Grafeas and Kritis, or GKE’s Binary Authorization, cluster policies require specified attestors to sign images; invalid or unsigned images are blocked before their containers start. Voucher supplies the missing signing component as a REST service running in Cloud Run or Kubernetes: build pipelines submit image paths, Voucher performs security checks, signs qualifying images, and records signatures in the metadata service. Its extensible checks can verify internal ownership, GitHub commits, approvals, builder identity, and vulnerability counts, while distinct cluster policies balance compliance requirements with faster experimentation; Voucher later moved into the Grafeas organization.


### [The November Fly Changelog](https://yomu.fyi/post/the-november-fly-changelog.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Nov 30, 2020
- Type: Announcement

The November 2020 Fly changelog records new example applications, Fly CLI enhancements, and platform updates. Guides now cover Gogs, MQTT with Mosquitto, MinIO, Node-RED, and Redis with and without TLS, and demonstrate preview persistent Volumes for database persistence, data storage, preserved programming state, and queued messages. The CLI added bulk secret imports from key=value files, including multiline values enclosed in triple double quotes, while list commands gained filtering, sorting, and JSON output; the Deno builtin added a Version setting defaulting to 1.5.4. Platform changes included an upgrade to Firecracker v0.23, improved restart and connection handling, timeout fixes, and stronger application-context isolation. The changelog also notes new crash-log context, humanized app times, improved image resolution, and corrected internal port and metrics handling.


### [More Appkata Examples to Try on Fly](https://yomu.fyi/post/more-appkata-examples-to-try-on-fly.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Nov 27, 2020
- Type: Announcement

Two new Appkata examples show how to run selected applications on Fly: Node-RED and Redis secured with TLS. Node-RED is a GUI flow-programming environment built on Node, with modules for connecting MQTT, Redis, Graphana, and other systems, plus creating web dashboards; the guide is intended to get it running quickly. The secured Redis example uses mkcert to simplify creation of TLS certificates, extending an earlier Redis example that presented Redis as a simple service. The post also points to Community.fly.io as a place for questions, hints, and tips, noting that Fly accounts can use single sign-on to make participation easier.


### [Democratising Fare Storage at Scale Using Event Sourcing](https://yomu.fyi/post/democratising-fare-storage-at-scale-using-event-sourcing.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Sourabh Suman
- Published: Nov 23, 2020
- Type: Problem & solution

Grab's legacy system stored booking and fare details in a single relational table, creating a bloated booking entity that tracked only the latest fare state and hindered rapid feature iteration. To resolve scalability, stability, and debugging challenges across millions of daily bookings, the team developed Fare Storage using the Event Sourcing pattern. The new architecture persists all fare modification events chronologically in DynamoDB, backed by a cache for eventually consistent reads and message streaming for downstream processing. The platform employs optimistic locking with versioning to manage concurrent updates, enforces idempotency through client-generated transaction UUIDs, and delegates metadata serialization to an SDK to prevent storage API changes.


### [Static Typing for Ruby](https://yomu.fyi/post/static-typing-for-ruby.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 19, 2020
- Type: Explainer

Shopify’s Ruby monolith spans 37,000 files, 622,000 methods, and more than 2,000,000 calls, making fast feedback and stability difficult despite rigorous reviews and 150,000 automated tests. Sorbet was selected after the team evaluated requirements for gradual typing, speed, and support for Ruby and Rails features including metaprogramming, overloading, and class reopening. Its RBI files represent constructs it cannot infer, while per-file sigils allow adoption to progress without blocking development, and SorbetMetrics tracks sigils, typed calls, and method signatures. Shopify treats typing as a product, combining CI enforcement, developer support, surveys, and interviews to guide rollout and measure sentiment. The excerpt reports 80% of monolith files, including tests, at typed: true or higher, with almost half of calls and methods covered and type checking under 15 seconds on developer machines.


### [Adopting Sorbet at Scale](https://yomu.fyi/post/adopting-sorbet-at-scale.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 19, 2020
- Type: Explainer

Shopify describes its adoption of Sorbet static typing across a Ruby monolith containing 37,000 files, 622,000 methods, and more than 2,000,000 calls. At the time of writing, Sorbet ran on every pull request, 80% of files were typed: true or higher, almost half of calls were typed, and half of methods had signatures. To handle Ruby and Rails idioms, the team built RuboCop Sorbet for compatibility rules, Tapioca for gem and DSL RBI generation, and Spoom for programmatic tooling, metrics, and LSP access. A controlled experiment found fewer production NoMethodErrors in files typed: true after typing about 20% of the application, although the results were preliminary and signatures had not yet been added. Shopify planned to reach typed: true across all files, improve Rails support, and continue collaboration around Sorbet and RBS.


### [How to Build a Production Grade Workflow with SQL Modelling](https://yomu.fyi/post/how-to-build-a-production-grade-workflow-with-sql-modelling.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 19, 2020
- Type: Problem & solution

Shopify’s Starscream data platform handled 76,000 jobs and wrote 300 terabytes daily, but its PySpark workflow was slow and cumbersome for commonplace reporting. An audit found that about 70 percent of PySpark jobs were full batch queries that did not require generalized computing, while users often wrote SQL before translating it into Python. The team built Seamster with dbt and Google BigQuery, adding source and model structures, ownership metadata, Python-based unit tests with fixed mock inputs, CI validation, and warehouse rules. A base layer isolates raw-source changes, while models are organized into base, application-ready, and presentation layers and defined through SQL, schema, README, and test files. After a multiweek beta, most users reported shipping models in days rather than weeks, although dbt’s incremental support could not safely handle some late-arriving-data, key-resolution, rebuild, and very large-model cases.


### [Postgres Views](https://yomu.fyi/post/postgres-views.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Nov 18, 2020
- Type: Explainer

Postgres views serve as query shortcuts that execute underlying SQL statements upon retrieval without generating new tables or persisting duplicate data. By encapsulating complex multi-table joins, standard views provide query consistency across applications, simplify repetitive calls, improve logical schema organization, and enhance security by restricting sensitive columns. In contrast, materialized views physically store query results on disk, dramatically reducing read latency for heavy queries spanning millions of rows. Because materialized views introduce the trade-off of stale data, administrators must periodically run the refresh command based on workload tolerances for use cases like analytics and internal dashboards. Materialized views should not substitute query optimization, as underlying query efficiency remains essential.


### [Appkata - Example Apps on Fly](https://yomu.fyi/post/appkata-example-apps-on-fly.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Nov 18, 2020
- Type: Tutorial

Appkata introduces four example backend apps for Fly, intended to show how to deploy common services while learning the platform’s deployment patterns. The examples cover Redis as a customizable, volume-backed key/value database; MinIO as a single-node, S3-compatible object store exposed through Fly’s TLS edge connections; Gogs as a private Git server using SQLite3 and Fly volumes; and Mosquitto as an MQTT broker with TLS, username/password authentication, and volume-backed message persistence. The examples emphasize adapting existing Docker images, attaching persistent storage, and exposing services securely. The post presents these as an initial set that will expand over the following weeks, alongside a Fly Community site for questions and guidance.


### [Leveraging Pre-trained Language Model Checkpoints for Encoder-Decoder Models](https://yomu.fyi/post/leveraging-pre-trained-language-model-checkpoints-for-encoder-decoder.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Patrick von Platen
- Published: Nov 9, 2020
- Type: Tutorial

Pre-training sequence-to-sequence transformer models incurs massive computational costs, limiting development primarily to large institutions. To mitigate these expenses, encoder-decoder architectures can be warm-started using existing pre-trained checkpoints from encoder-only or decoder-only models like BERT and GPT-2. This walkthrough details the methodology and implementation of warm-starting sequence-to-sequence architectures using Hugging Face Transformers. By utilizing the EncoderDecoderModel framework alongside Seq2SeqTrainer, practitioners can construct and fine-tune models such as BERT2BERT on datasets like CNN/DailyMail. The resulting fully trained BERT2BERT model achieves a ROUGE-2 score of 18.22 on the full evaluation set, matching competitive sequence generation baselines at a fraction of standard pre-training costs.


### [Porting fairseq wmt19 translation system to transformers](https://yomu.fyi/post/porting-fairseq-wmt19-translation-system-to-transformers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Stas Bekman
- Published: Nov 3, 2020
- Type: Tutorial

The fairseq WMT19 news translation system was ported to the Hugging Face transformers library under the FSMT architecture. Development began with an initial proxy notebook emulating the transformers API before implementing native configuration, modeling, tokenization, and checkpoint conversion scripts. The author first targeted English-Russian models to handle complex dual-vocabulary requirements and directly evaluate output quality during testing, later extending the implementation to merged-vocabulary language pairs. Although model ensembling was omitted due to library limitations, removing fairseq optimizer states reduced the final model download footprint from 13GB to 1.1GB. The implementation was validated locally against converted fairseq checkpoints and merged into the main codebase after two weeks of iterative review.


### [Supabase Alpha October 2020](https://yomu.fyi/post/supabase-alpha-october-2020.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Nov 2, 2020
- Type: Announcement

Eight months into building Supabase, the team released product updates focused on stability, reliability, and performance while preparing to transition from Alpha to Beta. The release introduces supabase-js version 1.0, incorporating extensive community feedback to deliver major developer experience improvements across client libraries. Platform interface updates include tabs and full-screen mode for the SQL Editor, alongside new keyboard shortcuts to help power users navigate and manipulate tables rapidly. For user authentication, Supabase added magic link capabilities, enabling passwordless login flows for both new and existing users either as a standalone method or alongside passwords. Under continuous improvement efforts, the team converted realtime-js to TypeScript, heavily optimized dashboard routing performance, and established a repository to benchmark all integrated open source tools.


[Newer posts](https://yomu.fyi/posts/page/227.md) · [Older posts](https://yomu.fyi/posts/page/229.md)