---
title: "Latest reads"
description: "The engineering internet, summarised so you can actually read it."
---

# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [Supabase Wrappers, a Postgres FDW framework written in Rust](https://yomu.fyi/post/supabase-wrappers-a-postgres-fdw-framework-written-in-rust.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: paul\_copplestone,bo\_lu,oli\_rice
- Published: Dec 15, 2022
- Type: Announcement

Supabase Wrappers is an Alpha framework for building PostgreSQL Foreign Data Wrappers that connect databases with external systems, including third-party APIs, through SQL. It initially supports Firebase and Stripe, while wrappers for ClickHouse, BigQuery, and Airtable are under development. The framework extends pgx, a Rust Postgres extension framework, with async support for RESTful API-based wrappers and limited push-down support currently provided as a proof of concept. The post presents Wrappers as a basis for QETL, allowing teams to query, extract, transform, and load remote data through Postgres, either on demand or into local tables, while joining it with operational data. It is released outside official platform support, with potential breaking changes.


### [Launching Redis by Upstash](https://yomu.fyi/post/launching-redis-by-upstash.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Dec 15, 2022
- Type: Announcement

Fly.io and Upstash launched a managed Redis service positioned close to Fly.io applications, with global replicas, a free tier, and private encrypted IPv6 connectivity. The integration uses Flycast, Machines API, WireGuard networking, and PROXY protocol headers: flyctl provisions a Flycast IP, routes it through global proxies to the nearest healthy Redis instance, and lets multitenant Redis identify the customer database without exposing its password. Applications can add replicas and write forwarding through flyctl while retaining a single Redis URI; writes go to the primary and reads and writes use geographically nearest replicas. The post illustrates edge HTML caching for a Rails recipe app, reporting low-millisecond global time to first byte for cached GET requests. It also describes read-your-own-write handling, including a 500ms replication wait and fallback reads from the source of truth.


### [Multi-factor Authentication via Row Level Security Enforcement](https://yomu.fyi/post/multi-factor-authentication-via-row-level-security-enforcement.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: joel,kangmingtay,stojan
- Published: Dec 14, 2022
- Type: Announcement

Supabase announces the release of multi-factor authentication (MFA) for everyone and describes dashboard Single Sign-On (SSO) as preparation for a future SAML release; SSO is available on Enterprise plans by contacting the company. The implementation starts with Time-Based One-Time Passwords (TOTP), where an authenticator generates a short-lived code from a shared secret and the current time, exchanged through a QR code or entered manually. Enrollment uses /enroll, /challenge, and /verify, while subsequent sign-ins require a challenge and verification before a JWT is issued with Authenticator Assurance Level (AAL) and Authenticator Method Reference (AMR) claims. Those claims support full or selective enforcement through restrictive Postgres Row Level Security policies, including checks that require aal2 or require it only for users with verified MFA factors; planned additions include WebAuthn, FIDO2 devices, and email notifications.


### [Shopify Embraces Rust for Systems Programming](https://yomu.fyi/post/shopify-embraces-rust-for-systems-programming.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 14, 2022
- Type: Announcement

Shopify has adopted Rust as its official systems programming language while retaining Ruby for server-side business logic, and has joined the Rust Foundation. The change addresses growing systems-programming needs including high-performance network servers, Ruby native extensions, and WebAssembly, where Shopify seeks speed, productivity, safety, and community-driven open source. Shopify cites Rust’s predictable native-code performance, fine-grained memory control, compile-time safety, support for concurrency, and interoperability with C and Ruby through tools and crates such as bindgen, rb-sys, and magnus. The company says Rust performed strongly in its safety evaluation and that its initial projects exposed more errors at compile time, while acknowledging that performance still depends on design and measurement and that internal education, tooling, and community participation remain unfinished work.


### [Supabase Storage v2: Image resizing and Smart CDN](https://yomu.fyi/post/supabase-storage-v2-image-resizing-and-smart-cdn.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: fabrizio,inian
- Published: Dec 13, 2022
- Type: Announcement

Supabase Storage introduces image resizing, webhooks, and a Smart CDN, positioning them as components of an on-demand image delivery system. Resizing uses imgproxy and supports width, height, and contain, cover, or fill modes through JavaScript clients or direct Storage API requests; images are not enlarged beyond their original dimensions, and aspect ratio is preserved by default. The Smart CDN stores resized results using cache keys based on an Etag, tenant ID, bucket, and image name, while webhook-driven metadata updates propagate invalidations globally within about 60 seconds. Image Resizing is beta for Pro and Enterprise customers, with 100 transformed images per project at no additional cost and $5 per additional 1,000; self-hosting is free.


### [3 (More) Tips for Optimizing Apache Flink Applications](https://yomu.fyi/post/3-more-tips-for-optimizing-apache-flink-applications.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 13, 2022
- Type: Tutorial

Shopify presents three additional practices for optimizing large stateful Apache Flink applications, focusing on parallelism, sink capacity, and combining heterogeneous sources. It recommends starting with execution-environment parallelism, then matching task-manager capacity to the highest parallelism value; for example, parallelism 100 requires 25 task managers with four slots each. Sink bottlenecks can propagate backpressure upstream, so batching writes and correcting data skew—including with key bucketing when a low-cardinality or uneven key is unavoidable—can improve distribution, though bucketed results must later be combined. For archived Kafka topics, HybridSource exposes cloud-storage history and the live Kafka topic as one ordered logical DataStream, automatically switching after the archive is exhausted and using object-storage partitions to accelerate historical backfills.


### [New Supabase Docs, built with Next.js](https://yomu.fyi/post/new-supabase-docs-built-with-next-js.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: jonny
- Published: Dec 12, 2022
- Type: Announcement

Supabase released a new version of its documentation, rebuilding the site with Next.js after outgrowing Docusaurus. The earlier setup became harder to maintain as client-library versioning expanded, contributions grew more difficult, and the docs remained separate from Supabase’s Next.js and Turborepo monorepo. To reduce migration risk, the team first created a one-week Next.js MVP that copied Docusaurus, then spent a month developing the new release with two full-time developers. It introduces a new design, a single sidebar for broader discoverability, and reference pages that group all functions in a library while improving TypeDoc-based generation. Future work described includes unified authentication, project-specific documentation, runnable Postgres WASM examples, interactive tutorials, improved Postgres full-text search, and balancing sparse and overly verbose content.


### [pg\_crdt - an experimental CRDT extension for Postgres](https://yomu.fyi/post/pg-crdt-an-experimental-crdt-extension-for-postgres.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: paul\_copplestone
- Published: Dec 10, 2022
- Type: Announcement

Supabase announces pg\_crdt, an experimental Postgres extension that adds CRDT support as a data type for collaborative and offline-oriented applications. It frames CRDTs as structures whose updates can be merged in any order to reach the same final state, avoiding the last-write-wins behavior of ordinary arrays. A content column can use crdt.ydoc, backed by Yjs, while the extension also supports Automerge through Rust libraries wrapped with pgx. The post contrasts storing CRDTs directly in Postgres with making Realtime an authoritative server, favoring the database-centered approach for simpler architecture and broader tooling, while noting that the decision is unsettled and pg\_crdt is not released on the Supabase platform. Known limitations include full-document WAL broadcasts, increased WAL and dead tuples for frequently updated documents, and serialization overhead for large CRDT values.


### [Launch Week 6 Hackathon](https://yomu.fyi/post/launch-week-6-hackathon.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: ant\_wilson
- Published: Dec 9, 2022
- Type: Announcement

Supabase announces its official Launch Week 6 Hackathon, which starts at 8:00am Pacific Time on Friday 9th December 2022 and ends at 00:01am PT on Monday 19th December. The event gives individuals or teams of up to five people ten days to create a new Open Source project using Supabase. Entries may be projects, apps, tools, or libraries in any language or framework, but all design elements and code must be created during the event and each team may submit once. Five categories offer winner and runner-up prizes, including $1500 in GitHub sponsorships paid as three monthly $500 sponsorships for the best overall project, plus Supabase keyboards for its team members. Supabase judges the main categories, while Deno judges Best Edge Functions Project; judging considers creativity, smooth functionality, visual appeal, technical impressiveness, and use of Supabase features.


### [Who We Hire at Supabase](https://yomu.fyi/post/who-we-hire-at-supabase.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: ant\_wilson
- Published: Dec 9, 2022
- Type: Explainer

Supabase describes the hiring traits and operating model it believes support its growth, arguing that people matter more than methodology. It looks for extreme asynchronous effectiveness, egoless execution, continuous incremental improvement, a bias toward immediate action, ambitious problem-solving, and a founder mindset; seniority does not exempt anyone from independent-contributor work, self-management, or maintenance. The 60-person team spans 25 countries, uses meetings as a last resort, and gives autonomous product teams access to shared metrics, user feedback, and equity through an ESOP. Teams are organised around products such as Postgres, Auth, Storage, Realtime, and Functions, while cross-cutting groups consult rather than merely receive delegated tasks. Three-month product cycles set ambitious goals, and Kaizen Weeks focused on QA, documentation, and performance help address technical debt and incorporate improvements into ongoing workflows.


### [Three Essential Remote Work Practices for Engineering Teams](https://yomu.fyi/post/three-essential-remote-work-practices-for-engineering-teams.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 8, 2022
- Type: Tutorial

After Shopify became a fully remote-first company, its engineering organization refined how teams communicate, collaborate, and work across time zones. The recommended team structure concentrates engineers in related time-zone regions to maximize synchronous availability for discussion, pair programming, mentorship, and questions, while relying on asynchronous writing and documentation between teams. Teams are also asked to define communication norms that specify which medium to use, expected response times, and whether contact is appropriate after hours; documenting these rules reduces ambiguity and protects focus. Individuals should intentionally shape routines and work-life boundaries around their own energy, productivity, health, and responsibilities, then integrate those preferences with team norms. Together, these practices support flexible remote work by combining synchronous collaboration within teams, asynchronous coordination across teams, and greater control over the working day.


### [Supabase Beta November 2022](https://yomu.fyi/post/supabase-beta-november-2022.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: ant\_wilson
- Published: Dec 7, 2022
- Type: Announcement

Supabase’s November 2022 update previews Launch Week 6 while reporting a smaller set of product releases and community activity. Remix Auth Helpers are now available for server-side authentication, updated for supabase-js V2 and usable with TypeScript; the post also introduces the Edgy Edge Functions YouTube series and three examples covering OG image generation, Oak API servers, and Puppeteer screenshots. Product fixes address auth.signOut cookies and session removal, magic links can be customized with {{ .TokenHash }}, functions support redirects, and Supabase can be installed through Conda. The update also announces a Fireship course building a Reddit-inspired app with Supabase, PostgreSQL, and React, alongside a broad list of community resources and projects.


### [Zero trust with Kafka](https://yomu.fyi/post/zero-trust-with-kafka.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Fabrice Harbulot
- Published: Dec 7, 2022
- Type: Problem & solution

Grab's real-time data platform team transitioned their large-scale Kafka infrastructure from basic network access controls to a zero-trust architecture. The platform implements mutual Transport Layer Security (mTLS) for offline peer authentication and encryption, driven by HashiCorp Vault's PKI engine and Strimzi on Kubernetes. Policy-Based Access Control is enforced using dedicated Open Policy Agent deployments per cluster, backed by GitOps workflows where topic owners approve JSON authorization rules. To simplify client integration, the team enhanced their Go SDK to handle ephemeral in-memory certificates, automatic renewals, and configurable retries. While the security posture improved, the Java encryption and decryption overhead caused a drop in streaming throughput.


### [The Supabase Content Storm](https://yomu.fyi/post/the-supabase-content-storm.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: ramiro
- Published: Dec 6, 2022
- Type: Announcement

The post announces Supabase’s Content Storm, a community campaign created to promote Launch Week 6 through a coordinated release of external content. Supabase invited well-known YouTubers, community members, makers, the SupaSquad, and others building with Supabase to publish related material on the same day, while giving participants an opportunity to build personal brands or promote products and startups. More than 30 creators took part, producing videos, livestreams, tutorials, project write-ups, and a Twitter thread covering topics such as authentication, migrations, Edge Functions, Storage, GraphQL, Flutter, Rust, and TypeScript integrations. The content was published on Tuesday, December 6, and the post provides a complete list of works and authors while directing readers to bookmark the collection, give feedback, and follow contributors.


### [Logbook: November 14 to December 5, 2022](https://yomu.fyi/post/logbook-november-14-to-december-5-2022.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Dec 6, 2022
- Type: Announcement

An edition of Fly’s Logbook covering November 14 to December 5, 2022 collects updates and tutorials for building and operating applications on Fly. It features a real-time collaboration example that connects Replicache with WebSockets and Fly-Replay, plus a LiveWire walkthrough for streaming server content to browsers and another for unloading data during large-dataset pagination. A separate guide explains how to run Supercronic from Fly Dockerfiles when precise cron timing is needed beyond Fly Machines’ monthly, weekly, daily, and hourly scheduling options. The edition also recounts how Fly Postgres was built, despite Fly’s stated preference for off-platform database services such as RDS, CrunchyData, and PlanetScale. The post notes that the update followed a longer break while many Fly staff spent time with family over Thanksgiving.


### [Planning in Bets: Risk Mitigation at Scale](https://yomu.fyi/post/planning-in-bets-risk-mitigation-at-scale.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Dec 6, 2022
- Type: Explainer

Preparing for BFCM and other high-traffic events, Shopify describes risk mitigation for a globally distributed, complex, interconnected infrastructure platform that peaked at 75.98M requests per minute, or 1.27M per second. At this scale, identifying and mitigating every possible failure is impossible, so the process starts with “what could go wrong” exercises, then uses voting, expert review of likelihood and severity, and assigned ownership to prioritize action. Decision makers are empowered to weigh risks and rewards, while summarized findings, including key risks and single points of failure, are shared to maintain alignment and awareness. The post frames engineering choices as probabilistic bets: decision quality should be judged separately from outcomes, and uncertainty cannot be eliminated, only addressed through thoughtful prioritization and communication.


### [How KartaCam powers GrabMaps](https://yomu.fyi/post/how-kartacam-powers-grabmaps.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Shuangquan Hou
- Published: Dec 1, 2022
- Type: Problem & solution

Grab's Geo team required an efficient, low-cost way to collect fresh street-level imagery across Southeast Asia, where professional mapping equipment is prohibitively expensive and smartphone crowdsourcing yields inconsistent quality. To address this, Grab built KartaCam, a custom mapping device equipped with a 12MP sensor, dual-band GNSS, 4G LTE, and onboard edge AI. Edge machine learning models evaluate scene suitability, check image quality, filter for map-relevant objects, and blur personal data directly on the device prior to upload. Deploying a four-camera KartaCam 360 array delivers panoramic coverage and point-of-interest data comparable to commercial mapping rigs at roughly one-twentieth the hardware cost.


### [Using Server Sent Events to Simplify Real-time Streaming at Scale](https://yomu.fyi/post/using-server-sent-events-to-simplify-real-time-streaming-at-scale.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 30, 2022
- Type: Problem & solution

Shopify’s 2022 BFCM Live Map needed to deliver real-time sales and product data to thousands of concurrent users while processing data from millions of merchants. Its 2021 architecture used WebSocket delivery to a presentation layer, which stored messages in a mailbox that clients polled at least every 10 seconds, creating latency and bottlenecks. Shopify replaced that path with a streamlined Flink-based pipeline and a Golang SSE server that subscribes to Kafka topics and pushes JSON data to registered, authenticated clients as soon as it is available. The SSE server ran horizontally behind Nginx load balancers, with a configurable Java client used to load-test concurrent connections. The 2022 system maintained 100 percent uptime, delivered SSE data within milliseconds of availability, and visualized data within 21 seconds of creation, including pipeline processing.


### [How We Built Fly Postgres](https://yomu.fyi/post/how-we-built-fly-postgres.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Nov 29, 2022
- Type: Explainer

Fly.io describes how it built Fly Postgres as an automated database app rather than a managed database service. After initially resisting durable storage and database work, the company added attached disk storage and designed a deployable Postgres application with convenience commands, using Postgres streaming replication, Stolon, and Consul to support high availability. A two-node cluster can designate a writable leader and asynchronous standby, while Stolon’s sentinels, keepers, and proxies coordinate state, elect a replacement leader, and route connections; Consul introduces an additional failure mode. Backups use point-in-time, block-level snapshots of the volume, a platform capability that can also support other file-system databases. The post concludes that automated Postgres serves many apps, while managed database partnerships may suit users seeking a simpler experience.


### [How to Export Datadog Metrics for Exploration in Jupyter Notebooks](https://yomu.fyi/post/how-to-export-datadog-metrics-for-exploration-in-jupyter-notebooks.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Nov 29, 2022
- Type: Tutorial

Datadog dashboards can constrain metric analysis because they offer a limited set of visualizations and lack tooling for complex work such as statistical modeling. They also aggregate data over wider time ranges: the examples contrast one-second metrics across 15 minutes with two-hour intervals across 30 days, potentially hiding interesting events. The guide uses Datadog’s REST API and Python in Jupyter notebooks, requiring an API key, an APP key, and a metric query such as CPU utilization over time. It defines a time range, splits it into buckets whose width is controlled by time\_delta, requests each window in a loop, appends the results, and converts them into a dataframe. The resulting data can be examined with tools such as seaborn; a KDE plot is used to inspect the distribution of system CPU utilization, with the exported data offering greater granularity than the dashboard example.


[Newer posts](https://yomu.fyi/posts/page/200.md) · [Older posts](https://yomu.fyi/posts/page/202.md)