Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,535 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Supabase Beta February 2023
Supabase’s February 2023 beta update covers product changes across its dashboard, Auth, Postgres, Edge Functions, GraphQL, documentation, and community resources. GraphiQL is now built into the dashboard for exploring and querying pg_graphql APIs, while Docs search uses pgvector and OpenAI to support Supabase-specific questions. Supabase CLI 1.36+ can serve multiple Edge Functions simultaneously, and a multi-stage Postgres Dockerfile reduces the image from 1.3GB to 250MB, enabling faster boot time. The release also adds dashboard management for database roles, moves API documentation into the table editor with type-file generation, and provides a full OpenAPI 3.0 specification for Auth. Additional updates include inferred supabase-js response types, Deno 1.30.3 for Edge Functions, and Realtime behavior for deletes under RLS.
Migrating from Role to Attribute-based Access Control
Grab's streaming data platform team migrated the Kafka Control Plane from Role-Based Access Control to Attribute-Based Access Control to eliminate operational bottlenecks and manual permission management. The previous model required defining hundreds of roles, permissions, and group mappings in an internal IAM service, leading to approval delays and stale memberships. Under the new architecture, user attributes sync from the HRMS and token payloads, while resource attributes are tagged upon creation or backfilled to reflect department and team ownership. Open Policy Agent evaluates access requests defined in Rego via middleware by comparing user attributes with resource metadata. This transition eliminated over 200 roles, 200 permissions, and roughly 3,000 unused IAM resources while automating access provisioning for new joiners.
Responsibly mid-size: What Matters, Suffers
Moving into a mid-size company requires adopting processes and attitudes piecemeal rather than suddenly operating like a large enterprise, while preserving the ability to deliver customer value. The post challenges “clean code” as a universal standard, arguing from Sandi Metz’s discussion of churn and semantic complexity that important, frequently changed files are often large and complicated, making “bad code” an emergent feature of successful systems. It also uses Joel Spolsky’s case against rewrites to explain that ugly code may contain accumulated fixes for real bugs and operating conditions. The resulting advice is to distinguish code that manages responsibilities from code that does not, develop engineering judgment, and avoid letting aesthetic reactions damage collaboration, team cohesion, or delivery.
Geo Queries with PostGIS in Ionic Angular
This tutorial shows how to build an Ionic Angular cross-platform app that stores and queries store locations with Supabase and PostGIS. It enables PostGIS, defines a stores table with a geography(POINT) location, adds a GiST spatial index, creates storage policies, and seeds example coordinates. SQL functions named nearby_stores and stores_in_view calculate distances or select stores inside a coordinate box, while Ionic uses Capacitor Google Maps and geolocation to display markers and query nearby data. The app also uploads store images to Supabase Storage, retrieves them with a 300-pixel contain transformation, and opens store details in an Ionic bottom-sheet modal after marker selection. The completed code is provided on GitHub and requires project and Google Maps credentials plus the included SQL.
Securing GitOps pipelines
Grab's real-time data platform team transitioned from an Atlantis-driven Terraform workflow to an in-house GitOps platform called Khone to manage streaming infrastructure resources like Kafka topics and Flink pipelines. The earlier setup suffered from coarse-grained access controls, required manual merge request comments, and lacked flexible validation capabilities within native configuration files. Khone derives environment parameters directly from standardized directory paths and uses Python with the python-hcl2 library to inspect and validate resource definitions before executing Terraform stages in parallel. To prevent configuration tampering in merge requests, CI/CD pipeline definitions and execution scripts are isolated in a separate administrative repository and fetched during job runs using shallow Git clones.
Improving the Developer Experience with the Ruby LSP
The Ruby LSP is a language server intended to improve Ruby’s developer experience by providing editor features through the Language Server Protocol, without typechecking code or requiring type annotations. It targets performance and stability in Shopify’s Core monolith, where it must analyze thousands of files that may themselves contain thousands of lines, and is opinionated toward Ruby on Rails. The article explains LSP’s client-server model, in which editors exchange JSON requests and responses over STDIN and STDOUT, then focuses on nonpositional features such as Folding Range, including document synchronization and parsing. It also outlines future work on parallel request processing, codebase indexing, caching, and a plugin system for framework-specific capabilities such as Rails navigation and model information.
Gossip Glomers
Gossip Glomers is a series of distributed-systems programming challenges created by Fly.io and Kyle Kingsbury, author of Jepsen, to connect practical coding with distributed-systems theory. Each challenge supplies a workload, a simulated network that can partition or delay messages, and a Jepsen-based verification system; participants write the node binary. The progression covers Echo, a coordination-free unique ID generator, fault-tolerant broadcast, a grow-only counter over Maelstrom’s sequentially consistent key/value store, a Kafka-like replicated log over a linearizable store, and totally available transactions across consistency levels. Initially developed as a resume-blind work-sample evaluation for staff-engineer hiring, the challenges were released publicly because the creators found them too useful and enjoyable to keep internal, with the hardest challenge remaining part of Fly.io’s staff-engineer evaluation.
New zoom freezing feature for Geohash plugin
Grab introduced a zoom freeze capability to its Geohash plugin for the Java OpenStreetMap Editor (JOSM). Previously, zooming in or out forced the plugin to calculate and render new geohashes automatically, causing visual clutter when users needed to focus on a specific region. The updated interface introduces a toggle label allowing editors to switch between enabling and disabling zoom freezing. While frozen, the current geohash boundaries remain locked during map scaling, and options to show smaller or larger geohashes are disabled. Disabling the feature instantly updates the display with the appropriate geohash subdivisions for the current zoom level.
Shipping Logs
Shipping Logs describes how Fly.io application stdout moves through the platform and how to centralize and retain it beyond Fly.io’s log lifetime. Apps run in Firecracker VMs, where an injected Rust init process captures stdout and sends it through host-side Go code to Vector, which publishes the logs to an internal NATS cluster; Flaps limits access to an organization’s logs. The Fly Log Shipper runs Vector as a NATS client and forwards organization-wide streams to a configured sink such as Logtail, Loki, Datadog, or Cloudwatch, using secrets for provider credentials. Deployment involves launching the image, removing the generated services section, adding an internal port, and running fly deploy. A SUBJECT environment variable can narrow collection with NATS subjects such as logs.sandwich.>, covering an app across regions and instances.
The Case Against Monkey Patching, From a Rails Core Team Member
The post argues that Ruby monkey patches—global changes to existing objects or libraries—should be used sparingly because they are brittle, dangerous, and often unnecessary. They can make Rails and Ruby upgrades harder by depending on private APIs, leave applications exposed to vulnerabilities if security fixes are not mirrored, and accumulate poorly documented technical debt. One Shopify example, activerecord-pedant-adapter, changes Active Record’s MySQL2Adapter so database connections report MySQL query warnings; upstreaming that behavior into Rails allowed the gem to be archived and the patch removed. The recommended alternative is to investigate whether an upgrade or documentation resolves the issue, fix genuine defects upstream, and, when patching is unavoidable, document and test it while maintaining a removal plan.
Type Constraints in 65 lines of SQL
PostgreSQL’s extensible type system can encode validation and presentation behavior for a custom Semantic Versioning (SemVer) type. The article stores major, minor, patch, pre-release, and build-metadata components in a composite type, then wraps that type in a domain whose check constraint rejects null or negative numeric components and invalid array elements. A SQL helper validates nullable arrays with regular expressions, and the domain automatically enforces these rules when values are inserted into a table. For ergonomics, a function formats the composite value as a SemVer string, while a stored generated column persists that representation; the article also notes possible parsers, operators, and aggregation functions. It concludes that the ceremony is worthwhile when data integrity is critical and the specification is stable or changes infrequently.
HappyTeams unlocks better performance and reduces cost with Supabase
HappyTeams, a bootstrapped startup, needed its database to support thousands of users and continued feature development without consuming scarce technical resources. Its Heroku Postgres deployment had developed scaling problems, including sluggish or uneven performance and limits on concurrent connections, prompting a database change during an application rebuild in early 2022. The company migrated to Supabase in less than an hour using pg_dump and pg_restore, and adopted PgBouncer for connection pooling; it selected the service for performance, connection flexibility, support and documentation, onboarding, and lower costs. The source reports that Supabase removed the performance issue, made pooling effortless, and helped HappyTeams release features faster, including a comments feature completed in two days, while the company considered expanding into Edge Functions and the API client.
The 25 Percent Rule for Tackling Technical Debt
Technical debt is framed as four categories that need different responses: yearly debt may involve rewrites or gradual replacement, monthly debt requires team-sized planning, weekly debt can become a sprint card, and daily debt is code tidying during ordinary work. The article cautions that some apparent yearly debt reflects growth or changing markets rather than a past mistake, while smaller debt can be deferred only if it is recorded and scheduled. The proposed 25-percent allocation gives 10% to daily improvements, 10% to weekly work, and 5% to planning monthly and yearly debt, with the latter focused on prioritization rather than implementation. Its conclusion is that sustainable remediation depends on team culture: engineers need permission and encouragement to refactor, teams need space to schedule broader work, and leaders need to weigh costs and tradeoffs.
How to build a real-time multiplayer game with Flutter Flame
The tutorial builds a real-time multiplayer shooting game with Flutter, the Flame game engine, and Supabase Realtime. Each player controls a UFO by dragging, while the game automatically fires bullets in three directions; opponents’ positions and health points are synchronized over Supabase’s low-latency web socket connection. The implementation starts with Flutter widgets for the game page and lobby, then defines Flame components for the game, players, bullets, collision detection, movement, and health changes, before wiring Supabase broadcast events to client updates. A lobby waits for at least another player, and starting a match creates a shared channel so both clients can begin together. The completed application supports gameplay and a post-game dialog, allowing players to return to the lobby without managing their own networking infrastructure.
ShopifyQL Notebooks: Simplifying Querying with Commerce Data Models
ShopifyQL Notebooks gives merchants a self-serve way to investigate changing business metrics, understand their causes, and share findings as data stories. It combines the ShopifyQL query language with commerce-specific data models designed for users without SQL expertise, covering domains such as products, orders, and customers. The modeling process starts with business questions, translates them into columns and an appropriate grain, validates the design with mock data and queries, and evaluates upstream data, freshness, and performance. To support large merchants, the implementation uses incremental models and roll-ups, which can reduce build and query times while adding maintenance, pipeline, and monitoring complexity.
Bringing Javascript to WebAssembly for Shopify Functions
Shopify is introducing a Local Developer Preview that lets developers run JavaScript Shopify Functions locally, alongside Rust, while production deployment remains unavailable. Shopify Functions execute as WASI modules that read JSON from stdin and write JSON to stdout, with a 256KB size limit and a five-millisecond runtime limit; the post notes that the timing constraint varies by machine load. Javy addresses JavaScript’s WebAssembly runtime challenge by bundling the QuickJS interpreter into a Wasm binary, using Rust wrappers and Wizer to snapshot compiled QuickJS bytecode. Initial tests found Javy-created modules about three times slower than equivalent Rust modules, but both are likely to stay under five milliseconds for realistic use cases; Shopify is also pursuing SpiderMonkey in Wasm and researching JIT support for future performance.
Supabase Beta January 2023
Supabase’s January 2023 beta update gathers releases across Postgres, GraphQL, Edge Functions, client libraries, image transformation, and self-hosting. The update adds pgvector, a PostgreSQL extension for storing OpenAI embeddings and performing vector similarity search, and presents Supabase Clippy, a one-week, fully open-source documentation-search MVP combining pgvector with OpenAI. pg_graphql now supports Views, Materialized Views, and Foreign Tables, while Image Transformations enable automatic WebP conversion by default, with format: origin available to opt out. Other changes include Python and C# reference documentation, pg_repack, multi-tab authentication logout through BroadcastChannel, lz4 compression, ICU locales, downloadable Edge Function source, and a DigitalOcean deployment guide using Sendgrid, Packer, and Terraform. Supabase also reports reaching 45,000 GitHub stars and a position among the top 200 most-popular repositories.
The Complex Data Models Behind Shopify's Tax Insights Feature
Shopify Tax’s Tax Insights feature provides merchants with state-level information about sales-tax liability and approaching liability, amid U.S. rules that vary by jurisdiction, product, threshold, and evaluation period. The implementation modified existing data models, added four new ones, and represented changing tax rules with lookup data containing valid_from and valid_to dates, including thresholds, date ranges, inclusions, exclusions, and inclusive or exclusive comparisons. Teams prototyped models in BigQuery SQL, implemented them in PySpark and Spark jobs, and combined merchant monthly taxable-sales aggregates with the rule model to evaluate economic nexus; the architecture moved source data through GCS and a warehouse into a final model. Each run publishes merchant-keyed insights to Google Cloud Bigtable, where the core application retrieves them for the frontend, while warehouse data supports analytics and other processes.
Supabase Clippy: ChatGPT for Supabase Docs
Supabase introduced “Supabase Clippy,” a ChatGPT-like interface trained on its documentation, as a hidden MVP in the Supabase docs. It responds to questions by retrieving relevant documentation and streaming GPT-3's answer in realtime. The implementation parses docs into sections, creates OpenAI embeddings, stores them in Postgres with pgvector, and injects matching documents as context for generation. The project was motivated by the growth of Supabase's product suite and documentation, plus the need to support developers unfamiliar with Postgres; the current version uses only Guides and is presented as a one-week MVP intended to gather feedback. Planned directions include adding reference docs and community discussions, caching repeated queries, thumbs-up relevance feedback, and interactive guidance.
Storing OpenAI embeddings in Postgres with pgvector
Post describes how embeddings represent semantic relatedness as vectors and how OpenAI’s text-embedding-ada-002 model produces 1536-dimensional embeddings for text. It addresses storing and searching large or dynamically changing collections that may not fit comfortably in memory, using PostgreSQL and the pgvector extension instead of a small CSV-and-NumPy workflow. The implementation enables the vector extension, creates a documents table with a vector(1536) column, and defines match_documents() with cosine distance, thresholds, counts, and ordering. It also shows JavaScript code for generating embeddings, a search flow that retrieves matching documents for prompt context, and token limiting before a completion request, while noting that documents can be chunked. For larger tables, the post recommends an ivfflat index with vector_cosine_ops and gives 4 * sqrt(table_rows) as a starting point for lists.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.