Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,535 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Supabase Series B
Supabase announced an $80 million Series B in May, bringing total funding to $116 million, and reported growth to more than 150,000 hosted PostgreSQL databases and 110,000 developers. Community metrics also increased, with Discord membership rising from 4,000 to 8,000, Twitter followers from 18,000 to 37,000, and GitHub stars from 19,000 to 36,000. The round is led by Felicis, with Coatue, Lightspeed, and Square Peg Capital joining, while Supabase describes its open-source support through contributor payments, maintainer hiring, sponsorship, and community collaboration. The company is moving into its third phase, focused on cloud-native PostgreSQL capabilities including branching, scalable storage, distribution, ephemeral compute, snapshots, and time travel. It invested $500,000 in OrioleDB and hired a developer to work on it, while stating that OrioleDB is not currently used on the Supabase platform and that the company will not run a PostgreSQL fork.
Rate limiting with Redis
Rate limiting at Ramp must handle third-party API quotas, globally paced Celery tasks, and inbound API traffic limited by authenticated application identities rather than only mutable IP addresses. Celery’s per-worker controls were insufficient for global limits, while queue prefetching could let rate-limited tasks block workers and stall queues. The team created a general framework for comparing algorithms and evaluated fixed-window, sliding-window, leaky/token-bucket, and Generic Cell Rate Algorithm (GCRA) approaches. They selected GCRA for implementation simplicity and compute and memory efficiency, storing a theoretical arrival time in Redis and using a Redis lock to prevent races. The implementation also uses floating-point timestamps, Redis server time, key expiration, unit tests, and manual testing; it has been introduced gradually across the three use cases, with monitoring, while additional algorithms remain planned.
When is JIT Faster Than A Compiler?
JIT compilation can outperform ahead-of-time compilation or interpretation for highly dynamic languages such as Ruby by optimizing runtime behavior under assumptions that static approaches cannot safely make. Ruby permits redefining operators, equality, iteration, hashing, and other behavior at any time, even from a background thread, forcing an interpreter to check whether definitions changed and limiting AOT code to conservative behavior. YJIT addresses this by compiling frequently used methods with assumptions about current definitions, then invalidating and discarding compiled versions when those assumptions become false; later calls can run interpreted code until a new version is compiled. The post presents this speculative optimization and de-optimization model as the reason a JIT can be faster, while noting that YJIT had known bugs in Ruby 3.1 and was expected to be closer to production-ready for some uses in Ruby 3.2.
Launch Week 5 Hackathon
Supabase’s Launch Week 5 Hackathon invites individuals and teams of up to five to build a new open-source project using Supabase during a 10-day event. It starts at 8:00am PT on Friday 12 August 2022, with submissions due by 11:59pm PT on Sunday 21 August; projects may use any language or framework and must be created during the event. The best overall project receives $1,500 in GitHub sponsorship paid as $500 per month for three months, while the listed category prizes include Gold Supabase Caps, with winner and runner-up prizes and caps for every winning team member. Supabase-js v2.0 and Flutter SDK v1.0 are announced during Launch Week, and using them earns extra credit; the supabase-js release candidate is installable with npm install @supabase/supabase-js@rc.
How SQLite helps you do ACID
SQLite’s rollback journal provides atomicity, isolation, and durability by preserving database state before a transaction changes the main file. Without journaling, an interrupted 4KB page write could corrupt the database; the journal instead records original pages, then permits recovery if commit steps fail. A journal header stores a nonce, the database’s pre-transaction page count, sector size, and page size, while each page record includes its page number, a copy of prior contents, and a 32-bit checksum. The mode allows concurrent readers but makes writers wait for readers and blocks new readers, using SHARED, RESERVED, and PENDING filesystem locks. Its simplicity brings a concurrency bottleneck; WAL and wal2 provide better support for concurrent readers, which the post says will be covered next.
Slack Consolidate: a slackbot built with Python and Supabase
Slack Consolidate is a Python Slackbot that watches configured Slack channels and reposts messages to destination channels according to routing rules stored in Supabase. The tutorial sets up Slack app permissions from a manifest, installs the Python SDKs, configures environment variables for Supabase and Slack tokens, and creates slack_channels and slack_watcher PostgreSQL tables. Its polling loop loads channel mappings, fetches up to 20 recent messages, converts Slack timestamps to ISO-compatible values, inserts messages into the watcher table, builds message links, and posts through a bot client; it waits 1.4 seconds between channel processing and rescans channel configuration hourly. Public-channel posts contain a link, while private-channel posts include the message text as well. The project also provides a Google Colab notebook for trying the example.
Six Tips for Staying Technical as a CTO
The article examines how a CTO can remain technically credible after moving from the individual contributor track into leadership. It argues that leadership inevitably reduces technical depth because performance reviews, one-on-ones, organizational communication, and other people responsibilities limit time in code, making the transition an intentional choice for those who value managing people. Its recommended practices include voracious reading, coding as a hobby, participating in hackathons, learning directly from more technical colleagues, maintaining a private network of technical advisors, and aggressively clearing bugs, incident-remediation work, and backlog. The author illustrates hands-on learning with Shopify’s Spin environment, which provisions a cloud development instance and opens a repository in about 30 seconds, and concludes that controlling operational issues preserves time for innovation and technical leadership.
Volume Expansion and Snapshot Restores
Fly.io introduces two improvements to its persistent storage volumes: extending an existing volume and restoring a regular volume from a snapshot. Previously, increasing capacity required provisioning an empty volume and copying data, whereas the new fly volume extend command resizes it after the attached app restarts; regular apps restart automatically, while Machines VMs require manual restarts. Individual restores are available from flyctl v0.0.363 by supplying --snapshot-id when creating a new volume, after listing snapshots and selecting the desired ID. Snapshots are taken daily, stored in object storage for five days, and can restore data only to a volume at least as large as the source.
Spin Cycle: Shopify’s SFN Team Overcomes a Cloud-Development Spiral
Shopify’s Fulfillment Network (SFN) worked to run its monolithic Rails application in Spin, a cloud-based development tool that provides on-demand Kubernetes containers instead of local service environments. Because SFN glues together Shopify and many external services, the team had to decide whether to run, mock, or share each dependency, while frequent changes to SFN, Spin, and those dependencies created regressions. The three-engineer team identified a human feedback dynamic: painful Spin experiences discouraged adoption, which allowed more Spin-specific breakage, while regular use could expose regressions before merge. To break that spiral, the team prioritized basic functionality for common validation flows, fixed disruptions there, and asked developers to use Spin daily and add missing capabilities; the intended outcome was a self-reinforcing “Loop of Happy Equilibrium,” though the transition is described as a hypothesis.
Supabase Beta July 2022
Supabase published its July 2022 beta update alongside the announcement of Launch Week 5 scheduled for August 15-19. Key feature releases include a developer preview of Flutter SDK 1.0 with an emphasis on developer experience and Auth Helpers supporting server-side rendering environments like SvelteKit. The team introduced pg_jsonschema, an experimental 10-line Postgres extension created with pgx to enforce structure on json and jsonb columns. Security and identity features were expanded with an hCaptcha integration in authentication settings to block bot attacks. Additionally, email one-time passwords now accommodate lengths between 6 and 10 digits, while the GenerateLink method returns explicit verification metadata.
Supabase Flutter SDK 1.0 Developer Preview
Supabase released the Developer Preview of version 1.0 for its Flutter SDK, prioritizing improvements to developer experience across Dart applications. The new version eliminates the need for boilerplate state classes, automating authentication state persistence and deep link processing directly after initialization. To better align with Dart conventions, the SDK now throws exceptions across auth, storage, and Postgrest instead of returning error objects, while also deprecating the .execute() method for database queries. Switching to the app_links library expands deep linking support to macOS and Windows alongside existing mobile and web platforms. Supabase also introduced a companion Auth UI library for prebuilt authentication interfaces and announced plans to integrate the Multiplayer Realtime engine in the stable release.
Mastering React’s Stable Values
React stable values are values, often returned by hooks, that retain the same value across renders, and the post explains why their identity matters in effects, callbacks, memoization, and component rendering. useState and useReducer update functions and useRef objects are stable, while array literals, object literals, and anonymous functions are recreated on each render; useCallback and useMemo can produce stable values under appropriate dependencies. Unstable dependencies prevent useEffect behavior from remaining limited to intended changes and stop useCallback and useMemo cached results from being reused, while an unstable callback passed to useFocusEffect causes repeated unsubscribe and resubscribe cycles. The post also contrasts invoking an inline helper with rendering it as a component, explains why the latter can rebuild a React Native subtree on every render, and recommends defining components outside other components and using React.memo or callbacks judiciously.
10 Tips for Building Resilient Payment Systems
The post distills lessons from five years working on Shopify’s payment infrastructure into guidance for building systems that tolerate failure and large-scale traffic. It frames the advice partly around onboarding developers who were new to payments or Shopify’s scale, and notes that load testing for BFCM 2021 began in July with globally distributed load generation. It recommends low timeouts across HTTP clients and data stores, circuit breakers such as Semian, and capacity reasoning with Little’s Law, rate limiting, and load shedding. Monitoring should cover latency, traffic, errors, and saturation, while structured, machine-readable logs make centralized records searchable; load tests use benchmark gateways that mimic production partner latency and capacity. For incidents, Shopify uses defined roles, Slack-based coordination, and retrospectives that turn incorrect assumptions into safeguards and training material.
SQLite Internals: Pages & B-trees
SQLite stores table rows as encoded byte arrays, and understanding that representation explains how a simple database preserves compact records and supports efficient growth. Using a sandwich table as an example, the post decodes a row’s varints, type values, UTF-8 text, IEEE-754 floating-point value, and integer count, then examines 4KB pages, cell-pointer indexes, and page headers. SQLite’s b+tree places records in leaf pages and key-range indexes in interior pages; page splits add levels as the database grows, while searches traverse only one page per tree level. The examples estimate that a depth-four tree can hold about 500³ leaf pages, or roughly 476GB, while requiring four page reads for a lookup. The discussion concludes that integer storage is more space-efficient than floating-point storage and that internals such as record encoding and PRAGMAs become less opaque when understood.
Launching Laravel Bytes
Fly is launching Laravel Bytes, a new publication focused on Laravel and on deploying Laravel applications across the globe. The announcement frames the site as a place for future discussions of global-user-base considerations, along with Laravel tips, strategies, how-tos, and coverage of Livewire and real-time apps. Laravel is supported by the fly launch command, and the documented quick start is to enter an existing Laravel app directory and run fly launch. A more comprehensive guide covers running Laravel on Fly with Redis, MySQL, queue workers, and cron tasks, while the post directs readers to deployment documentation and says a Laravel app can be launched in a few minutes.
Data-Centric Machine Learning: Building Shopify Inbox’s Message Classification Model
Shopify Inbox needed to classify millions of monthly customer messages so merchants could prioritize responses and guide product decisions. The team adopted a data-centric approach, improving the dataset rather than primarily changing model architectures. An initial prototype used Universal Sentence Encoder embeddings, UMAP and HDBScan clustering, semi-supervised labeling, and a neural classifier, but reached about 70% accuracy with 35% high-confidence coverage. For version 2.0, the team created a more explicit taxonomy, used human-labeled data and fine-tuned DistilBERT, while addressing privacy and class imbalance through data handling and targeted annotation. In production, version 2.0 achieved about 90% accuracy and 80% high-confidence coverage, although the taxonomy and training data require ongoing updates as conversation trends change.
5 Steps to Bounce Back from a Negative Performance Review
An unfavorable performance review can feel like failing an exam, but it can also identify a clearer path to success when treated as feedback about role expectations rather than a judgment of personal worth. The five-step process is to avoid taking the review personally, adopt a growth mindset, clarify expectations with a lead, define measurable success, and establish a feedback loop. The author illustrates the approach with feedback about taking too long to ship code, which led to using smaller incremental improvements, and recommends tying goals to KPIs or other tangible metrics, supported by a lead or mentor. Regular check-ins, an up-to-date brag doc, and documented accomplishments can reduce surprises at the next review and help connect feedback to quarterly goals and observable progress.
How LiveView Took on a Life of Its Own
Phoenix LiveView began as an attempt to build dynamic server-rendered applications without writing JavaScript, using the server for interactions such as realtime reservation updates. Its initial design used stateful Phoenix Channels, a WebSocket connection, client event bindings, and full template re-renders sent back as HTML, a model that worked but produced redundant server work and large payloads. Adding Phoenix PubSub let LiveViews subscribe to reservation events and update all connected browsers through ordinary Elixir callbacks, without client changes. The implementation then evolved around diffing dynamic values, shared static template references, and cached live component templates, reducing transferred data and avoiding growing client bundles. The post concludes that this programming model sidesteps HTTP for dynamic features, with compact payloads and less client-side code.
Implementing "seen by" functionality with Postgres
Tracking unique post views within Postgres presents trade-offs between counter accuracy, row bloat, and concurrent write performance. To evaluate options under an architecture constraint preventing external dependencies, a benchmark suite generated synthetic users, skewed post distributions, and replayed view actions. The benchmark tested four implementations: a naive counter column, an hstore key-value approach, an association table, and HyperLogLog (HLL). Benchmark results demonstrated that simple-hstore achieved the lowest average latency among deduplicating approaches at 2.15 milliseconds, closely followed by HLL at 2.16 milliseconds. Despite hstore's raw performance in prototypes, HLL is recommended for production because it circumvents row bloat as view counts grow while avoiding expensive row counts.
Logbook - 2022-07-18
This Logbook entry records Fly.io product, platform, documentation, and interface changes on 2022-07-18. Highlights include $99 in usage credits for the Pro Plan, SOC2 compliance, persistent-volume expansion through fly volumes extend, SFTP access to apps over 6PN and WireGuard, and routing changes intended to improve latency. flyctl gained debug JSON formatting, expanded machine-app deployment and status commands, preserved mounted volumes during deployments, and added rolling deploys that wait for each VM to start. The entry also describes fixes for autoscaling, the Fly Redis image, GraphQL validation, Phoenix launches, and dashboard presentation, alongside updated Laravel and persistent-volume documentation and improved Heroku token-failure emails.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.