Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,535 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Automatically Rotating GitHub Tokens (So You Don’t Have To)
Shopify needed automated GitHub workflows that could clone from private repositories, update organization-level projects, and trigger downstream workflows but could not use GITHUB_TOKEN. Personal access tokens supplied those capabilities, yet introduced developer churn, user-attributed actions, and no organization-enforced rotation or expiry. The proposed system uses GitHub Actions and a GitHub App to generate repository secrets that rotate automatically while preserving PAT-like references in existing workflows. Its serverless design delegates scheduling, managed infrastructure, approvals, secret management, auditing, and break-glass operation to GitHub, while a custom TypeScript action encrypts and writes generated secrets. Prototyping exposed that parallel one-second jobs could accrue multiple billable minutes, prompting sequential execution in a consolidated workflow; the stated goal is improved resilience to developer churn and reduced leaked-secret blast radius.
Custom Domain Names
Supabase is releasing Custom Domains and Vanity Subdomains to give projects a more branded user-facing address. Custom Domains let a project use its own domain, such as api.example.com, instead of an assigned supabase.co hostname; they are generally available as a paid add-on through the Supabase dashboard. Vanity Subdomains let paid-plan projects choose a relevant subdomain on Supabase's domain, such as my-example-brand.supabase.co, at no additional cost, and the feature is launching in closed beta with setup instructions in the documentation. The release currently supports only one custom domain or vanity subdomain per project, while Edge Functions must still be accessed through their existing endpoint because they do not support either feature yet.
Launch Week 6: Wrap Up
Supabase’s Launch Week Day 6 wrap-up consolidates the products and platform changes announced during the event. The roundup announces redesigned Next.js documentation, Storage v2 features for image resizing, webhooks, and a Smart CDN, plus multi-factor authentication usable in Row Level Security policies. It also introduces Wrappers, an Alpha framework for connecting Postgres to external systems, Vault for encrypted secrets, and pg_graphql 1.0 with general availability. Additional releases include dashboard-enabled point-in-time recovery for Pro Projects, custom domains, pg_crdt, PostgreSQL 15 for new projects, a PostgREST 11 pre-release, and the PGroonga multilingual full-text search extension. Community partnerships with Flutterflow and OneSignal provide Flutter app development and push-notification integrations.
What's new in Postgres 15?
PostgreSQL 15, released as a stable version in October 2022, adds security, performance, SQL, replication, logging, compression, and string-processing changes. CREATE permission is revoked from all users except the database owner, views can use caller permissions with security_invoker=on, and sorting is reported up to four times faster for in-memory and on-disk workloads; window functions also improve. The release introduces MERGE, supports two-phase commits and selective data in logical replication, allows conflicting transactions to be skipped or subscriptions disabled, and enables parallel workers for SELECT DISTINCT. It also adds JSON logging, Zstandard and ZL4 compression for WAL and pg_basebackup, and POSIX regexp functions such as regexp_substr() and regexp_count(). The post recommends upgrading for bug fixes and stability, while noting that migrated databases may need manual permission changes and upgrades should be checked for rare manual actions.
pg_graphql v1.0
pg_graphql 1.0 marks the general availability of a PostgreSQL extension that lets databases be queried with GraphQL and formalizes a SemVer-based guarantee against backward-incompatible changes. It was designed for Supabase Free Plan servers with 1 GB of memory, where alternatives such as Hasura and Graphile consumed more memory than the platform could sacrifice. The implementation began with a C parser and SQL transpiler exposed through PostgREST RPC, then moved from SQL to Rust after nested SQL functions, copied state, and JSONB AST processing slowed schema introspection; v0.5.0 completed the rewrite. On Free Plan hardware, version 1.0 adds less than 300 microseconds per request, reaches about 1,060 SQL queries per second per connection, and the full Auth and HTTP stack handles about 645 requests per second, while larger instances exceed 10,000 requests per second.
Point in Time Recovery is now available for Pro projects
Point-in-Time Recovery (PITR) is now available to eligible Pro Plan projects, expanding access beyond Enterprise customers and adding a Dashboard UI for enabling it. Projects created from August 2022 onward can enable PITR except in Northeast Asia / Seoul, and they must use the Small compute add-on; older projects are not yet generally eligible. The system combines daily physical database snapshots with archived Write Ahead Log (WAL) records, using the open-source WAL-G tool; snapshots are uploaded daily and WAL files are archived as transactions occur. WAL files are backed up at two-minute intervals, potentially faster when a file threshold is reached, reducing the recovery point from the up-to-24-hour exposure of daily backups. PITR backups retain up to seven days by default, with self-serve expansion to 28 days, and restoration can target a specified point in time.
PostgREST 11 pre-release
PostgREST 11 is still in pre-release, with the latest features and fixes available through the Supabase CLI rather than Supabase Cloud. The update improves querying related data with spread operators that flatten one-to-one and many-to-one relationships into the response, avoiding an extra view or function for that shape. Related-table ordering is also supported, such as ordering films by technical_specs.duration, with the same relationship limitation. Anti-joins and inner-join behavior can be expressed by filtering related resources with is.null or not.is.null; the latter can also be combined with an or filter and is more flexible than the existing !inner modifier. The release can be tried locally with supabase start, and the post asks users to report bugs, suggestions, and ideas.
Supabase Vault is now in Beta
Supabase Vault is a Postgres extension and accompanying dashboard UI for storing encrypted secrets and encrypting data in database tables, released progressively across the platform and later made available on all Supabase projects. It stores secrets and encryption keys using authenticated encryption on disk, exposes decrypted secrets through a Postgres view, and lets applications use them from SQL in functions, triggers, and webhooks. Its foundation is pgsodium Transparent Column Encryption (TCE), which can encrypt text or bytea values, authenticate associated row columns, and create row-by-row decryption views for encrypted columns. Encryption keys are kept outside the database while only key IDs are stored with encrypted data; Supabase manages root keys in internal backend systems and offers an alternate dashboard endpoint for external decryption, while the post also outlines possible future end-to-end, group, and public-key encryption features.
Supabase Wrappers, a Postgres FDW framework written in Rust
Supabase Wrappers is an Alpha framework for building PostgreSQL Foreign Data Wrappers that connect databases with external systems, including third-party APIs, through SQL. It initially supports Firebase and Stripe, while wrappers for ClickHouse, BigQuery, and Airtable are under development. The framework extends pgx, a Rust Postgres extension framework, with async support for RESTful API-based wrappers and limited push-down support currently provided as a proof of concept. The post presents Wrappers as a basis for QETL, allowing teams to query, extract, transform, and load remote data through Postgres, either on demand or into local tables, while joining it with operational data. It is released outside official platform support, with potential breaking changes.
Launching Redis by Upstash
Fly.io and Upstash launched a managed Redis service positioned close to Fly.io applications, with global replicas, a free tier, and private encrypted IPv6 connectivity. The integration uses Flycast, Machines API, WireGuard networking, and PROXY protocol headers: flyctl provisions a Flycast IP, routes it through global proxies to the nearest healthy Redis instance, and lets multitenant Redis identify the customer database without exposing its password. Applications can add replicas and write forwarding through flyctl while retaining a single Redis URI; writes go to the primary and reads and writes use geographically nearest replicas. The post illustrates edge HTML caching for a Rails recipe app, reporting low-millisecond global time to first byte for cached GET requests. It also describes read-your-own-write handling, including a 500ms replication wait and fallback reads from the source of truth.
Multi-factor Authentication via Row Level Security Enforcement
Supabase announces the release of multi-factor authentication (MFA) for everyone and describes dashboard Single Sign-On (SSO) as preparation for a future SAML release; SSO is available on Enterprise plans by contacting the company. The implementation starts with Time-Based One-Time Passwords (TOTP), where an authenticator generates a short-lived code from a shared secret and the current time, exchanged through a QR code or entered manually. Enrollment uses /enroll, /challenge, and /verify, while subsequent sign-ins require a challenge and verification before a JWT is issued with Authenticator Assurance Level (AAL) and Authenticator Method Reference (AMR) claims. Those claims support full or selective enforcement through restrictive Postgres Row Level Security policies, including checks that require aal2 or require it only for users with verified MFA factors; planned additions include WebAuthn, FIDO2 devices, and email notifications.
Shopify Embraces Rust for Systems Programming
Shopify has adopted Rust as its official systems programming language while retaining Ruby for server-side business logic, and has joined the Rust Foundation. The change addresses growing systems-programming needs including high-performance network servers, Ruby native extensions, and WebAssembly, where Shopify seeks speed, productivity, safety, and community-driven open source. Shopify cites Rust’s predictable native-code performance, fine-grained memory control, compile-time safety, support for concurrency, and interoperability with C and Ruby through tools and crates such as bindgen, rb-sys, and magnus. The company says Rust performed strongly in its safety evaluation and that its initial projects exposed more errors at compile time, while acknowledging that performance still depends on design and measurement and that internal education, tooling, and community participation remain unfinished work.
Supabase Storage v2: Image resizing and Smart CDN
Supabase Storage introduces image resizing, webhooks, and a Smart CDN, positioning them as components of an on-demand image delivery system. Resizing uses imgproxy and supports width, height, and contain, cover, or fill modes through JavaScript clients or direct Storage API requests; images are not enlarged beyond their original dimensions, and aspect ratio is preserved by default. The Smart CDN stores resized results using cache keys based on an Etag, tenant ID, bucket, and image name, while webhook-driven metadata updates propagate invalidations globally within about 60 seconds. Image Resizing is beta for Pro and Enterprise customers, with 100 transformed images per project at no additional cost and $5 per additional 1,000; self-hosting is free.
3 (More) Tips for Optimizing Apache Flink Applications
Shopify presents three additional practices for optimizing large stateful Apache Flink applications, focusing on parallelism, sink capacity, and combining heterogeneous sources. It recommends starting with execution-environment parallelism, then matching task-manager capacity to the highest parallelism value; for example, parallelism 100 requires 25 task managers with four slots each. Sink bottlenecks can propagate backpressure upstream, so batching writes and correcting data skew—including with key bucketing when a low-cardinality or uneven key is unavoidable—can improve distribution, though bucketed results must later be combined. For archived Kafka topics, HybridSource exposes cloud-storage history and the live Kafka topic as one ordered logical DataStream, automatically switching after the archive is exhausted and using object-storage partitions to accelerate historical backfills.
New Supabase Docs, built with Next.js
Supabase released a new version of its documentation, rebuilding the site with Next.js after outgrowing Docusaurus. The earlier setup became harder to maintain as client-library versioning expanded, contributions grew more difficult, and the docs remained separate from Supabase’s Next.js and Turborepo monorepo. To reduce migration risk, the team first created a one-week Next.js MVP that copied Docusaurus, then spent a month developing the new release with two full-time developers. It introduces a new design, a single sidebar for broader discoverability, and reference pages that group all functions in a library while improving TypeDoc-based generation. Future work described includes unified authentication, project-specific documentation, runnable Postgres WASM examples, interactive tutorials, improved Postgres full-text search, and balancing sparse and overly verbose content.
pg_crdt - an experimental CRDT extension for Postgres
Supabase announces pg_crdt, an experimental Postgres extension that adds CRDT support as a data type for collaborative and offline-oriented applications. It frames CRDTs as structures whose updates can be merged in any order to reach the same final state, avoiding the last-write-wins behavior of ordinary arrays. A content column can use crdt.ydoc, backed by Yjs, while the extension also supports Automerge through Rust libraries wrapped with pgx. The post contrasts storing CRDTs directly in Postgres with making Realtime an authoritative server, favoring the database-centered approach for simpler architecture and broader tooling, while noting that the decision is unsettled and pg_crdt is not released on the Supabase platform. Known limitations include full-document WAL broadcasts, increased WAL and dead tuples for frequently updated documents, and serialization overhead for large CRDT values.
Launch Week 6 Hackathon
Supabase announces its official Launch Week 6 Hackathon, which starts at 8:00am Pacific Time on Friday 9th December 2022 and ends at 00:01am PT on Monday 19th December. The event gives individuals or teams of up to five people ten days to create a new Open Source project using Supabase. Entries may be projects, apps, tools, or libraries in any language or framework, but all design elements and code must be created during the event and each team may submit once. Five categories offer winner and runner-up prizes, including $1500 in GitHub sponsorships paid as three monthly $500 sponsorships for the best overall project, plus Supabase keyboards for its team members. Supabase judges the main categories, while Deno judges Best Edge Functions Project; judging considers creativity, smooth functionality, visual appeal, technical impressiveness, and use of Supabase features.
Who We Hire at Supabase
Supabase describes the hiring traits and operating model it believes support its growth, arguing that people matter more than methodology. It looks for extreme asynchronous effectiveness, egoless execution, continuous incremental improvement, a bias toward immediate action, ambitious problem-solving, and a founder mindset; seniority does not exempt anyone from independent-contributor work, self-management, or maintenance. The 60-person team spans 25 countries, uses meetings as a last resort, and gives autonomous product teams access to shared metrics, user feedback, and equity through an ESOP. Teams are organised around products such as Postgres, Auth, Storage, Realtime, and Functions, while cross-cutting groups consult rather than merely receive delegated tasks. Three-month product cycles set ambitious goals, and Kaizen Weeks focused on QA, documentation, and performance help address technical debt and incorporate improvements into ongoing workflows.
Three Essential Remote Work Practices for Engineering Teams
After Shopify became a fully remote-first company, its engineering organization refined how teams communicate, collaborate, and work across time zones. The recommended team structure concentrates engineers in related time-zone regions to maximize synchronous availability for discussion, pair programming, mentorship, and questions, while relying on asynchronous writing and documentation between teams. Teams are also asked to define communication norms that specify which medium to use, expected response times, and whether contact is appropriate after hours; documenting these rules reduces ambiguity and protects focus. Individuals should intentionally shape routines and work-life boundaries around their own energy, productivity, health, and responsibilities, then integrate those preferences with team norms. Together, these practices support flexible remote work by combining synchronous collaboration within teams, asynchronous coordination across teams, and greater control over the working day.
Supabase Beta November 2022
Supabase’s November 2022 update previews Launch Week 6 while reporting a smaller set of product releases and community activity. Remix Auth Helpers are now available for server-side authentication, updated for supabase-js V2 and usable with TypeScript; the post also introduces the Edgy Edge Functions YouTube series and three examples covering OG image generation, Oak API servers, and Puppeteer screenshots. Product fixes address auth.signOut cookies and session removal, magic links can be customized with {{ .TokenHash }}, functions support redirects, and Supabase can be installed through Conda. The update also announces a Fireship course building a Reddit-inspired app with Supabase, PostgreSQL, and React, alongside a broad list of community resources and projects.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.