---
title: "Postgres"
description: "308 posts about Postgres, summarised, each linking to the original."
---

# Postgres
> 308 posts about Postgres, summarised, each linking to the original.

## Articles

### [Supabase Storage: major performance, security, and reliability updates](https://yomu.fyi/post/supabase-storage-major-performance-security-and-reliability-updates.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: fabrizio
- Published: Mar 5, 2026
- Type: Announcement

Supabase Storage has shipped updates live in all regions for projects using Storage, targeting performance at scale, security vulnerabilities, and reliability issues. As projects reached tens of millions of objects, concurrent uploads exposed write bottlenecks, certain SQL deletes could orphan files, and a crafted path could escape the file backend’s storage root. The object-listing rewrite removes the prefixes table, six triggers, and twelve helper functions, derives folder structure from the objects table with a hybrid skip-scan algorithm, and uses cursor-based pagination; deep pagination on 60 million rows is up to 14.8 times faster without a write penalty. Direct SQL deletes are blocked by default through a statement-level trigger, while a configurable 30-second timeout supports cancellation of disconnected clients via native Postgres protocol. Idempotent migrations, TUS lock-race prevention, orphan-scanner fixes, OpenTelemetry metrics, and adapter and API bug fixes are included, with no code changes required.


### [Innovating with MongoDB | Customer Successes, February 2026](https://yomu.fyi/post/innovating-with-mongodb-customer-successes-february-2026.md)
- Company: [MongoDB](https://yomu.fyi/company/mongodb.md)
- Author: Katie Palmer
- Published: Feb 25, 2026
- Type: Problem & solution

Modern artificial intelligence startups frequently face operational bottlenecks when deploying autonomous agents and dynamic interfaces on rigid relational schemas or restricted database engines. To address these limitations, platforms including Modelence, Thesys, Emergent Labs, and Heidi integrated MongoDB Atlas as their operational data foundation. Emergent Labs eliminated recursive schema migration failures previously encountered with PostgreSQL by adopting document structures that match native JSON generation, supporting millions of deployed applications. Meanwhile, Heidi migrated from Amazon DocumentDB to utilize MongoDB Vector Search for retrieval-augmented generation pipelines under a single unified API without requiring external search databases. These architectural transitions streamlined continuous agent iterations, eliminated operational scaling downtime, and accelerated deployment cycles across diverse production environments.


### [Welcome to Notion, Sequin team!](https://yomu.fyi/post/welcome-to-notion-sequin-team.md)
- Company: [Notion](https://yomu.fyi/company/notion.md)
- Author: Fuzzy Khosrowshahi
- Published: Jan 9, 2026
- Type: Announcement

Notion welcomes the Sequin team—Eric Goldman, Anthony Accomazzo, and Carter Pedersen—following discussions about the companies’ shared view of fragmented knowledge work. Sequin built a data-syncing product that brought information from systems including Salesforce, ServiceNow, and Stripe into Postgres, giving developers flexible building blocks instead of requiring them to recreate integrations. The founders say they were drawn to Notion’s enterprise search, work on AI and agents, ambitious direction, and strong user community, while Notion describes Sequin’s experience as aligned with its plans. The announcement frames the broader problem as information scattered across documents, conversations, and tools, and points to AI and agents as a way to help teams find information, automate repetitive work, and prepare for AI-native work.


### [The evolution of Grab's machine learning feature store](https://yomu.fyi/post/the-evolution-of-grab-s-machine-learning-feature-store.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Daniel Tai
- Published: Jul 24, 2025
- Type: Explainer

Grab describes how it evolved its machine learning feature store as increasingly complex models began requiring high-cardinality keys and high-dimensional data. It replaced its initial Amphawa approach with feature tables: data-lake tables whose contents are atomically ingested into serving storage, with emphasis on consistency and reproducibility. The workflow stores feature tables and version-control metadata as Parquet objects in Amazon S3, validates schemas, loads data through reverse ETL into Aurora Postgres, and warms new tables through shadow reads before switching. The redesign supports contextual retrieval, atomic collection-wide updates, and isolation between reading and writing capacity; its SDK remains a thin database-driver wrapper that translates queries to SQL. Grab also chose decentralised deployments for version flexibility, rollout strategies, noisy-neighbor mitigation, and improved caching, concluding that Amazon Aurora for Postgres provides a robust, scalable, cost-effective foundation.


### [Heroku Postgres Upgrade Guide: Simplify Your Move to Version 17](https://yomu.fyi/post/heroku-postgres-upgrade-guide-simplify-your-move-to-version-17.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Kavya Premkumar
- Published: May 29, 2025
- Type: Tutorial

Heroku Postgres 17 is now available, and the platform introduces a simplified, self-service path for upgrading managed databases. The new heroku pg:upgrade:\* commands cover preparation, manual execution, cancellation, dry runs, and progress tracking, while email notifications report major stages and detected issues. Essential-tier databases can be upgraded directly with pg:upgrade:run; Standard-tier and higher databases can use a dry run, schedule an upgrade for the next maintenance window, or run it manually, with followers upgraded automatically. The in-place workflow uses an internal replica, leaves DATABASE\_URL and other config vars unchanged, and typically reduces downtime to about five to ten minutes. The legacy heroku pg:upgrade command will be deprecated soon, although the traditional follower approach remains available.


### [Scaling Nextdoor’s Datastores: Part 5](https://yomu.fyi/post/scaling-nextdoor-s-datastores-part-5.md)
- Company: [Nextdoor](https://yomu.fyi/company/nextdoor.md)
- Author: Slava Markeyev
- Published: Mar 19, 2025
- Type: Problem & solution

Nextdoor addressed database and cache consistency issues caused by missed cache writes and concurrent read-fill operations in their look-aside architecture. While forward row versioning prevents out-of-order write inconsistencies, writer failures and race conditions during cache misses can leave stale data persisted in Redis. To resolve this, Nextdoor built a reconciliation pipeline that consumes PostgreSQL WAL replication logs with pg-bifrost, streams changes through Apache Kafka, and executes conditional deletions in Redis. The Go-based reconciler operates in two passes using a time wheel, running one pass in near real time and a second pass after a delay exceeding web request timeouts. Because conditional deletion evaluates row versions directly in the cache, the system processes change streams out of order and scales horizontally.


### [Scaling Nextdoor’s Datastores: Part 4](https://yomu.fyi/post/scaling-nextdoor-s-datastores-part-4.md)
- Company: [Nextdoor](https://yomu.fyi/company/nextdoor.md)
- Author: Ronak Shah
- Published: Mar 19, 2025
- Type: Problem & solution

Look-aside caching systems can become inconsistent when concurrent database updates execute cache writes out of order, allowing stale data to overwrite newer modifications. To prevent these racing writes, Nextdoor introduced a unique, monotonic db\_version column to Postgres tables using database triggers that initialize version numbers on insert and increment them on update. Application updates retrieve this new version inside a transaction block and attach it as a metadata header to serialized cache values. Redis then executes custom Lua scripts, specifically set\_if\_version and del\_if\_version, to perform atomic conditional updates that reject any incoming payload with a version lower than or equal to the stored version. This serializable check ensures that out-of-order writes are dropped and the cache remains strictly aligned with the latest database state.


### [Scaling Nextdoor’s Datastores: Part 1](https://yomu.fyi/post/scaling-nextdoor-s-datastores-part-1.md)
- Company: [Nextdoor](https://yomu.fyi/company/nextdoor.md)
- Author: Slava Markeyev
- Published: Mar 19, 2025
- Type: Explainer

Nextdoor relies on a Django backend connected to monolithic PostgreSQL databases and Redis look-aside caches. Migrating to distributed SQL datastores proved impractical because legacy business logic depends heavily on multi-table joins that could not be rewritten. Intermediate mitigations, including read replicas and data partitioning by severing foreign keys, extended infrastructure runways but left primary databases vulnerable to load bottlenecks. Furthermore, look-aside caching and replica usage introduced data staleness risks, atomicity losses across partitioned databases, and inconsistent cache-population behaviors. To address these limitations, Nextdoor initiated an architecture redesign focused on dynamic query routing to replicas, replica-driven cache hydration, time-bounded eventual consistency, and schema-resilient cache serialization.


### [Optimizing Data Reliability: Heroku Connect & Drift Detection](https://yomu.fyi/post/optimizing-data-reliability-heroku-connect-drift-detection.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Siraj Ghaffar
- Published: Jun 26, 2024
- Type: Explainer

Heroku Connect syncs Salesforce data with Heroku Postgres, but primary polling can miss changes when long-running Salesforce automations commit after the polling window advances, creating drift. The service uses secondary polls over the same bounds as primary polls to find and repair missed changes, with Bulk API or SOAP API selected according to dataset size and retries for failed polls. Recent improvements change Bulk API drift handling by creating one secondary bulk job for each primary bulk job, then adding corresponding batches instead of creating many jobs. Secondary tasks are now scheduled and executed with limited concurrency, reducing simultaneous database connections consumed while Postgres is updated and preserving capacity for applications. The post says these changes reduce load on Salesforce and improve operational reliability during heavy synchronization, while noting that Heroku Connect remains asynchronous rather than truly real-time.


### [Building a GPT Backed by a Heroku-Deployed API](https://yomu.fyi/post/building-a-gpt-backed-by-a-heroku-deployed-api.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Julián Duque
- Published: Mar 28, 2024
- Type: Tutorial

The walkthrough shows how to connect an OpenAI custom GPT to a Node.js backend that provides natural-language access to an employee directory. It builds an Express API with a POST /search endpoint, where OpenAI’s Chat Completions API converts a user’s question into a single SQL query, the database returns rows, and a second OpenAI call turns those rows into a natural-language response. The project uses Heroku Postgres, environment configuration, bearer-key authentication, Swagger UI, and an OpenAPI specification imported into the GPT action. The deployment flow covers seeding dummy records, deploying from a GitHub-connected repository, and configuring the action to call the hosted API, while noting that data leaves Heroku’s trust boundary for OpenAI services.


### [How to Use pgvector for Similarity Search on Heroku Postgres](https://yomu.fyi/post/how-to-use-pgvector-for-similarity-search-on-heroku-postgres.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Valerie Woolard
- Published: Nov 16, 2023
- Type: Tutorial

Heroku Postgres now supports pgvector on all Standard-tier or higher databases running Postgres 15, adding vector storage and functions for similarity search and machine-learning applications. Older Postgres versions require an upgrade to Postgres 15. The workflow generates embeddings, adds a VECTOR(100) column, and stores values using SQL or Python with psycopg, pgvector-python, and Wikipedia2Vec’s pretrained 100-dimensional model. Queries can rank vectors with Euclidean, cosine, or inner-product distance; the example uses <-> to return five animals nearest to shark, producing crocodile, dolphin, whale, turtle, and alligator. Results depend on the embedding model and corpus, while indexes introduce approximate nearest-neighbor search; pgvector supports IVFFlat and HNSW, with IVFFlat requiring table data for better recall.


### [Abstraction Engineering](https://yomu.fyi/post/abstraction-engineering.md)
- Company: [Ramp](https://yomu.fyi/company/ramp.md)
- Author: Rodda John, Jared Wasserman
- Published: Oct 31, 2023
- Type: Explainer

Ramp’s early product encoded opinionated limits, but by fall 2022 customers needed configuration based on departments, accounting fields, subsidiaries, budgets, and HRIS fields, making incremental feature work increasingly complex. The team reframed these requests as a generic workflows platform in which Python functions served as actions, boolean logic as conditions, and workflows as graphs persisted in Postgres. After rejecting a dependency-based design, they built a graph-traversal engine, a Postgres-backed queue that retries asynchronous tasks, and an SDK for expressing constructs such as elif and else. The infrastructure now supports customer-facing policies and internal tasks; more than 45 million workflows have run, at over a million per day, while the core engine has not changed since launch. The project took weeks upfront but saved the engineering organization months by turning complex features into applications of broader abstractions.


### [pgvector: Fewer dimensions are better](https://yomu.fyi/post/pgvector-fewer-dimensions-are-better.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: gregnr,oli\_rice,egor\_romanov
- Published: Aug 3, 2023
- Type: Benchmark

Embeddings support semantic search, retrieval augmented generation, and clustering in pgvector, but high-dimensional vectors increase storage, memory, and distance-computation costs as datasets grow. The post explains how IVF indexes reduce the records examined and compares embedding models using Massive Text Embedding Benchmark results, noting little correlation between dimension count and performance. In a 224,482-vector Wikipedia dataset, all-MiniLM-L6-v2 with 384 dimensions delivered 78% higher queries per second than OpenAI text-embedding-ada-002 with 1536 dimensions at accuracy@10 of 0.99, while using 4GB rather than 7.5GB of RAM. The 384-dimensional gte-small model produced more than a 200% queries-per-second improvement over all-MiniLM-L6-v2 at the same accuracy, though model choice still depends on task performance, sequence length, language support, and operational costs.


### [pgvector 0.4.0 performance](https://yomu.fyi/post/pgvector-0-4-0-performance.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: egor\_romanov,pavel
- Published: Jul 13, 2023
- Type: Benchmark

The post benchmarks pgvector 0.4.0 for approximate nearest-neighbor search and assesses whether PostgreSQL can support production-scale vector workloads. Using the ANN Benchmarks methodology, pre-warmed RAM, the dbpedia-entities-openai-1M dataset of 1 million 1,536-dimensional OpenAI embeddings, and Supabase ARM instances, it varies probes, lists, concurrency, and compute size. On an 8-core, 32 GB instance, pgvector reached 0.91 accuracy@10 at 380 QPS with 10 probes and 0.98 accuracy at 140 QPS with 40 probes; a 64-core, 256 GB server reached about 1,800 QPS at 0.91 accuracy. The results indicate predictable scaling, while higher lists can improve query speed at the cost of index-build time, and the roadmap identifies HNSW, product quantization, and parallel index builds as planned improvements.


### [What is new in PostgREST v11.1?](https://yomu.fyi/post/what-is-new-in-postgrest-v11-1.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: steve\_chavez
- Published: Jul 12, 2023
- Type: Announcement

PostgREST v11.1 is available on the Supabase platform with configuration and querying improvements, plus a small set of documented breaking changes from v10. It lets JWT-authenticated impersonated roles such as anon, authenticated, and service\_role receive ALTER ROLE settings, including statement timeouts, pg\_plan\_filter statement cost limits, and configurable transaction isolation levels. The release also supports bulk JSON inserts that preserve column defaults when defaultToNull is false, and adds any/all modifiers for several filters, including eq, like, ilike, and comparison filters. For HTTP clients outside Supabase libraries, Range is now ignored on non-GET requests and RPC requests no longer use the deprecated Prefer: params=multiple-objects header; existing Supabase projects can upgrade by pausing and unpausing.


### [Supabase Beta June 2023](https://yomu.fyi/post/supabase-beta-june-2023.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: ant\_wilson
- Published: Jul 6, 2023
- Type: Announcement

Supabase’s June 2023 beta update announces product, tooling, authentication, billing, and community developments. Mozilla chose Supabase Vector and OpenAI for AI Help, an AI chatbot that supports search and discovery for MDN, while Supabase Auth added native Sign in with Apple and Google support for one-tap methods including web integrations and Chrome extensions. The Supabase CLI added Dashboard migration visibility with executed SQL, local serving for all Functions, and a usage flag for monitoring key Postgres statistics; Studio also gained usage, overage, plan-limit, subscription, billing, and compute-detail improvements. Other updates include Kakao login, a Realtime Inspector for testing endpoints with JWTs and Row Level Security policies, an object ID in Storage upload responses, OrioleDB parallel index builds, database partitioning guidance, community projects, and an upcoming Launch Week.


### [Supabase Beta May 2023](https://yomu.fyi/post/supabase-beta-may-2023.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: ant\_wilson
- Published: Jun 9, 2023
- Type: Announcement

Supabase’s May 2023 update announces two major product areas: Supabase Vector, an open-source toolkit for storing, indexing, and querying Postgres vector embeddings at scale, and Supabase Vault, a Postgres extension and UI for encrypted secrets and data. Auth Helpers now support client and server use of the Next.js App Router and server-side authentication by default with PKCE. The release also adds Turnstile CAPTCHA support, password reauthentication nonces, Wrappers integrations for Firebase, Stripe, S3, and clickhouse, CLI deployment for all Edge Functions, custom and vanity domains, and generally available Image Transformation. Supabase says it reached 50,000 GitHub stars, while the team solicits Dashboard SQL Editor feedback and reports community projects, AI resources, and a first-ever Flutter Hackathon.


### [Supabase Vecs: a vector client for Postgres](https://yomu.fyi/post/supabase-vecs-a-vector-client-for-postgres.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: oli\_rice
- Published: May 29, 2023
- Type: Announcement

Vecs is a new Python library for managing embeddings in Postgres databases that use pgvector, offering a vector-store-like interface for table creation, indexing, querying, and metadata filtering. Collections are stored in a vecs schema with an id text primary key, a vector column, and JSONB metadata; queries support cosine distance, l2 distance, and maximum inner dot product, with optional filters and returned values. A short Python workflow can create a collection, upsert records, create an index after insertion, and search by vector; the collection can also be queried without an index. With Supabase, a collection can be exposed through a public-schema view and accessed from supabase-js, while proposed preprocessing transforms and an asynchronous client remain under debate.


### [ChatGPT plugins now support Postgres & Supabase](https://yomu.fyi/post/chatgpt-plugins-now-support-postgres-supabase.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: egor\_romanov,paul\_copplestone
- Published: May 25, 2023
- Type: Tutorial

Retrieval plugins let ChatGPT access relevant content from private datasets by chunking documents, generating embeddings with OpenAI’s text-embedding-ada-002, and querying a vector database. The article describes Postgres and Supabase datastore implementations using pgvector, with a shared documents schema and a match\_page\_sections function that filters by source, author, document, and date before ranking vectors with inner product distance. It then gives a local example: downloading the Postgres 15 documentation as a PDF, uploading it through the plugin’s /upsert-file endpoint, and configuring environment variables for Supabase or Postgres. The example produced 7,904 document records, and an optional IVFFlat index can trade exact nearest-neighbor search for approximate search; after registering the local plugin in ChatGPT, users can ask questions answered from the documentation.


### [Vanilla with Candy Sprinkles](https://yomu.fyi/post/vanilla-with-candy-sprinkles.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: May 18, 2023
- Type: Tutorial

Vanilla with Candy Sprinkles presents a baseline full-stack Node.js application: a stateful visitor counter that increments across refreshes, tabs, browsers, and machines. The post starts with a dependency-free package.json and a 72-line server.js, then uses node-demo to generate variants with ECMAScript modules, EJS or Mustache templates, and Express. It adds SQLite3 or PostgreSQL for persistence, Tailwind CSS and TypeScript with a build step, and Drizzle, Knex, or Prisma as ORM options, noting differing language and build requirements. WebSockets provide real-time updates, htmx can handle client-side connection and DOM work, and Redis coordinates notifications across multiple servers; Fly.io deployment can scale the setup across eight instances in four regions. The concluding section points to further alternatives, including React, Angular, Lit, SolidJS, Svelte, Vue, esbuild, rollup, and other deployment targets, while inviting alternate implementations.


[Newer posts](https://yomu.fyi/topic/postgres/page/7.md) · [Older posts](https://yomu.fyi/topic/postgres/page/9.md)