---
title: "Latest reads"
description: "The engineering internet, summarised so you can actually read it."
---

# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [Scaling Nextdoor’s Datastores: Part 1](https://yomu.fyi/post/scaling-nextdoor-s-datastores-part-1.md)
- Company: [Nextdoor](https://yomu.fyi/company/nextdoor.md)
- Author: Slava Markeyev
- Published: Mar 19, 2025

Nextdoor relies on a Django backend connected to monolithic PostgreSQL databases and Redis look-aside caches. Migrating to distributed SQL datastores proved impractical because legacy business logic depends heavily on multi-table joins that could not be rewritten. Intermediate mitigations, including read replicas and data partitioning by severing foreign keys, extended infrastructure runways but left primary databases vulnerable to load bottlenecks. Furthermore, look-aside caching and replica usage introduced data staleness risks, atomicity losses across partitioned databases, and inconsistent cache-population behaviors. To address these limitations, Nextdoor initiated an architecture redesign focused on dynamic query routing to replicas, replica-driven cache hydration, time-bounded eventual consistency, and schema-resilient cache serialization.


### [Stripe’s new AI Assistant in VS Code](https://yomu.fyi/post/stripe-s-new-ai-assistant-in-vs-code.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: Mathew Varughese
- Published: Mar 18, 2025

Stripe has introduced an AI Assistant in its VS Code extension to answer developer questions using current Stripe knowledge rather than relying on LLM pre-training. The extension retrieves API reference entries, integration guides, code examples, and curated summaries of developer Discord threads, while inserting the user’s API key into generated snippets and supporting either GitHub Copilot’s @stripe agent or its own chat interface. Its retrieval pipeline classifies queries, combines BM25 keyword and k-nearest-neighbor embedding search, reranks results, and sends selected sources, code, user code, and the question to Claude Sonnet through a RAG prompt. A nightly Temporal workflow refreshes indexed content, while evaluation uses golden and synthetic datasets, Mean Reciprocal Rank, and human and automated review. On its synthetic test dataset, the system includes the best source about 91.11% of the time and reports an MRR of about 78%.


### [Real-time payment analytics: Building a data pipeline from Stripe to AWS](https://yomu.fyi/post/real-time-payment-analytics-building-a-data-pipeline-from-stripe-to-aw.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: James Beswick
- Published: Mar 14, 2025

Payment processors such as Stripe emit numerous transaction events, while dashboard and API-based or batch approaches can limit historical analysis, real-time visibility, and correlation with business metrics as volumes grow. The proposed pipeline sends signed Stripe webhooks through API Gateway to Lambda for validation and enrichment, buffers them in Kinesis Data Streams, and uses a second Lambda to transform and index records in OpenSearch. OpenSearch Dashboards provides visualizations, while CloudWatch tracks pipeline health; Kinesis retention and replay support recovery and reprocessing. The post also describes a Kinesis Data Firehose route that can remove the consumer Lambda and reduce costs but may add batching latency. The resulting design is presented as scalable to millions of transactions per day, with sub-minute payment metrics visibility and flexible historical querying, while security guidance includes encryption, IAM configuration, and regular audits.


### [Building a Spark observability product with StarRocks: Real-time and historical performance analysis](https://yomu.fyi/post/building-a-spark-observability-product-with-starrocks-real-time-and-hi.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Huong Vuong
- Published: Mar 6, 2025

Grab redesigned its Spark observability platform, Iris, to overcome limitations associated with its previous Telegraf, InfluxDB, and Grafana stack. InfluxDB presented operational challenges due to limited SQL compatibility, poor handling of string metadata, and query degradation on high-cardinality identifiers. The team replaced InfluxDB with StarRocks to serve as a unified analytical engine for both real-time cluster metrics and historical analysis. StarRocks ingests metrics directly from Kafka via routine load tasks, storing worker and Spark event data in partitioned duplicate-key OLAP tables linked by worker and application identifiers. This architecture eliminated intermediate ingestion agents, simplified S3 data lake backups, and enabled a custom web application alongside Superset for consistent querying.


### [Load balancing Stripe API calls from multiple AWS regions](https://yomu.fyi/post/load-balancing-stripe-api-calls-from-multiple-aws-regions.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: James Beswick
- Published: Mar 6, 2025

The guide describes a multi-region payment-processing architecture for applications that use Stripe’s API and need resilience against regional outages, latency, rate limits, and payment-data compliance constraints. It routes requests with Amazon Route 53 health checks and configurable latency-based, weighted, or geolocation policies, while identical regional API Gateway and Lambda deployments process payments. DynamoDB Global Tables replicate payment state and support distributed rate limiting, with optimistic locking, regional token-bucket allocation, retries, and exponential backoff for Stripe requests. The sample Lambda records a payment as PENDING, creates and confirms a Stripe PaymentIntent, then updates the state with the Stripe payment ID and returned status; the conclusion presents the design as a foundation for regional resilience, state consistency, and API-rate management.


### [Importing sales data from Stripe into AWS](https://yomu.fyi/post/importing-sales-data-from-stripe-into-aws.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: Hidetaka Okamoto
- Published: Mar 4, 2025

The article presents a low-code method for analyzing Stripe business and customer data and moving scheduled report results into AWS. Stripe Sigma lets users query payment, subscription, and plan-related data with SQL, while Sigma Assistant can generate and execute queries from natural-language prompts such as cancellation analysis. For delivery, the article uses the sigma.scheduled\_query\_run.created event and Stripe's Event Destination to Amazon EventBridge, where a partner event bus and event rule trigger AWS services without a separate public webhook API. Its Lambda example downloads the report CSV with the Stripe API key, parses it into JavaScript objects, and publishes the result through Amazon SNS; the same event-driven pattern can support Lambda, Step Functions, or Glue workflows.


### [TechDocs at Grab: Cultivating a culture of quality documentation](https://yomu.fyi/post/techdocs-at-grab-cultivating-a-culture-of-quality-documentation.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: David Khu
- Published: Feb 27, 2025

Engineering organizations frequently struggle with fragmented documentation, stale content, and a lack of clear ownership across disparate tools. To address these issues, Grab established TechDocs on its central Helix platform, embedding a Docs-as-Code workflow into daily engineering routines. Feedback gathered from quantitative surveys and one-on-one sessions shaped governance policies, separating stable platform documentation stored in GitLab from collaborative artifacts like RFCs in Confluence. To sustain document freshness, the platform assigns mandatory points of contact, displays last-updated timestamps, and flags pages untouched for more than three months.


### [Building serverless usage notification with AWS](https://yomu.fyi/post/building-serverless-usage-notification-with-aws.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: Hidetaka Okamoto
- Published: Feb 27, 2025

Pay-as-you-go services need a way to warn customers when consumption reaches a configured limit, but building usage aggregation, monitoring, and notification workflows adds operational complexity. This tutorial uses Stripe Billing Alerts to define a customer-specific usage threshold with a customer\_id, meter\_id, gte value, and recurrence, then sends billing.alert.triggered events through Stripe Event Destinations to an Amazon EventBridge partner event source. It tests the flow with Stripe Meter Events and shows AWS CLI rules routing the event to Amazon SNS, as well as a Lambda example that retrieves the customer's email from Stripe and sends mail with Amazon SES. The result is a minimal-code notification path using AWS services, with sandbox setup required because some APIs and features may not be available in conventional test mode.


### [How the Tinder iOS App reduced the size of our localizations by 95% using Emerge](https://yomu.fyi/post/how-the-tinder-ios-app-reduced-the-size-of-our-localizations-by-95-usi.md)
- Company: [Tinder](https://yomu.fyi/company/tinder.md)
- Author: Tinder
- Published: Feb 26, 2025

Supporting over 50 languages across numerous statically linked targets led to a substantial build size footprint in Tinder's iOS application. Because Apple's code signing mandates a minimum 4KB per file, shipping dozens of localized files per target inflated the final application package. Tinder addressed this by stripping comments and whitespace, merging localized strings into a single file per language via custom Bazel rules and Aspects, and compressing the strings using Emerge's SmallStrings tool into LZFSE files. The runtime decompresses these files dynamically through existing code-generated string accessors. Consequently, Tinder reduced download size by 10.7MB and install size by 51.3MB with no impact on developer workflows.


### [Behind the scenes of Canva's DesignDNA campaign](https://yomu.fyi/post/behind-the-scenes-of-canva-s-designdna-campaign.md)
- Company: [Canva](https://yomu.fyi/company/canva.md)
- Author: Divya Patel
- Published: Feb 24, 2025

Canva launched DesignDNA in December 2024 as a personalized year-in-review campaign to highlight user achievements and showcase generative AI capabilities. Because internal privacy rules strictly prohibited inspecting personal designs, the engineering and creative teams inferred user preferences from the style and theme metadata tags on public templates. An initial keyword-matching algorithm paired 95% of users with one of seven emerging design trends, while generative AI keyword expansion increased coverage to 99%. The team also synthesized over one million localized poems and distinct design personalities using tools like Magic Write and Dream Lab. Finally, dynamic Canva template elements populated through URL parameters assembled 95 million distinct multi-page stories across nine locales.


### [Securing Stripe API Keys in AWS with automatic rotation](https://yomu.fyi/post/securing-stripe-api-keys-in-aws-with-automatic-rotation.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: James Beswick
- Published: Feb 21, 2025

The article presents a production-grade approach to securing Stripe API keys in AWS, motivated by the risks and operational limits of basic secret storage for payment processing. It compares AWS Parameter Store with Secrets Manager, emphasizing built-in rotation, CloudTrail audit trails, IAM integration, larger secrets, and cross-account access despite higher cost. The implementation uses environment-specific secret paths, tags, and IAM conditions to isolate development, staging, and production credentials, with separate rotation schedules. For zero-downtime rotation, applications refresh cached credentials after authentication failures while both old and new keys remain valid during a transition. The design also covers CloudWatch monitoring, emergency rotation procedures, cross-region replication, region-specific schedules, and cost considerations, concluding that ongoing review is necessary for secure and reliable payment operations.


### [Grab AI Gateway: Connecting Grabbers to multiple GenAI providers](https://yomu.fyi/post/grab-ai-gateway-connecting-grabbers-to-multiple-genai-providers.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Bjorn Jee
- Published: Feb 19, 2025

Grab built the AI Gateway to centralize access, cost control, and security across external and open-source Generative AI providers such as OpenAI, Azure, AWS, and Google. Designed as a set of lightweight reverse proxies, the gateway manages authentication, rate limiting, and authorization while translating payloads into a unified OpenAI-compatible interface. The platform archives request metadata and calculated per-call costs into a central data lake for auditing and showback, dynamically routing traffic across shared reserved capacity and regions to mitigate quota throttling. Supporting over 300 internal use cases, the system integrates directly with internal development notebooks and deployment tools to power applications ranging from real-time audio safety analysis to automated content moderation.


### [Tracking customer spend in an omnichannel or multiprocessor environment](https://yomu.fyi/post/tracking-customer-spend-in-an-omnichannel-or-multiprocessor-environmen.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: Andrew Robinson
- Published: Feb 19, 2025

Merchants operating across websites, mobile apps, physical stores, and multiple payment processors struggle to connect spending because customers use physical cards, digital wallets, and tokenized methods. Payment Account Reference (PAR), a 29-character alphanumeric identifier introduced by EMVCo, provides a non-financial, non-reversible reference that maps one PAN to multiple tokens without exposing sensitive payment data. Because the PAR is independent of payment method and processor, merchants can link transactions across channels and preserve continuity when a card is replaced with a new PAN. The post presents an omnichannel retail example and describes potential benefits including unified customer profiles, personalization, cross-channel reconciliation, reduced PAN-related PCI burden, and processor flexibility.


### [Building rock-solid Stripe integrations: A developer's guide to success](https://yomu.fyi/post/building-rock-solid-stripe-integrations-a-developer-s-guide-to-success.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: James Beswick
- Published: Feb 6, 2025

The guide presents ten practices for building production-ready Stripe integrations, covering payment forms, subscriptions, and operational concerns. It recommends verifying webhook signatures with the Stripe-Signature header and endpoint secret, recording event IDs for idempotency, and returning a fast 2xx response while processing asynchronously. Test and live API keys, webhook endpoints, logs, and monitoring should remain separate, while error handling should distinguish card, request, and API failures and use exponential backoff for retries. The guide also recommends testing successful, declined, insufficient-funds, and 3DS scenarios with Stripe test cards, alongside simulated webhook events. Its conclusion emphasizes edge-case planning, logging, monitoring, and ongoing review as foundations for reliable, secure payment processing that can scale with business needs.


### [AI Core Team Lead Mike Schuster on How to Get the Most From LLMs](https://yomu.fyi/post/ai-core-team-lead-mike-schuster-on-how-to-get-the-most-from-llms.md)
- Company: [Two Sigma](https://yomu.fyi/company/two-sigma.md)
- Author: Joy Looney
- Published: Feb 4, 2025

Mike Schuster, Head of the AI Core Team at Two Sigma, advocates for grounding large language model adoption in practical tasks rather than speculative industry hype. Realistic enterprise applications focus on accelerating data processing, running faster experiments, and extracting domain-specific features from transcripts such as earnings calls and Federal Reserve speeches via prompt engineering. Because financial data faces inherent volume limits across trading days, successful deployments require multidisciplinary human teams to balance rapid technical experimentation with rigorous domain expertise and analytical reasoning. Schuster also dismisses predictions that programming will become obsolete, comparing coding to learning a musical instrument that cultivates structured thinking, problem decomposition, and scientific common sense essential for building complex predictive models.


### [Building resilient webhook handlers in AWS: Implementing DLQs for Stripe events](https://yomu.fyi/post/building-resilient-webhook-handlers-in-aws-implementing-dlqs-for-strip.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: James Beswick
- Published: Jan 30, 2025

Reliable Stripe webhook processing must account for lost deliveries from network or service outages, out-of-order events, and duplicates caused by Stripe retries. The proposed AWS architecture uses API Gateway for signature validation and throttling, an SQS FIFO queue for ordered delivery and content-based deduplication, Lambda for processing, DynamoDB for event-ID idempotency, and an SQS DLQ for failed messages. In the CloudFormation example, the main queue has a 300-second visibility timeout and sends messages to the FIFO DLQ after three receives; DynamoDB records expire through a seven-day TTL. Lambda retries failures with exponential backoff, while CloudWatch monitors queue depth, latency, and errors. The design is presented as a scalable foundation, with multi-region failover available at added cost and complexity, though single-region deployment may suffice for many applications.


### [Sharing Tinder’s latest contributions to the open source community](https://yomu.fyi/post/sharing-tinder-s-latest-contributions-to-the-open-source-community.md)
- Company: [Tinder](https://yomu.fyi/company/tinder.md)
- Author: Tinder
- Published: Jan 29, 2025

Tinder open-sourced several iOS development repositories, including Layout, Nodes Architecture Framework, and CombineUI, to share the engineering patterns supporting its iPhone application. As the app expanded across diverse device profiles and scale, engineering teams faced reliability, consistency, and memory challenges under their legacy architecture. In response, Tinder developed Nodes, a plugin-based architecture framework using compile-time dependency injection and lifecycle hooks that enforce complete memory release upon feature dismissal. For interface construction, Tinder created Layout, a domain-specific Auto Layout wrapper offering declarative syntax for UIKit views to eliminate storyboard merge conflicts while preserving native capabilities. The resulting stack enables isolated testing of business logic, native reactive event binding, and incremental adoption of SwiftUI across the codebase.


### [New to Stripe? Learn the key concepts for software developers.](https://yomu.fyi/post/new-to-stripe-learn-the-key-concepts-for-software-developers.md)
- Company: [Stripe](https://yomu.fyi/company/stripe.md)
- Author: James Beswick
- Published: Jan 29, 2025

The guide introduces software developers to Stripe’s main objects and workflows for integrating payments, recurring billing, and related account activity. It begins with Payment Intents, which specify an amount and currency, track a payment’s lifecycle, and produce a client secret for completing the frontend flow, then covers Payment Methods and Customers, including Setup Intents and attaching a default payment method. For subscriptions, it distinguishes Products, Prices, and Subscriptions, and explains using events and webhooks to react to asynchronous payment and subscription changes while verifying webhook signatures. It also describes Disputes, refunds, and expanded API responses, including retrieving related resources in one call. The guide recommends starting in test mode and using dashboard logs while consulting Stripe’s documentation and changelog as the platform evolves.


### [Image replacement in Canva designs using reverse image search](https://yomu.fyi/post/image-replacement-in-canva-designs-using-reverse-image-search.md)
- Company: [Canva](https://yomu.fyi/company/canva.md)
- Author: Sam Jacobs
- Published: Jan 28, 2025

Canva needed an automated way to replace media in design templates, such as when third-party licensing partnerships expire across more than 150 million images. Existing recommendation engines, perceptual hashing, and text metadata searches failed to capture visual similarity hierarchies or ensure replacement relevance. To build a reverse image search system, engineers evaluated embedding models including CLIP, ViTMAE, DreamSim, CaiT, and DINOv2 alongside an external vector database supporting metadata filtering. Evaluation on sample datasets identified DINOv2 as the best model for preserving subjects, background context, and color tones in photos. Integrated into the Template Assistant as a human-in-the-loop tool, the automated suggestions increased image replacement speeds by 4.5 times during initial pilot testing.


### [The foundations of Canva’s continuous data platform with Snowpipe Streaming](https://yomu.fyi/post/the-foundations-of-canva-s-continuous-data-platform-with-snowpipe-stre.md)
- Company: [Canva](https://yomu.fyi/company/canva.md)
- Author: Jack Caperon
- Published: Jan 6, 2025

As Canva expanded to over 200 million monthly active users, ingestion throughput reached 25 billion records daily, causing AWS Data Firehose costs to consume nearly half of the product analytics platform budget. To reduce these expenses and eliminate intermediate file staging, the engineering team integrated Snowflake's Snowpipe Streaming directly with their Java-based Kinesis Data Streams pipeline. The architecture streams records directly into Snowflake tables using logical channels with offset checkpoints while configuring client buffering up to a five-minute maximum lag. In handling Kinesis Client Library edge cases, record processors coordinate with the LeaseCoordinator to drop leases when channels become unhealthy. Operating in production for over six months, the system ingested more than 20.35 petabytes of data, decreased query latency to roughly ten minutes, and reduced overall cloud spend by 45 percent.


[Newer posts](https://yomu.fyi/page/33.md) · [Older posts](https://yomu.fyi/page/35.md)