Loading…
Latest reads
The engineering internet, summarised so you can actually read it.
Grab ·
Embracing passwordless authentication with Grab’s Passkey
Grab introduced Passkey to replace vulnerable traditional passwords and cumbersome multi-factor methods with a seamless, phishing-resistant alternative based on the FIDO standard. The architecture relies on an authenticator located on the user's device, a frontend client, and a backend storing only public keys and metadata. During registration and login, the frontend invokes WebAuthn APIs such as navigator.credentials.create and navigator.credentials.get using server-generated challenges to prevent replay attacks. Passkeys synchronize across ecosystems via Google Password Manager and Apple iCloud Keychain, allowing users to authorize logins with their device lock screen. This implementation improves user experience, eliminates the need to store secrets in backend databases, and cuts third-party communication costs associated with OTP delivery.
Ocean NguyenCanva ·
Canva incident report: API Gateway outage
On November 12, 2024, canva.com suffered an outage lasting approximately one hour due to cascading API Gateway failures. The disruption began when a stale Cloudflare routing rule induced severe latency and packet loss between Ashburn and Singapore during an editor asset deployment. Cloudflare's concurrent streaming mechanism consolidated over 270,000 user requests for a single delayed JavaScript chunk; when the asset finally loaded, clients simultaneously generated a 1.5 million requests-per-second thundering herd to Canva's API Gateway. This surge collided with a telemetry performance regression involving lock contention on Netty event loop threads, exhausting off-heap memory and causing the Linux Out Of Memory Killer to terminate API Gateway containers. Canva mitigated the failure by temporarily blocking all traffic at the CDN level, stabilizing replacement Amazon ECS tasks, and gradually restoring traffic under strict rate limits.
Brendan HumphreysStripe ·
Crush errors with Sandbox testing
Payment integrations can encounter errors caused by external events, such as network issues or outages, or by bugs in application code, and unhandled failures can stall checkout. The post presents Stripe Sandboxes as isolated environments for reproducing live-integration failures without changing live payment settings or traffic, optionally copying settings when a sandbox is created. Developers can use Workbench’s Errors and Logs views to diagnose a card_declined response, then route test requests through sandbox keys and a test card. On the Python backend, a try/except implementation catches Stripe errors and generic exceptions and returns an error response instead of leaving the payment page stalled. Sandboxes remain separate after creation, but each Stripe account can have at most five, so unused environments may be deleted or have their test data cleared while settings are preserved.
David Edoh-BediStripe ·
Testing Connect onboarding with Sandboxes
Stripe Connect testing can become unwieldy as platforms cover payment methods, US states, international countries, and merchant verification conditions, while the existing test mode stays synchronized with live settings. Sandboxes provide an isolated environment that can replicate a platform’s live configuration without affecting live payment traffic. The workflow covers creating a Sandbox, confirming that Connect settings such as negative-liability handling were preserved, and onboarding test connected accounts through Stripe-provided or custom flows. To reproduce a website-verification failure, a custom account can use “https://inaccessible.stripe.com”; after testing an update to “https://accessible.stripe.com” in the Sandbox, the platform can apply the fix to its production onboarding flow.
David Edoh-BediStripe ·
Developing a modern architecture for energy utilities with embedded finance
Energy utilities can use embedded finance to create marketplace-like experiences for tariffs, solar installations, smart-home devices, storage, and related services while capturing more energy-related revenue. The proposed foundation combines Stripe Online Payment Flows, Billing, Payments, Connect, and Data Pipeline with AWS serverless services, including Lambda, EventBridge, DynamoDB, Redshift, S3, and QuickSight. A tariff enrollment or payment can trigger webhooks and real-time event processing, while transaction and customer data support dashboards, usage insights, personalized recommendations, subscriptions, and broader partner offerings. The architecture also describes PCI Level 1 security, encryption, Radar, Strong Customer Authentication, AWS IAM, KMS, Shield, WAF, Config, Security Hub, and CloudWatch for compliance and monitoring. It concludes that this integrated payment, analytics, and security foundation can help utilities improve customer engagement, support sustainable programs, and develop new revenue streams.
Rajan PatelStripe ·
How do I store inventory data in my Stripe application
The DevRel Swag Store uses Stripe payments and AWS services to keep product inventory accurate and visible in near real time. Stripe products retain core payment information, while Amazon DynamoDB stores inventory and other attributes, keyed by Stripe product ID and store ID; EventBridge routes payment events to Lambda, which atomically decrements stock and publishes updates through AWS IoT Core to the frontend. DynamoDB condition expressions prevent decrements when stock is insufficient, while post-payment validation and refunds address the delay before a Stripe Payment Link is disabled, although Stripe fees may be non-refundable. A custom Payment Intent flow can check stock before payment, authorize funds with manual capture for up to seven days, update inventory after payment success, and then capture the charge.
Ben SmithStripe ·
Japan community highlights: Effective testing and security
This article reports practical lessons from two September 2024 JP_Stripes events in Aizuwakamatsu and Sapporo for developing and operating Stripe-integrated services more efficiently and at lower cost. Sandboxes let teams create up to five separate test environments, reproduce payment failures and state transitions through Stripe's API, and use CI-specific workspaces without production access. Stripe Connect examples show how a three-person codoc team launched in nine months by staggering account creation and embedding payout and payment-management interfaces. For fraud prevention, the article recommends Radar or Radar for Teams with webhook automation, including early-fraud-warning events and preemptive cancellation or refunds when dispute fees exceed transaction value. Together, these cases emphasize using managed Stripe capabilities and community-shared implementation experience to reduce testing effort, UI work, and fraud-related costs.
Hidetaka OkamotoGrab ·
Turbocharging GrabUnlimited with Temporal
GrabUnlimited experienced scaling bottlenecks, corrupted membership states, and elevated production incidents after its subscriber base grew by over 1000%. The original architecture relied on Amazon SQS state machines, 5-minute Redis locks, and daily batch cron jobs that overwhelmed the database and lacked granular idempotency during upstream retries. To eliminate these failure modes, the engineering team migrated the core membership lifecycle to Temporal's workflow orchestration engine. Replacing batch cron jobs with Temporal Timers distributed renewal operations throughout the day, while matching workflow IDs prevented race conditions between renewals and cancellations. This architectural transition resolved database bottlenecks and yielded an 80% reduction in open production incidents.
Michel ParrenoCanva ·
The science of routing print orders
Canva's global print network requires selecting optimal suppliers to balance delivery times, packaging counts, and environmental emissions. To resolve these challenges before user checkout, the engineering team designed a modular routing architecture that decouples graph construction, decision logic, and path traversal. During graph traversal, the system generates action objects capturing forward paths and decision query results, which are compiled into timestamped routing logs in blob storage for asynchronous auditing. Utilizing preprocessed graph queries alongside ElastiCache, Redis, and database read replicas, the infrastructure sustains high-throughput evaluation without coupling cost logic to traversal code. As a result, print routing completes within an average of 50 milliseconds at the 99th percentile during peak usage while maintaining 99.999% data availability.
Constantinos KavadiasStripe ·
Enhance your monitoring by integrating Stripe events with AWS CloudWatch Log Groups
Stripe events notify account owners about changes such as successful charges, failed invoice payments, and available reconciliation reports. Because Stripe retains events for 13 months but exposes older events than 30 days only as summaries, Amazon EventBridge can route them into an AWS account and Amazon CloudWatch Log Groups can provide longer-term monitoring and analysis. The setup uses CloudWatch metric filters to match event types such as invoice.payment_failed, convert matches into custom metrics, and support alarms, Amazon SNS notifications, dashboards, and Logs Insights queries. For example, a failed-invoice alarm can use a five-minute observation period, Sum statistics, and a threshold based on expected failure volume or anomaly detection. The resulting setup supports near-real-time visibility into successful charges and failed payments, historical trend analysis, troubleshooting, and responses to unusual activity.
Andrew RobinsonGrab ·
How we seamlessly migrated high volume real-time streaming traffic from one service to another with zero data loss and duplication
Grab split a backend service's read and write functionalities into separate services to allow independent scaling. Migrating the write path required transferring processing from 16 source Kafka streams—averaging 20,000 reads per second into DynamoDB tables and output streams—with zero data loss or duplication. Standard feature flags were ruled out because rollout propagation delays could introduce minutes of duplicate or missing data during flag toggling. Instead, engineers extracted processing logic into a shared monorepo commons package that used coordinated timestamps to trigger simultaneous cutovers across both services. Temporary validation sinks verified processing accuracy in production prior to the cutover, completing the stream-by-stream migration across three weeks without downtime.
Md RiyadhGrab ·
Supercharging LLM application development with LLM-Kit
Grab developed LLM-Kit to resolve standardization, security, observability, and infrastructure provisioning bottlenecks across its generative AI development efforts. Submitting an initial application request automatically triggers project generation in GitLab, outputting a modular codebase along with Terraform definitions for Amazon EKS and ECR. The scaffolded architecture incorporates FastAPI, LangChain, OpenID Connect authentication helpers, and PGVector alongside HashiCorp Vault for credential management. Observability and quality tracking are built in through direct integrations with Datadog and LangSmith evaluations. By providing these standardized paved-road templates, the framework has onboarded hundreds of generative AI applications and saved teams an estimated 1.5 weeks of initial setup time.
Boon Zhan ChewStripe ·
Data access patterns for simple Stripe integrations
The article examines how applications should store and access product data in simple Stripe integrations as requirements for security, performance, and scalability evolve. It compares using Stripe’s built-in product fields with a separate database, while explaining that publishable API keys cannot retrieve product details and that secret or restricted keys must remain server-side. A web backend or serverless function can proxy requests securely, while CloudFront caching reduces repeated calls and latency but requires an appropriate refresh cadence. For richer metadata, inventory, variants, or custom attributes, the article describes combining DynamoDB with Lambda and Stripe, using Event Destinations or webhooks to synchronize changes and weighing that flexibility against database overhead.
Ben SmithStripe ·
Managing multiple Stripe test environments from your AWS-hosted application
Stripe sandboxes provide isolated test accounts that let teams manage multiple environments from one account. Unlike legacy test mode, each sandbox has separate data, its own API keys, and configurable user access, while up to five sandboxes can exist per account. They support simulated external events, fake balances, Test Payouts with API v2 keys, and CLI or SDK access by changing keys. For AWS-hosted applications, the recommended pattern stores sandbox and production keys in AWS Secrets Manager, using secret names as environment-neutral aliases rather than embedding environment logic in code. Teams can combine Stripe sandbox permissions with AWS IAM resource policies and directory groups to restrict which users or accounts can retrieve keys and prevent accidental production access.
James BeswickStripe ·
Getting started with Stripe in the UAE: A comprehensive guide for developers
The guide explains how developers can use Stripe for UAE-based e-commerce businesses, following its public debut in the country in April 2021. It covers Stripe Billing for recurring billing, Stripe Connect for marketplaces, Stripe Radar's machine-learning fraud protection, and the platform's support for Apple Pay, Google Pay, and more than 135 currencies including AED, BHD, and KWD. Account setup requires business details, KYC documentation based on entity type, and a linked bank account, with sole proprietors and free zone establishments allowed to use personal accounts under the stated conditions. The Dashboard supports payout and transaction tracking, customer and product management, real-time updates, data exports, tax configuration, and Payment Links, while UAE payouts are described as arriving in AED or USD on a T+5 business-day schedule.
Soad AbuelnagaGrab ·
How we reduced initialisation time of Product Configuration Management SDK
GrabX operates as Grab's central platform for product configuration management, where client services fetch configuration data via an eventually consistent SDK. Services handling around 400 MB of configuration data experienced startup cold starts taking approximately four minutes, creating service stress during traffic spikes. The engineering team resolved this bottleneck through a multi-phase optimization of how the SDK retrieves data from AWS S3. First, sequential downloads of common and service-specific datasets were replaced with concurrent fetching. Next, concurrent downloading and memory loading were applied across large configurations within subscribed services, followed by the complete removal of an outdated disk-caching fallback mechanism. Benchmarks across diverse configuration payloads showed an overall initialisation time reduction of up to 90%.
Ram Dilip PradhanStripe ·
Using demo data for testing Stripe integrations in AWS-hosted applications
Stripe sandboxes let developers manage multiple test environments from one Stripe account for AWS-hosted applications, extending Stripe’s test mode with simulated external events, fake balances, and Test Payouts using API v2 keys. The post describes seeding a sandbox with products and other Stripe objects by exporting production data to CSV for import or generating records with the Stripe CLI and scripts. For local use, developers authenticate with stripe login, while CI scripts pass a sandbox key to each CLI request and can optionally delete the created products afterward. It recommends storing the key in AWS Secrets Manager, retrieving it at runtime through the AWS CLI or SDK, restricting access with IAM, avoiding logs and source repositories, and rotating keys; this lets CI scripts use sandbox data without exposing credentials.
James BeswickStripe ·
Avoiding silent errors in your Stripe integration
Stripe integrations can appear healthy while API and webhook failures accumulate unnoticed when applications lack robust logging and alerting. The post demonstrates Stripe Workbench, a Dashboard view that stores available information by default, obfuscates sensitive data, and provides searchable logs without code changes or additional software. In the example, about 50% of daily API calls fail; the Errors tab groups failures and connects each type to sample requests and related logs, revealing repeated card-verification errors and a request associated with an “Attacker Engineering Test Key” whose Go user-agent conflicted with the stated .NET backend. The investigation identifies a team testing attack-like API calls in production and recommends stopping production-key testing, avoiding test requests in production, and using Sandboxes.
Abe HaskinsTinder ·
How Tinder Eased Development Pain With Ignis
Managing hundreds of microservices with disparate configurations forced Tinder engineers to deploy code to staging environments to verify every change, slowing development velocity. To shorten feedback loops, the team created Ignis, a Go-based CLI tool that lets engineers run an isolated service locally while proxying ingress and egress traffic through a multi-tenant development Kubernetes cluster. Ignis pairs with an API gateway for client testing and uses a custom Smart Router proxy to dynamically intercept downstream gRPC and HTTP requests via Kubernetes informers and header propagation. Since launching in 2019, the tool enabled Tinder to eliminate a dozen staging environments, decrease cloud spend, and accelerate onboarding, followed by a V2 release adding remote IDE support and monitoring.
TinderGrab ·
Metasense V2: Enhancing, improving and productionisation of LLM powered data governance
Grab scaled Metasense to automate metadata generation, column-level classification, and sensitivity tiering across its entire data lake. Post-rollout analysis revealed that high tag volumes, wide tables, and mixed content—such as nested JSON and customer communications—strained model capacity and led to missed Personally Identifiable Information (PII). To overcome these limitations, the team split the classification process into distinct PII and non-PII tasks, reduced prompt word counts, and partitioned tables with over 150 columns into smaller units. Integrating LangChain and LangSmith modernized the architecture by enabling direct prompt experimentation, custom metric tracking, and version-controlled deployments. The updated pipeline achieved low misclassification rates alongside automated alert thresholds to trigger model improvement protocols if errors increase.
Nick Buhrer