Loading…
Topic
812 posts about Performance. Every summary links to the original.
See the idea
Bare metal gives engineers direct control over GPU, network, and virtualization layers, but it shifts elasticity and lifecycle work onto the team. This trade-off map shows when that bargain pays off.
Bun packs a runtime, package manager, bundler, and test runner into one binary. Here is how those four parts fit together and where the seams still show.
A five-step how-to for deciding when an AMD Instinct GPU should stay dedicated and when SR-IOV virtual functions, VMs, or sidecar isolation can safely share it.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
The article presents a five-step workflow for deciding when and how to build machine learning products in a business setting. It recommends delaying machine learning for new products, establishing data-collection foundations, and testing simple heuristics as baselines before investing in a model. For an initial model, teams should favor simple algorithms, limited feature sets, and off-the-shelf solutions where appropriate, then measure business-aligned impact before optimizing. The workflow also calls for planned model iteration and explicit comparison with existing versions, while accounting for prediction stability when changes could affect user trust. The authors say they used these practices at Shopify to help build and scale machine learning products and to productionize models intended to drive impact.
The investigation starts with production Ruby code that chains Hash#fetch calls and uses an empty-hash default to return an IdentityObject for missing nested data. It compares that workaround with Hash#dig, which is cleaner but does not provide fetch's same control over missing keys, defaults, and explicit nil values. To prototype a combined Hash#dig_fetch method, the author modifies TruffleRuby, using Primitive's hash_get_or_undefined and undefined? to distinguish absent keys from keys containing nil, then replaces recursive traversal with an iterative implementation in shared Diggable logic. For a hash with nine nested keys, iterative changes lift dig from about 2.5M to 16M iterations per second and lift dig_fetch from about 2.5M to 15.5M. The dig_fetch prototype enables the original refactor, but it is not ready for general Ruby adoption because Array and Struct interoperability and corresponding MRI work remain.
GrabFood.com faced high cloud costs while serving over 1 TB of weekly network egress across 175 million requests. To minimize egress and improve page performance, the engineering team audited and reduced their webpack dependencies using tools such as webpack-bundle-analyzer and dependency-cruiser. Their strategies included lazy loading rarely used libraries, unifying duplicate modules under single entry points, and substituting utility libraries like axios with native browser Web APIs. Additionally, altering implementation approaches, such as replacing client-side signed JWT cookie encoding with plain JSON strings, eliminated heavy Node dependencies like crypto. These optimizations reduced JavaScript static assets from 750 KB to 250 KB, decreased CloudFront costs by 20%, and accelerated build times by 3.6x.
Supabase announced a series of platform upgrades and open-source contributions centered around PostgreSQL and its broader tooling ecosystem. Every new project on the platform now deploys on PostgreSQL version 13.3, accompanied by marketplace images on Digital Ocean and AWS. PostgREST version 8.0 incorporates upstreamed performance enhancements, achieving up to 50% more throughput on GET requests by adopting prepared statements for SELECT queries and decreasing log verbosity. The release eliminates downtime during schema cache reloads and replaces the pg_listen requirement with standard NOTIFY commands for dynamic in-database configuration updates. Additionally, community-led efforts introduced beta releases of Flutter and Dart client libraries alongside a dedicated Discord server for developer discussions.
New Supabase projects now run on Supabase Postgres 13.3, transitioning the database platform forward from PostgreSQL 12.0. This release introduces features such as native UUID generation with gen_random_uuid, parallel index vacuuming, incremental sorting, smaller btree indexes, and extended statistics improvements. The underlying PostgreSQL binary was built from the ground up to enable Large System Extensions on AWS Graviton2 ARM instances, optimizing locking and synchronization performance. Alongside upgrading the underlying operating system to Ubuntu 20.04 for long-term support through 2025, the team integrated fail2ban to protect direct database connections on ports 5432 and 6543 against brute-force attacks. Finally, Supabase Postgres version numbering now mirrors upstream PostgreSQL major and minor versions to make tracking installed releases easier.
Shopify describes a playbook for running large engineering programs as platform complexity grows across products, tooling, and architecture. The framework begins with a Program Plan covering the problem statement, objectives, guiding principles, definition of done, risks, mitigation, staffing, scope, and a path to completion. Execution uses six-week cycles that revisit unfinished goals, regressions, emerging risks, and the planned route to completion; the described program spans 200 people, nine sub-organizations, and 92 projects. Supporting rituals include weekly updates, status checks, risk and escalation triage, cycle reviews, retrospectives, RFCs, and performance testing with Lua scripts orchestrated by Genghis. The post concludes that engineering program management must adapt to each organization and that a combination of scheduled and ad hoc rituals helped Shopify pursue its goals.
Grab transitioned its monolithic mobile application into a modular architecture to resolve increasing code conflicts, slow releases, and difficult team collaboration. The team decomposed the single module by establishing base infrastructure modules, shared UI and utility libraries, discrete feature modules, and bridge kit modules for inter-module communication. Dependency injection using Dagger ties these components together in the main app module while preventing feature modules from directly depending on one another. The architecture spans over 1,000 modules across the app, with more than 200 modules in the Grab Financial Group payments domain where over 95% of modules build in under 15 seconds. This approach accelerated Gradle CI and local builds through parallel compilation and caching, though it increased Gradle sync times, IDE memory usage, and configuration maintenance overhead.
Shopify’s Trino-based interactive query infrastructure faced inconsistent latency and operational failures as workloads grew to 15 Gbps and more than 300 million rows per second. The team analyzed query volumes, resource-heavy datasets, CPU wall time, and failure scenarios, then used a Trino Query Replicator to reproduce historical degradation and isolate issues involving compressed Kafka JSON, shared cluster classes, resource allocation, JVM settings, and dataset statistics. They added workload-specific and ephemeral clusters, reduced concurrent queries, applied recommended JVM recompilation settings, and limited task drivers to 16; they also scaled worker pods to 61 cores and 220 GB memory, although scaling alone was unstable. The infrastructure changes produced a stable environment and reduced P95 execution time to below 30 seconds, moving more queries into the zero-to-five-second range, while the five-second target remained a future goal.
Unpredictable traffic spikes, slow third-party payment requests, and resource-intensive image processing can overwhelm an application, making it slow or unavailable; the text presents background jobs as a way to preserve meaningful user interactions. High availability is defined not merely as server uptime, but as users being able to interact with the application when needed. The application server places a task message in a queue and continues serving requests, while workers process jobs asynchronously, with support for retries, parallelization, prioritization, and event- or time-based scheduling. Offloading improves response times and buffers spikes, but changes immediate guarantees: an upload may be accepted before processing succeeds, and errors may be handled later. For distributed systems, data-based messaging such as RabbitMQ can connect different services, while Kafka's append-only, replayable event log supports shared consumption and event sourcing.
The article introduces GraphQL as an open-source data query and manipulation language for APIs, contrasting it with REST's fixed response shapes and multiple endpoint calls. GraphQL lets clients describe exactly which fields they need, returning that requested structure through a single endpoint rather than a cookie-cutter response. Using a Toronto Eagles example, the article explains how one GraphQL query can combine requests for a team's founding year, captain's name, and last championship, avoiding three separate RESTful trips whose latency can vary. It also uses burger and bank withdrawal analogies to explain over-fetching, under-fetching, flexible frontend iteration, and fewer endpoints, then points readers to a forthcoming Ruby on Rails implementation in part two.
Market-Store, Grab's feature store for real-time machine learning features, experienced latency spikes from under 200 milliseconds to 2 seconds as traffic grew. Metrics and logs showed no direct correlation to API issues, but heap profiling with PPROF revealed continuously increasing memory held by child contexts. Further analysis tracked the leak to an update in Grab's open-source Async Library, which switched background contexts to uncancelled task contexts for worker runners. Because parent contexts maintained references to these uncancelled child contexts, the garbage collector could not reclaim their memory. This progressive memory exhaustion directly degraded API latency.
Shopify’s GraphQL Admin API uses calculated query costs to address two limitations of request-based rate limiting: identical credits for responses of different sizes and equal treatment of reads and side-effecting writes. The server statically analyzes each query, assigning one point to objects and interfaces or unions, zero to scalars and enums, two plus the requested object count to connections, and 10 to mutations. Clients receive 50 points per second up to a 1,000-point limit, while responses expose requested and actual costs; when fewer connection records are returned, the difference is refunded. The post reports a linear correlation between calculated query complexity and execution time, giving Shopify more predictable load for infrastructure scaling and a way to identify performance outliers.
GraphQL clients can face slower, larger queries and difficult roll-outs as product screens gain features and data. The Orders & Fulfillments teams examined these issues while targeting a sub-one-second mobile page load on a reliable network. Starting with a product-list query that loads 100 products, the post recommends pagination with page-size/index controls and a hasNextPage field, plus performance tripwires to track loading time. It describes @include and @skip directives, runtime query construction, and chained queries for feature flags or parameters, while warning that chained requests add latency. For growing screens, it separates unrelated filters, permissions, and banners into parallel queries, enabling independent scaling, partial rendering, and less redundant pagination work, while cautioning that server load and partial-data state require monitoring and possible refactoring.
Shopify’s Polaris team needed to update an illustration style across a product whose assets had become fragmented across more than 6,000 repositories. Illustrators and developers could not reliably identify an image’s repository, code references, reuse, or file type, while GitHub API limits and search behavior prevented dependable large-scale discovery. An initial Node.js monorepo cloned repositories, used globby to find PNG, SVG, and WebP files, searched source lines, and served JSON results through a Preact interface, but rebuilding 24 repositories took more than 35 minutes and produced inaccurate data. The team replaced it with @shopify/get-repo-images, a Go executable that parallelizes file operations, bundles crawling and site generation, and exposes a searchable Next.js interface through NPM. Search and build time fell to eight minutes, with repository cloning as the remaining bottleneck, and the tool was open-sourced.
A May 19, 2021 Shipit! event brought together Colin Gray, Haris Mahmood, Guil Varandas, and Michelle Fernandez, developers setting Shopify’s React Native standards, to discuss writing performant apps. The Q&A covers accessibility and design guidance through Restyle and Polaris documentation; their responsive layouts avoid fixed sizing where possible and use breakpoints for fonts and spacing. They describe React Native state as the usual source of truth while native modules manage internal state and send updates through EventEmitter, with SQLite in Point of Sale as an exception that still has one source of truth. Jest handles unit and integration tests, Buildkite manages CI builds, nightly builds are automatically distributed to testers, and failing feature-branch tests block merges to main.
Scaling BERT-like model inference on CPUs presents practical deployment challenges as transformer architectures proliferate in production environments. Using a reproducible Hydra-based benchmarking framework, inference workloads were evaluated on an Amazon Web Services c5.metal instance featuring an Intel Xeon Platinum 8275 processor with AVX512 and VNNI support. Out-of-the-box evaluations showed PyTorch 1.8.1 delivering lower latency than Google TensorFlow 2.4.0, likely due to PyTorch relying on OpenMP and Intel MKL/oneDNN rather than Eigen. Implementing multiple inference streams by pinning parallel model instances to dedicated CPU cores yielded near-linear throughput scaling. Matching instance core allocations to specific problem sizes also demonstrated that smaller workloads can achieve optimal latency on fewer cores, enabling up to a sixfold reduction in cloud compute costs.
Agent typing time represented a large portion of Grab's chat support journey, and 85% of messages were still free typed because agents customized static templates to fit their personal style. To accelerate typing across multilingual markets without robotic templates, Grab built SmartChat, a machine learning feature that provides contextual sentence completion. The team opted for a lightweight seq2seq architecture using single-layered GRU encoder-decoders in TensorFlow instead of bulky attention models to keep model latency under 100ms. The user interface was implemented in React using a content-editable div with inline typeahead suggestions activated via keyboard shortcuts.
Transformer-based models face severe computational bottlenecks due to quadratic time and memory complexity, making sequences longer than 512 tokens impractical. BigBird addresses this limitation by approximating full attention with block sparse attention, scaling context lengths up to 4096 tokens at a lower computational cost. Instead of attending to all tokens, BigBird combines sliding attention for local context, global tokens for long-range relationships, and random tokens to accelerate information transfer. The model is integrated into HuggingFace Transformers, enabling fine-tuning for tasks like long-document summarization and extractive question answering. When sequences are shorter than 1024 tokens, using original full attention remains recommended.
Grab faced operational bottlenecks managing direct-response Google Ads campaigns across thousands of ad groups due to its hyperlocal marketing across Southeast Asian markets. To eliminate the manual burden of tracking and updating ad creatives, the team built CARA, an in-house automation tool deployed on AWS serverless compute. CARA utilizes standardized file naming conventions to map assets to specific campaigns and connects with Google Ads and YouTube APIs to detect and replace low-performing assets. During an experimental rollout across more than 8,000 active ad groups, CARA replaced nearly 2,000 underperforming creatives. The automated asset replacement workflow produced an 18% to 30% increase in clickthrough and conversion rates.
The post documents Maxence Dominici’s effort to deploy a Transformers sentiment-analysis pipeline on Google Cloud for classifying customer reviews from Discord. After considering AI Platform Prediction and App Engine, he chose Cloud Run with a Docker image because custom checkpoint handling, dependencies, and resource requirements made the other options unsuitable. The implementation uses a PyTorch DistilBERT model fine-tuned on SST-2, Flask, Gunicorn, and a GET request protected by an optional API key; the shown handler retrieves a review but passes the hard-coded text “that was great” to the classifier. Cloud Run is configured with 4 GB of memory, one worker, and one thread to limit memory pressure and billing. Requests take under five seconds including model loading, while cold starts may add about ten seconds, and the author reports that the pretrained model’s accuracy was already impressive.
The best engineering reads, once a week.
