Loading…
Topic
812 posts about Performance. Every summary links to the original.
See the idea
Bare metal gives engineers direct control over GPU, network, and virtualization layers, but it shifts elasticity and lifecycle work onto the team. This trade-off map shows when that bargain pays off.
Bun packs a runtime, package manager, bundler, and test runner into one binary. Here is how those four parts fit together and where the seams still show.
A five-step how-to for deciding when an AMD Instinct GPU should stay dedicated and when SR-IOV virtual functions, VMs, or sidecar isolation can safely share it.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Standard transformer models face severe memory and compute bottlenecks due to the quadratic scaling of self-attention with sequence length. To scale beyond short sequence limits, efficient architectures introduce varied mechanisms such as custom sparse attention patterns, compressed recurrence, low-rank projections, and kernel approximations. Longformer integrates local dilated windowed attention with task-specific global tokens, functioning as a drop-in replacement that enables standard pre-trained models to process long inputs without full retraining. Concurrently, Compressive Transformers extend temporal memory by compressing discarded activations, while models like Linformer and Performer reduce computational complexity through projection and kernel estimation. Progress in long-range modeling also reveals that incrementally training models from short to longer sequences accelerates convergence and improves downstream performance.
The guide prepares candidates for PostgreSQL interviews by concentrating on transactional SQL rather than mixing it with analytical topics such as window functions, ranking, and aggregates. It covers relational modeling with foreign keys and join tables, PostgreSQL’s richer types, schemas, indexing strategies, and EXPLAIN ANALYZE for comparing estimated and actual query costs. It also addresses views, automatically updatable views and their limitations, database permissions, roles, PUBLIC grants, row-level security policies, and business logic implemented with PL/pgSQL functions and triggers. The guide recommends thinking in sets with SQL operations such as JOIN and UNION, using common table expressions for readable or recursive queries, and practicing these topics to build a strong foundation for PostgreSQL interview answers.
Shopify’s Test Infrastructure team tackled a 45-minute 95th-percentile CI time for its Core monolith, which had more than 170,000 tests and ran on every git push. They instrumented Buildkite commands and prioritized agent preparation, dependency building, and test execution using command frequency and duration. Disk I/O from writing large caches slowed Docker startup, so the team increased disk capacity and write speed, mounted most caches read-only, skipped unchanged database and asset work through hashes, parallelized setup, and expanded changed-file test mappings. These changes reduced container-start p95 from 90 to 25 seconds, raised builds avoiding the full suite from 45% to over 60%, improved test stability from 88% to 97%, and lowered overall CI p95 to 18 minutes.
Database replicas can improve read-heavy application performance and reduce pressure on primary servers, but differing replication lag can make multi-query reads produce incoherent datasets. Shopify’s Database Connection Management team compared tight consistency and GTID-based causal consistency before choosing opt-in monotonic read consistency, in which related reads are routed to one replica and therefore follow a consistent timeline rather than necessarily seeing the latest data. Their ProxySQL fork accepts a UUID in the consistent_read_id query comment, hashes it into a pseudorandom index over a weighted hostgroup server list, and disables later rebalancing that could override the selection. The implementation indexes configured servers before handling outages, though a selected server failure can still break consistency. The approach offers low overhead but tolerates occasional inconsistencies.
Huggingface Transformers added the Retrieval Augmented Generation architecture to combine standard sequence-to-sequence models with external document retrieval for knowledge-intensive tasks. Scaling RAG fine-tuning across multiple GPUs previously created bottlenecks because the torch.distributed implementation forced the single rank 0 worker to handle all retrieval index lookups. To resolve this synchronization bottleneck, Ray was integrated into the contextual document retrieval mechanism using stateful actor abstractions. Dedicated Ray processes separate from the training workers now load the large index and process retrieval requests concurrently. This architectural separation delivers up to a 2x speedup per retrieval call and removes the strict dependency on PyTorch for training.
PyTorch / XLA integrates Cloud TPU training into Hugging Face transformers while preserving the standard Trainer interface. The implementation adds an xla device type and uses lazy execution to trace intermediate representation graphs before lowering them to XLA Higher Level Operations for compilation. Parallel data loading through MpDeviceLoader overlaps CPU tracing with TPU execution, while xm.optimizer_step coordinates gradient consolidation across data-parallel replicas. Checkpoints use xm.save to transfer tensors via CPU storage and restrict disk writes to the master process. Benchmarks on a Cloud TPU v3-8 system training bert-large-uncased on WikiText-103 demonstrate completion times of 178.4 minutes in FP32 precision and 106.4 minutes in BF16 precision.
Supabase released a collection of platform updates during January 2021 covering database tools, authentication integrations, regional availability, and infrastructure performance. Client libraries added support for PostgREST exact, planned, and estimated count queries to eliminate counting friction commonly experienced in platforms like Firebase. Authentication capabilities expanded to include Facebook and Azure login providers, an in-dashboard GoTrue audit trail, a helper function for user emails, and a prebuilt React Auth Widget. The platform also enabled new database hosting regions in London and Sydney while transitioning storage to GP3 for improved throughput. Furthermore, migrating API subdomains to Route53 alongside custom Let's Encrypt certificates yielded read performance benchmark gains of up to twelve percent.
Grab's Drivers Data service handles up to 10,000 requests per second during peak hours to supply driver information across backend microservices. The original setup used MySQL with Redis and standalone in-memory local caches, but yielded a low 25% local cache hit rate due to traffic patterns characterized by high burst frequency for individual drivers alongside redundant database calls across nodes. To solve this, the team developed Mirror Cache, an in-memory caching system that pairs Dgraph's Ristretto library with an asynchronous gRPC replication layer to mirror updates across cluster nodes. The replicator batches updates within the same AWS availability zone and forwards data to single nodes across zones to minimize transfer overhead. Production deployment increased the in-memory cache hit rate to approximately 75% and reduced direct MySQL queries by 5%.
Hugging Face updated Transformers to improve the computational performance and robustness of its TensorFlow models, including BERT, RoBERTa, ELECTRA, and MPNet. In version 4.2.0, the optimized BERT implementation runs up to approximately 10 percent faster than Google's official implementation and twice as fast as Transformers release 4.1.1 when tested on a V100 GPU. The release also improves TensorFlow SavedModel generation by allowing flexible sequence lengths across runs, exposing all model inputs, and grouping hidden states or attention outputs into single tensors. In addition, users can customize serving signatures by subclassing models and decorating custom methods with explicit input specifications. Models exported to the SavedModel format can be packaged into Docker containers and served via TensorFlow Serving using either HTTP REST or gRPC APIs for production inference.
GraphQL is presented as a way to address mobile developers’ REST API pain points, especially repeated JSON-to-model mappings and failures when server fields or types change. It provides a strongly typed, self-documented schema contract, while introspection enables tooling to generate client code and surface deprecations at compile time. The post describes mobile practices including using reusable UI-oriented fragments, aliases for readable data models, and directives that conditionally include or skip fields; Shopify uses the open-source Syrup tool to generate strongly typed Swift and Kotlin code from application queries. This flexibility shifts responsibility toward query design: Shopify calculates field costs, limits calls through a replenishing per-user cost bucket, and requires clients to consider query complexity for responsiveness and server resources. GraphQL is positioned as language-independent and suited to bandwidth, latency, and UX-sensitive clients.
Machine learning model sizes frequently outpace consumer GPU memory, preventing practitioners from loading or fine-tuning large architectures on standard hardware. To address these resource constraints, the Hugging Face Trainer introduced experimental support for Zero Redundancy Optimizer features from DeepSpeed and FairScale starting in transformers v4.2.0. In dual-GPU benchmarks with a t5-large model, FairScale and DeepSpeed integrations expanded allowable batch sizes and shortened training durations compared to baseline DistributedDataParallel runs. Furthermore, DeepSpeed permitted single-GPU fine-tuning of a t5-3b model on a 24GB RTX-3090 card at batch size 20, whereas the baseline execution crashed immediately with an out-of-memory error. These optimizations manage memory allocation internally to curb memory fragmentation while requiring changes only to training scripts rather than model code.
Deploying large transformer models in production poses severe latency and scaling hurdles for machine learning engineers. Hugging Face achieved a cumulative 100x inference acceleration on its Accelerated Inference API by pairing library-level algorithmic adjustments with low-level hardware compilation. The initial 10x improvement reduces forward-pass computations, such as restricting attention calculations to the latest token in GPT generation tasks, while utilizing Rust-based tokenizers and caching mechanisms. The subsequent 10x speedup requires custom static graph compilation targeted to selected CPU or GPU hardware profiles, applying unused flow removal, instruction-specific layer fusion, and precision quantization via ONNX Runtime. These coordinated techniques enable real-time prediction latency across varied model sizes and demand profiles without sacrificing task accuracy.
Trident serves as Grab's internal real-time event-processing and workflow automation engine, driving user campaigns, rewards, and notifications across multiple business lines. To handle peak loads exceeding 2,000 events per second without duplicate execution, the system consumes decoupled Kafka streams and enforces exactly-once semantics using Redis and MySQL deduplication checks. Processing efficiency relies on server autoscaling aligned with Kafka partition counts, combined with dynamic goroutine allocation per consumer. To minimize rule evaluation overhead, Trident indexes active campaigns into an in-memory hash map by event type, cutting processing time by at least 90%. Furthermore, condition evaluation is optimized through lazy loading and a weighted sorting algorithm that checks low-cost in-memory data prior to executing expensive database queries or external service calls.
Supabase transitioned to Beta following ten months of development concentrated on performance, security, and reliability. The platform introduced inline table documentation updates, storing descriptions directly as PostgreSQL comments and exposing them through OpenAPI specifications. Developers can now utilize a table view featuring immediate realtime update reflection alongside pagination for handling large datasets. Performance measurements show a 26 percent increase to 1460 requests per second in regions supporting AWS Graviton instances. Furthermore, the update introduces a new region in Sao Paulo, editing support for native Postgres arrays in the grid editor and side panel, and broader compatibility for custom database types.
Shopify describes using WebAssembly (Wasm) outside the browser to run untrusted Partner code on its own infrastructure, reducing partners’ need to operate scalable web services and avoiding network latency for time-sensitive customizations. Wasm provides a sandboxed, stack-based environment with explicit host imports, protected call stacks, and runtime type checking, while targeting near-native performance and multiple source languages. Shopify wraps Fastly’s open-source Lucet runtime and compiler in a Rust web service called the Wasm Engine, which manages module I/O and storage and applies results during runtime processes such as web requests. In a test executing 100,000 modules per minute for roughly five minutes, Lucet execution hovered around 100 μs and total p99 execution time was about 4 ms. Shopify selected AssemblyScript, with TypeScript-like syntax, for first-class developer support and is extending its tooling, SDKs, monitoring, and language support.
Adding features to the Supabase single-page application dashboard risked performance regressions due to increasing JavaScript bundle sizes. To establish a baseline, the engineering team introduced next-bundle-analyzer and tracked Real User Monitoring alongside Core Web Vitals using Sentry. Optimization efforts focused on pruning dependencies, which included swapping Moment.js for day-js, replacing Joi with ajv, reverting crypto-js to version 3.3.0, and moving the 388 KB zxcvbn password module to a backend API. The team also implemented Next.js dynamic imports for heavy components like spreadsheet parsing, removed legacy server-side props to unlock Automatic Static Optimization, and configured long cache headers on assets. These architectural adjustments brought Core Web Vitals within recommended thresholds and lowered client-side page transition payloads below 200 KB of JavaScript.
The post explains how a new Ruby-based server-side Storefront Renderer reduced the time required to serve Shopify storefront requests. It combines MySQL multi-statement queries, handcrafted SQL, a thin data-mapping layer built from plain old Ruby objects, query book-keeping with eager- and lazy-loading, multiple LRU caching layers, and techniques for reducing memory allocations. For a product page, one database round trip can load the product, variants, images, shop, theme, and related resources; later requests can replay previously observed queries early, while less frequently used data remains lazy-loaded. The resulting renderer serves 75% of requests in under ~45ms, 90% in under ~230ms, and 99% in under ~900ms, with average response time nearly five times faster than the previous implementation.
Shopify organized a cross-company program to prepare its platform for Black Friday and Cyber Monday (BFCM), coordinating more than 2,500 people responsible for over 1,000 deploys across services. Engineering Program Managers absorbed administrative, communication, and reporting work so BFCM leads could focus on technical challenges, while departmental champions assigned preparation tasks. Preparation covered maintenance, game days, load tests, capacity increases, weekly tracking, and a Caution Tape bot requiring risk-to-value assessments for pull requests in critical areas. Deploys were slowed through smaller batches, longer canaries, and a planned weeklong lock while communication used Slack, GitHub, documentation, presentations, emails, and office hours. The effort coincided with $5.1+ billion in BFCM sales and load testing plus resource scaling paid off, but developer productivity declined amid inconsistent messaging; future plans called for a more balanced approach, automation, and reusable communication templates.
The 2020 Black Friday Cyber Monday Globe and a related Live View visualization had to represent orders from one million merchants within two months while remaining realistic and performant. The team used a reusable layer architecture with three.js, physically based rendering, 32-bit EXR environment maps, and custom materials and shaders for the Earth and carbon-offset bubble effects. To display thousands of orders, it replaced arc-heavy rendering with particle-based visuals built from Points, BufferGeometry attributes, and ShaderMaterial. An automatic optimizer monitors frame rate and, when it stays below 55 fps for more than two seconds, lowers canvas pixel ratio to 1x and reduces environment-map resolution, helping low-power devices return to 60 fps. The resulting implementation became an internal library for future globes.
Shopify prepares for high-traffic sales events such as BFCM by repeatedly simulating customer traffic on representative test shops, aiming to expose bottlenecks before merchants experience them. Its Platform Conditioning team supports service-owning teams with approachable performance-testing tools and processes, distinguishing load tests for known traffic levels from stress tests that push services toward failure. A Go load generator runs Lua flows through lightweight VMs, models realistic traffic shapes and costly endpoints, and can issue tens of millions of requests per minute or more. Tests are initiated through Slack using the internal ChatOps tool Spy, which returns links to dashboards, logs, and results. Cronograma provides formal experiments with hypotheses, observations, conclusions, alert detection, Slack timelines, and comparable metrics such as response times, 5xx errors, and requests per minute; the tools have led to performance improvements and give Shopify confidence in future sales events.
The best engineering reads, once a week.
