Loading…
Topic
816 posts about Performance. Every summary links to the original.
See the idea
Bare metal gives engineers direct control over GPU, network, and virtualization layers, but it shifts elasticity and lifecycle work onto the team. This trade-off map shows when that bargain pays off.
Bun packs a runtime, package manager, bundler, and test runner into one binary. Here is how those four parts fit together and where the seams still show.
A five-step how-to for deciding when an AMD Instinct GPU should stay dedicated and when SR-IOV virtual functions, VMs, or sidecar isolation can safely share it.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Grab moved its macOS CI/CD infrastructure from a US cloud vendor to a colocation cluster in Malaysia, closer to its Git servers in Southeast Asia. The change followed growth from one office Mac Pro to more than 250 Mac minis, rising demand, job queues, and cloud costs that made ownership or colocation more attractive at scale. Grab selected bare-metal hardware after comparing cloud, colocation, and on-premises options, then migrated progressively while monitoring failure rates, performance, and reliability; the later infrastructure also uses zero-touch provisioning to reduce manual setup as the cluster grows. The migration produced projected savings of more than $2.4 million over three years and performance improvements of 20–40%, while the team chose allocated-kilowatt power commitments and retained bare metal because macOS virtualization’s performance or stability trade-offs did not justify it.
Puma 6 and earlier could let a keep-alive connection “cut the line” when all threads were busy, allowing up to ten requests from that connection to run consecutively while previously accepted work waited. The behavior came from reusing a “fast inline” optimization intended for HTTP pipelining, where requests arrive together, for keep-alive requests that can arrive later. Puma 7 changes queueing so pipelined and keep-alive requests return to the back of the queue, and adjusts wait_until_not_full so interleaving is possible; its accept loop now slows proportionally with load instead of stopping completely. The fix does not add capacity, but distributes overload delay more fairly across requests. The post recommends upgrading to Puma 7.1.0 or later, especially for applications that had disabled keep-alives after latency spikes.
Etsy’s search and web performance teams tested the Speculation Rules API (SRA) to reduce loading time for product listing pages reached from desktop organic search. They added a speculationrules configuration that prefetches eligible listing HTML after a 200-millisecond hover, using URL and data-attribute selectors with moderate eagerness; unlike link prefetch, SRA uses both memory and HTTP caches. The experiment required attention to a two-page memory limit, five-minute retention, non-cacheable listing HTML, video hover behavior, cookies, new-tab targets, and analytics events triggered during prefetches. Results showed 23.6% faster 75th-percentile TTFB, with improvements of 20.7% in FCP, 21.1% in LCP, 20.4% in DOMContentLoaded, and 10.6% in Page Load, while roughly one in 14 prefetches led to navigation.
Composer is a mixture-of-experts agent model for software engineering that targets interactive use, with reported frontier coding results and generation speed four times faster than similar models on the authors’ benchmarks. It is trained with reinforcement learning on real-world software engineering challenges in large codebases, using production search and editing tools, terminal commands, and codebase-wide semantic search. Training responses may be code edits, plans, or informative answers, while the process rewards efficient tool use, parallelism, minimal unnecessary responses, and evidence-based claims. Evaluation uses Cursor Bench, built from real agent requests and curated optimal solutions, measuring correctness alongside adherence to existing abstractions and software engineering practices. The training stack combines asynchronous reinforcement learning with PyTorch and Ray, MXFP8 MoE kernels, expert parallelism, hybrid sharded data parallelism, and sandboxed cloud environments scaled to hundreds of thousands concurrently.
Training machine learning models on multi-terabyte datasets often requires long download times or suffers from network request storms when multiple DataLoader workers initialize independently. Hugging Face revamped the backend of its datasets library to make dataset streaming viable for large-scale distributed training without modifying the standard API. Startup latency was reduced by caching resolved data file lists across workers and bundling API calls, cutting initial request volume up to one hundred times. The team also implemented background prefetching for Parquet files and added configurable buffering parameters to keep GPUs fully saturated during training runs. Benchmark tests demonstrated up to twice the sample processing throughput and zero worker crashes across 256 concurrent workers on 64 H100 GPUs.
LeRobot v0.4.0 expands the open-source robot-learning stack with scalable datasets, simulation benchmarks, processing tools, policies, and hardware integrations. LeRobotDataset v3.0 introduces chunked episodes, video streaming, unified Parquet metadata, improved loading and memory efficiency, plus utilities for editing and converting datasets. A modular Processor system separates policy and robot pipelines, while direct Accelerate integration simplifies multi-GPU training; the release also adds LIBERO, Meta-World, PI0, PI0.5, and GR00T N1.5 support. A plugin architecture enables third-party robots, cameras, and teleoperators to be installed without changing the core library, with Reachy 2 and phone teleoperation included, alongside a self-paced Robot Learning Course and hands-on tutorial.
Intel and Hugging Face benchmarked OpenAI GPT OSS text generation to evaluate performance on Google Cloud C4 virtual machines powered by Intel Xeon 6 processors against previous-generation C3 instances. The benchmark evaluated the unsloth/gpt-oss-120b-BF16 Mixture of Experts model across batch sizes from 1 to 64 using bfloat16 precision, static KV cache, and SDPA attention. To eliminate redundant computation, Hugging Face merged an optimization directing each expert sub-network to process only its routed tokens rather than all tokens. Testing demonstrated that C4 instances deliver 1.4x to 1.7x higher throughput per vCPU compared to C3 instances. At a batch size of 64, C4 achieved a 1.7x Total Cost of Ownership improvement alongside lower hourly pricing.
Deploying Vision Language Models locally presents computational challenges despite providing data privacy and independence from external servers. Optimum Intel and OpenVINO address this by optimizing lightweight models like SmolVLM2-256M for low-resource hardware, including Intel CPUs and GPUs. The process involves exporting the model to the OpenVINO Intermediate Representation format, applying 8-bit weight-only or static quantization, and executing inference with the transformed model. In single-image CPU benchmarks on an Intel Core Ultra 7 265K processor, OpenVINO with 8-bit weight-only quantization reduced time to first token from 5.150 seconds under PyTorch down to 0.247 seconds. The optimized configuration also raised decoding throughput from 0.722 to 63.928 tokens per second.
To alleviate a disk throughput bottleneck on a MongoDB server, this optimization pass focuses on shrinking document sizes across application revisions appV6R0 through appV6R4. The implementation replaces an array of items with a dynamic schema document, using date segments such as days or combined months and days as field names to store status totals. Redundant date components already encoded in the binary _id field are omitted from internal document keys. Bulk upsert operations rely on updateOne with the $inc operator to update matching date keys or create documents when missing. Aggregation pipelines process reports using the $objectToArray and $reduce operations to calculate totals across date intervals while preserving a single index on _id.
An overloaded database faced heavy traffic from campaign usage tracking, motivating an in-memory counter rather than an immediate SQL-to-NoSQL migration. Because exact counts were not required, increments could be cached in the application server and flushed periodically to reduce database queries and CPU use. The implementation uses Go’s Sync.Map, with LoadOrStore and CompareAndSwap for concurrent increments, plus LoadAndDelete to atomically remove values for asynchronous persistence; the source notes a fresh-map swap as an alternative locking optimization. On an Apple M4 Pro, the 2,000-key benchmark reported 53 ns/op and 1 billion operations per 60 seconds, versus 160 ns/op and 466 million operations for a Mutex-protected map, but Sync.Map used 88 B/op and four allocations. In production, update queries fell from 140 QPS to 45 QPS, master CPU from 35% to 18%, and replica CPU from 19% to 12%.
Transitioning from relational databases to MongoDB often leads developers to replicate SQL habits, causing severe performance bottlenecks, brittle schemas, and operational challenges. Effective document design requires balancing referencing and embedding based on query access patterns and update frequencies, rather than placing every entity into separate collections or excessively nesting large documents. Database performance depends on matching indexes to specific query shapes and field orders, which developers can systematically verify by inspecting execution plans using explain plans. Utilizing the built-in aggregation framework allows filtering, grouping, and calculations to run natively inside the database, eliminating slow, manual data transformations in application code. Establishing early monitoring for latency, replication lag, memory usage, and cluster failover resilience ensures stability before adopting advanced tools like Atlas Search and Vector Search.
Running competitive models like RedNote's 3B parameter dots.ocr on-device provides zero-cost, network-free optical character recognition while leveraging power-efficient hardware like Apple's Neural Engine. Because the Neural Engine requires Core ML, converting the 1.2B parameter NaViT vision encoder from PyTorch requires resolving multiple graph tracing incompatibilities. The initial conversion pipeline targets FLOAT32 execution on GPU by capturing the execution graph with torch.jit.trace and compiling it via coremltools. Simplifying the model for single-image inference eliminates dynamic sequence masking, dynamic tensor iteration, and multi-attention complexity. Although the resulting Core ML model accurately matches PyTorch baseline numerical precision, the unoptimized artifact spans over 5GB and takes over one second per forward pass.
Local agentic AI workflows generate multi-step reasoning traces that increase token counts and strain real-time inference on personal computers. To accelerate execution, researchers implemented speculative decoding using OpenVINO.GenAI on an Intel Lunar Lake integrated GPU, pairing a 4-bit Qwen3-8B target model with a smaller Qwen3-0.6B draft model. By measuring angular distance across layers, the team pruned six of the draft model's twenty-eight layers to minimize draft latency and fine-tuned it on synthetic data generated from 500,000 prompts. This depth-pruned draft elevated speculative decoding throughput from a 1.3x speedup to a 1.4x speedup over the standalone baseline. Finally, deploying the optimized model alongside the Hugging Face smolagents framework allowed local agents to perform tool invocation and code generation tasks efficiently.
MongoDB has released the general availability of the MongoDB SQL Interface for MongoDB Enterprise Advanced on self-managed deployments running MongoDB 6.0 or higher. Previously, teams relying on SQL-based business intelligence tools faced maintenance bottlenecks when learning MongoDB Query Language or creating custom extract, transform, and load pipelines. The new interface translates standard SQL queries into native MongoDB operations in real time using MongoSQL, a SQL-92 compatible dialect. Users utilize a Schema Builder CLI to generate JSON schemas that map nested document structures and store them alongside data for query validation. This native query translation replaces the MongoDB BI Connector, which will remain supported until September 2026, while offering improved performance and direct connectivity via JDBC and ODBC drivers.
BharatPE, an Indian fintech processing over ₹12,000 crore monthly across 450 cities, operated a 45-terabyte self-hosted MongoDB Community Edition deployment. Managing three sharded three-node clusters resulted in uneven data distribution, high maintenance costs, scaling friction, and disaster recovery gaps under strict regulatory requirements. To modernize infrastructure, the database operations team partnered with MongoDB to execute a structured five-stage migration to MongoDB Atlas. The initiative utilized mongosync for secure data transition alongside environment mirroring and automated integrity validation scripts. Transitioning to MongoDB Atlas yielded a 40 percent improvement in query response times and provided automated failover backed by a 99.995 percent uptime SLA.
Migrating core insurance platforms from legacy relational databases with PL/SQL to Java and MongoDB Atlas frequently degrades batch processing performance. Like-for-like migrations can cause batch jobs to run 25 to 30 times slower due to high network round-trips, inefficient per-record operations, and under-utilized bulk capabilities. To resolve these bottlenecks, an optimization framework was developed that leverages MongoDB native bulkWrite operations, intelligent reference data prefetching, and parallel execution. An architecture combining a Spring Boot controller and a dedicated executor framework partitions workloads across threads or event processors. This framework recovered lost performance, completing previously failing batch jobs within service-level agreements and achieving 10 to 15 times better execution speeds than legacy systems in certain workloads.
At MongoDB.local NYC, MongoDB introduced new database capabilities, AI integrations, and modernization tools designed for agentic and generative AI workloads. The release of MongoDB 8.2 arrives alongside public previews of Search and Vector Search for both MongoDB Community Edition and Enterprise Server. MongoDB also highlighted Voyage AI embedding models and rerankers to improve retrieval accuracy across raw data, metadata, and embeddings. To address the costs and constraints of legacy infrastructure, the new MongoDB Application Modernization Platform combines AI-assisted tooling and specialized expertise to migrate legacy systems. According to MongoDB, early migrations using this platform run two to three times faster, while code rewriting tasks accelerate by an order of magnitude.
Grab’s decade-old Go monorepo grew to 12.7 million commits, 73GB of Git-tree metadata, 176GB of blobs, 12 million references, and 444,000 files, causing replication lag, slow developer workflows, and GitLab operational bottlenecks. A staging proof of concept compared full and single-commit histories under production-like load: replication delay fell from 160–240 seconds to 1–2.5 seconds, nearly a 100x improvement. Because custom migration tooling processed about 100 commits per hour, the team retained commits reachable from more than 2,000 critical tags, one month of recent history, and simplified merge commits, while archiving older history. The resulting prune reduced commits by 99.9%, improved replication performance by 99%, preserved code and tags without data loss, and left teams with minor migration adjustments.
Stripe built a real-time streaming analytics system for Billing after customers said they needed to detect emerging behavior and adapt pricing quickly, replacing a batch pipeline whose subscription updates lagged by 24 hours on average. The architecture uses Apache Flink to maintain compressed subscription history and incrementally process analytics events, while a custom Apache Spark job generates initial Flink state from billions of historical events and supports validation and exports. A new Apache Pinot v2 query engine performs windowed aggregations and joins without offline preaggregation; most updates are processed in under one minute, nearly all reach users within 15 minutes, and production query latency is under 300 milliseconds. Metric-definition changes trigger historical recalculation while new events are buffered and later replayed, preserving a consistent Dashboard view during the transition.
LeRobotDataset:v3.0 is a new release for lerobot, Hugging Face’s robotics library, addressing file-system limitations encountered when LeRobotDataset:v2 stored one episode per file at million-episode scale. Version 3 packs multiple episodes into shared Parquet and MP4 files, while relational JSON and chunked episode metadata identify episode boundaries and locate the corresponding tabular and visual data. The format separates tabular data, visual data, and metadata; it stores low-dimensional readings in Apache Parquet, concatenates camera frames into MP4 videos, and supports native temporal windowing through delta_timestamps. It also introduces StreamingLeRobotDataset for processing batches from the Hugging Face Hub without downloading the full dataset locally, and provides a one-line conversion utility for datasets in the LeRobotDataset format. The release is planned for lerobot-v0.4.0 and is presented as a step toward datasets with millions of episodes.
The best engineering reads, once a week.
