Loading…
Topic
812 posts about Performance. Every summary links to the original.
See the idea
Bare metal gives engineers direct control over GPU, network, and virtualization layers, but it shifts elasticity and lifecycle work onto the team. This trade-off map shows when that bargain pays off.
Bun packs a runtime, package manager, bundler, and test runner into one binary. Here is how those four parts fit together and where the seams still show.
A five-step how-to for deciding when an AMD Instinct GPU should stay dedicated and when SR-IOV virtual functions, VMs, or sidecar isolation can safely share it.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Uber, Oracle Cloud Infrastructure, and Ampere co-optimized the OCI AmpereOne M A4 instance family after migrating large workloads from on-premises data centers to Arm-based cloud infrastructure. Early A1 and A2 deployments exposed dual-socket latency, Go performance degradation associated with smaller TLB and cache sizes, and insufficient single-thread performance for Gurobi services. Uber shared workload characteristics and targets that informed AmpereOne M silicon and an OCI FlexSKU, while the teams evaluated frequencies, core counts, deployment models, memory ratios, and storage. For latency-sensitive stateless services, the resulting guidance favors higher-frequency CPUs, about 8 GB per vCore, and virtual machines; throughput-oriented storage systems use parallelism, flexible memory-to-core ratios, and local NVMe or remote block storage according to locality. The collaboration is presented as improving price-performance and supporting energy-efficient, cost-optimized multi-architecture computing at scale.
Liquid AI released DSpark draft checkpoints for LFM2.5-1.2B-Instruct, LFM2.5-2.6B, and LFM2.5-8B-A1B, adding speculative decoding with a minimal memory increase. DSpark uses a DFlash-style parallel backbone conditioned on target context features, a lightweight sequential Markov-chain head, and a confidence-scheduled verifier that prunes low-confidence suffixes. The draft models are simplified attention-only networks with five layers and a block of nine, trained for 15 epochs on SFT, chat, code, and function-calling data; each contains roughly 300M parameters. Across five datasets, LFM2.5-2.6B averaged 2.67x speedup on an H100 and 2.27x on an M4 Max, while reducing multi-tool function-calling latency by 57% on average. Greedy decoding remains identical to baseline by construction, and integrations are available in llama.cpp and SGLang, with on-device gains for the 8B-A1B model limited by its current MoE Metal implementation.
At POSETTE 2026, Sai Srirampur’s talk examines how storage limits can produce slow ingestion, erratic P95 reads, lagging autovacuum, slower checkpoints, and logical replication lag in large PostgreSQL workloads. The benchmark ran the same 3.3-billion-row, random-UPDATE workload on four local-NVMe clusters and four baseline gp3 EBS clusters, using m6id.4xlarge instances, 64 clients, and 16 worker threads. NVMe reached median 16,030 TPS versus 1,734 TPS on EBS, reduced median latency from 36.9 ms to 4.0 ms, and completed the tested VACUUM and replication-backlog tasks faster. The analysis attributes most of the gap to storage waits rather than CPU work, while the proposed production model combines local NVMe with quorum-based replication, external base backups, and archived WAL for recovery.
Generative recommenders (GRs) recast recommendation as next-item or next-action sequence prediction conditioned on user histories, replacing traditional embedding-similarity objectives and potentially unifying retrieval and ranking. The post presents HSTU, which uses timestamp-ordered interleaved items and actions, SiLU-based attention weighting, relative attention bias, and gating, alongside Semantic IDs, which derive a smaller vocabulary through hierarchical clustering of item embeddings. For production, recsys-examples combines TorchRec, DynamicEmb, Megatron-Core, recommender-specific KV and decoding components, while nv-embedding-cache provides tiered GPU, CPU, and remote embedding storage. On an H100, the Semantic ID path was reported as 1.85x higher in online throughput and 46% lower median latency than SGLang under the stated long-context, beam-search workload, and DLRM v3 reached 99,997 queries/sec.
GoLand 2026.2 introduces day-one support for Go 1.27 language features, tooling improvements, and profiling capabilities. The Go 1.27 release adds generic methods, promoted field names in struct composite literals, improved function type inference, and a dedicated profile for detecting goroutine leaks. To assist with code maintenance, GoLand integrates official go fix modernizers directly into editor inspections, the Problems tool window, and optional pre-commit checks. Developers can capture, visualize, and analyze goroutine leak profiles alongside existing CPU, memory, and mutex profiles within the IDE. Additionally, updated Modern Go Code Guidelines provide AI coding agents with Go 1.27 context and API changes aligned with the version specified in go.mod.
Shopify rebuilt its observability infrastructure after metrics, logs, and traces became fragmented across vendors, with unpredictable costs and limited cross-signal analysis. It created Observe, a unified platform that runs metrics, logs, traces, profiles, and exceptions on self-hosted ClickHouse, treating them as structured, time-ordered, high-dimensional events and serving roughly 50 million events per second normally and 100 million at BFCM peak, or about 110 GB/s. Kafka buffers telemetry for large synchronous durable writes, while materialized views support flexible schemas, autocomplete, typed hot keys, and identifier-based correlation; the latter reportedly makes targeted searches 10x faster. Shopify reports a 16x query improvement out of the box, exceeding 30x at peak performance, with data queryable in under a minute, and says it would likely choose ClickHouse Cloud if starting again because managed operations could reduce the self-hosting burden.
Code-switching automatic speech recognition faces major performance challenges due to the scarcity of code-switching training data. To mitigate this data limitation, an iterative pseudo-labeling framework leverages a large unlabeled audio corpus to generate semi-supervised speech datasets. The system structure executes across three sequential phases: pseudo-label generation, two-stage bilingual model pre-training with supervised code-switching fine-tuning, and progressive iterative improvements. Successive refinement cycles further enhance the neural model's recognition accuracy when processing complex code-switching scenarios across alternating languages. When evaluated on the SEAME benchmark, this training pipeline achieved Mix Error Rate reductions of 6.35 percent on the devman subset and 8.29 percent on the devsge subset.
Yelp developed Menu Vision to help diners recognize menu items and view corresponding user-generated photos and reviews in real time using their phone cameras. The feature evolved from a two-day hackathon Android prototype into a production system deployed across iOS and Android. Client devices perform all optical character recognition, text cleanup, and dish matching locally using native machine learning frameworks, while dish metadata is prefetched upon opening a restaurant page. Backend pipelines combine partner data, reviews, and photo captions into a Cassandra database, filtering exclusively for dishes with photos. To handle optical character recognition errors and name variations, the matching pipeline runs exact matching, bidirectional substring matching, and Jaro-Winkler similarity matching.
NVIDIA Holoscan is a platform for real-time AI applications at the edge, and HoloHub provides reference applications and components. This post describes an engineer-guided workflow in which a coding agent builds a separate Python endoscopic tool segmentation application from the existing MONAI model, sample video, and Holoscan patterns. The Holoscan CLI, invoked through ./holohub, exposes shared commands for scaffolding, running, modes, tests, and benchmarking, while development skills and documentation guide the agent through project-specific operations. The application connects video replay, preprocessing, TensorRT inference, segmentation postprocessing, telemetry, and HoloViz visualization, with visual, smoke, and benchmark modes; the post reports that the CLI-plus-skills-plus-docs/examples configuration produced a separate application in 40 minutes using 11M tokens, and concludes that this combination offered the lowest resource overhead among the compared configurations.
Federating vision-language models (VLMs) across institutions requires keeping images, text, and prompts local while coordinating training and combining updates from sites with different task and modality mixes. NVIDIA FLARE addresses this workflow through server-scheduled rounds, client-local execution, and an explicit update contract, while supporting large-object externalization, pull-based tensor streaming, and disk-backed aggregation for large payloads. The post presents FedUMM as an adapter-based design in which clients train LoRA adapters over a frozen BLIP3o backbone and send only adapter updates, while full-model training remains possible when required. In reported eight-client experiments, adapter-only federation reduced per-client communication from 28.6 GB to 0.094 GB per round, improved VQA v2 by 0.7 points, and retained about 97% of centralized performance on VQA v2 and GenEval; the evaluation used simulated sites and does not establish clinical performance or formal privacy guarantees.
Cloudflare reassessed remote Spectre attack feasibility within Cloudflare Workers production workloads to evaluate defenses against modern stabilization techniques. The experimental proof of concept bypassed Dynamic Process Isolation (DyPrIs) by keeping invocations alive using WebSocket traffic, which also diluted hardware performance counter ratios below detection thresholds. By combining branch mistraining, cache line eviction, and speculative type confusion, researchers transiently accessed out-of-bounds memory via raw 64-bit TypedArray backing store pointers. This attack demonstrated cross-tenant data leakage reaching up to 12 bit/s with over 99% accuracy in production. In response, Cloudflare mitigated the vulnerability by integrating the V8 memory sandbox, deploying Memory Protection Keys for hardware-assisted in-process isolation, and redesigning DyPrIs to monitor active long-lived executions.
Deploying large world models on robots is constrained by onboard memory and the control latency needed for continuous manipulation. The tutorial post-trains NVIDIA Cosmos 3 Edge, a 4B omni-model with a 2B NVIDIA Nemotron-based reasoner, into a DROID action policy using 76,000 successful teleoperated trajectories, then serves it through an OpenPI-compatible WebSocket server on Jetson Thor. The configuration predicts 32 future 8-dimensional joint-position actions at 15 Hz, while a receding-horizon loop executes a prefix and replans from fresh camera and robot-state observations. On an NVIDIA Jetson AGX Thor T5000, inference takes about 1.53 seconds per action chunk covering roughly 2.13 seconds of motion, and the policy reaches 22.9% success in closed-loop RoboLab evaluation; the source contrasts this with Cosmos 3 Nano’s 36.8%.
Long system prompts improve agent behavior but increase inference time, cost, and the GPU capacity needed to serve a given traffic level. Shopify compresses the Sidekick GraphQL agent’s roughly 6,000-token system prompt into about 1,500 gist tokens, using special vocabulary tokens whose embeddings are learned by knowledge distillation while the model weights remain frozen. Training compares teacher logits from the full prompt with student logits from the gist-token prompt using KL divergence; at inference, the prompt is replaced with the gist-token string, requiring no custom serving path. In load tests at 350 requests per minute, compression reduced median time to first token from 438ms to 354ms and end-to-end latency from 6.8s to 4.2s, while raising throughput from 20.2 to 23.4 QPS and enabling 14% fewer GPUs in production without losing prediction quality.
Modern artificial intelligence agents increasingly rely on search infrastructure to run complex, neuro-symbolic reasoning workflows that compile into deeply nested, non-monotonic Boolean queries. Standard inverted index evaluation methods struggle with these query structures, facing either exponential query complexity under Document-at-a-Time models or universe-scale space penalties under Term-at-a-Time models during logical negation. To address these theoretical limitations, the author formalizes a DAG-based retrieval language named L_R and proves its query evaluation problem is strictly P-Complete. The paper introduces ComputePN, a deterministic and sparsity-aware algorithm that employs a Positive-Negative dual representation alongside native DAG memoization. This mechanism bounds evaluation time to O(|Q| · |U_active|), effectively evaluating P-Complete queries natively over the index while bypassing combinatorial tree expansion and universal scan overhead.
Databricks introduces Precision Mode for its ai_extract document extraction API, targeting enterprise workloads that challenge existing LLM and rules-based systems. The mode addresses long documents with cross-references, large nested outputs such as thousands of line items, and schemas requiring reasoning across financial statements or applying discounts. It combines custom-trained, efficient extraction models with an agent harness that decomposes jobs semantically, runs smaller tasks in parallel, preserves intermediate results, and reconciles them into a final structured output. On benchmarks covering roughly 9,000 documents, including files up to 2,000 pages and schemas with more than 300 deeply nested fields, Precision Mode reached 94.7% accuracy, seven points above the strongest frontier-model chunk-and-merge baseline, GPT-5.6 Sol; the source says it is now available through ai_extract and the Information Extraction UI.
The post examines how much agentic memory different models can use effectively, evaluating ALTK-Evolve's self-distilled guidelines across eight models on AppWorld's 585 multi-step tasks. Memory is produced by extracting and consolidating lessons from successful and unsuccessful trajectories, then delivering either the full guideline set on every ReAct step or a fixed core plus task-relevant retrieval, without updating model weights. Results identify three patterns: strong models with headroom benefited from the full set, gpt-oss-120b gained 16.1 percentage points in task completion from curated retrieval, and GLM-5 showed no measurable gain; stricter scenario completion sometimes improved more. Curated retrieval gave gpt-oss-120b that gain with only 5% more tokens versus 51% for its full set, while prompt caching is presented as a way to reduce recurring input cost, supporting calibration to model capability rather than simply accumulating memory.
Atomistic simulation combines scientific judgment, compute-efficient implementation, and an accessible interface to the simulation stack, but the MLIP ecosystem lacks the mature interfaces available for classical force fields. NVIDIA ALCHEMI Toolkit addresses implementation with composable, PyTorch-native, GPU-accelerated building blocks and in-flight batching, while its agent skills and reference files supply API patterns to coding agents. The post benchmarks 45 generated pipelines across silicon equation of state, copper adsorption, and lithium self-diffusion, checking code features and running representative workloads on NVIDIA H200 GPUs. All 45 scripts used batched GPU execution, 38 completed screening, and all 15 production representatives completed; however, agents did not assess physical validity, and thermostat choices could suppress measured lithium diffusion by 3–5×, so independent validation against DFT, experiment, or another reference remains necessary.
Dropbox manages growing infrastructure demands from AI and standard workloads by maximizing the efficiency of its existing data centers rather than just building more capacity. Through a system-level approach spanning software, hardware, and physical facility design, the company dynamically manages power and balances workloads. These optimizations, combined with higher storage density and data-driven hardware lifecycle management, have significantly reduced energy use per unit of storage.
UMAP workflows on tens to hundreds of millions of vectors are constrained by the cost of constructing an all-neighbors k-nearest-neighbor graph, especially when repeated runs support exploration and parameter tuning. The NVIDIA cuML and cuVS 25.06 feature distributes this graph-construction stage across multiple GPUs while retaining the out-of-core design that partitions data into balanced clusters, overlaps vectors across nearby clusters, computes local graphs, and merges them into a global graph. knn_n_clusters and knn_overlap_factor expose the memory, runtime, and graph-quality trade-off: more clusters reduce per-GPU data, while more overlap can improve nearest-neighbor recall at higher computational and memory cost. On eight NVIDIA H100 GPUs, the reported evaluation found up to 74x end-to-end speedup over a projected CPU runtime for the 106M-vector MIRACL dataset, completing it in 8 minutes. The post states that performance continues to improve from one to eight GPUs while preserving comparable embedding quality across configurations.
Sony LIV’s legacy analytics environment combined batch pipelines, Elasticsearch, BigQuery, and other telemetry stores, creating slow, costly, fragmented analysis as streaming activity reached billions of events per day. During live sports, the platform needed sub-second queries, high ingestion throughput, cost-efficient long-term storage, and horizontal scaling for sudden traffic spikes. Sony LIV consolidated these workloads on ClickHouse Cloud: client, streaming, CDN, and advertising systems send events through Amazon Kinesis and ClickPipes into a ClickHouse cluster serving dashboards, product analytics, QoE, and SRE workflows. The source reports that queries previously taking tens of seconds to minutes now run in sub-second to low-single-digit seconds, improving operational responsiveness, engineering productivity, and the handling of major live-event traffic.
The best engineering reads, once a week.
