Loading…
Topic
812 posts about Performance. Every summary links to the original.
See the idea
Bare metal gives engineers direct control over GPU, network, and virtualization layers, but it shifts elasticity and lifecycle work onto the team. This trade-off map shows when that bargain pays off.
Bun packs a runtime, package manager, bundler, and test runner into one binary. Here is how those four parts fit together and where the seams still show.
A five-step how-to for deciding when an AMD Instinct GPU should stay dedicated and when SR-IOV virtual functions, VMs, or sidecar isolation can safely share it.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
On June 24, video transcoding infrastructure reached maximum capacity, causing hours of publishing delays for Spotify video podcast episodes. Four converging factors drove the disruption: insufficient headroom for content spikes, an active routine batch processing job, higher compute costs from a recent video quality upgrade, and a scheduling bug reducing hardware throughput by roughly 10%. Creators exacerbated the backlog by re-uploading unconfirmed episodes, while four hours elapsed between the earliest alerts and formal incident response. Engineers halted the batch job, patched the scheduling defect, and brought an extra compute cluster online to clear all backlogs overnight. Spotify subsequently increased transcoding capacity by approximately 67% and initiated cross-team investments in workload prioritization, rate limiting, and earlier capacity alerting.
Scale-up networking has emerged as a critical architectural component for AI factories supporting trillion-parameter models, mixture-of-experts architectures, and disaggregated serving. NVIDIA presents its sixth-generation NVLink scale-up fabric and NVLink 6 Switch to address latency-sensitive all-to-all GPU communication and offload collective operations. In simulation benchmarks on models like DeepSeek-R1, Qwen 235B, and a simulated 2T parameter model, NVLink achieved up to 2.3X higher decode throughput than off-the-shelf Ethernet in a 72-accelerator domain. The architecture integrates NVLink-C2C to supply 1.8 TB/s of coherent CPU-GPU bandwidth in the Vera Rubin NVL72 platform, as well as NVLink Fusion for integrating semi-custom silicon XPUs. Through co-designed hardware, SHARP in-network compute, and NCCL libraries, the platform optimizes token throughput, factory uptime, and deployment economics.
Measuring MongoDB performance in a .NET application requires command details, durations, and enough context to distinguish isolated spikes from recurring query patterns. The post presents an end-to-end setup that uses MongoDB’s built-in OpenTelemetry Activity instrumentation, the .NET OpenTelemetry SDK, Sentry.OpenTelemetry.Exporter, and OTLP ingestion rather than a dedicated Sentry.MongoDB integration. Configuration initializes Sentry with UseOtlp(), subscribes to MongoTelemetry.ActivitySourceName, adds MongoDB instrumentation, and exports spans with AddSentryOtlp(...), while avoiding duplicate spans from overlapping hooks. Query context appears in Sentry traces, span samples, and the Queries experience, with Relay generating parameterized values for aggregation but retaining raw db.query.text per span. The post also demonstrates client-side redaction and describes server-side scrubbing, while exporter propagation bridges preserve correlation with errors and replays.
Netflix established an in-house serving platform to run large language model inference directly inside existing production environments alongside traditional machine learning models. Built upon NVIDIA Triton Inference Server and vLLM, the unified architecture handles member-scale routing, candidate generation, feature fetching, and model execution over gRPC and OpenAI-compatible HTTP endpoints. The team shifted their primary engine from TensorRT-LLM to vLLM to support custom architectures, simpler debugging, and non-trivial constraint logic. Addressing production obstacles required patching Triton's frontend for guided decoding, pinning dependent library versions, rewriting logits processors in C++ for vLLM V1, and handling state machine resets during engine preemptions. The resulting platform unifies deployment flows while preserving operational stability across evolving model schemas.
Deploying Eclipse Dataspace Components (EDC) connectors on AWS presents challenges in forecasting infrastructure expenses without clear benchmarks. Estimated costs differ substantially between business-critical setups and non-critical environments based on compute and database sizing. A business-critical configuration relying on Amazon Aurora PostgreSQL db.r6g.large and standard Amazon ECS with AWS Fargate totals an estimated 387.00 USD monthly in the eu-south-2 region. Sizing down to db.t4g.medium and switching to AWS Fargate Spot lowers the non-critical monthly expense to 164.00 USD, representing a 58% reduction. Core cost drivers remain baseline infrastructure components, while storage, API calls, and data transfers contribute marginally under baseline assumptions.
Capcom integrated path tracing into RE ENGINE across PRAGMATA and Resident Evil Requiem to deliver unified indirect and direct lighting. Over two years, the team built a reference path tracer validated against standard DCC tools, scaling it down into a real-time version optimized for NVIDIA DLSS Ray Reconstruction denoising. To manage rendering performance, Capcom avoided expensive AlphaTest texture lookups by creating a ScreenSpaceAlphaTest method comparing ray-hit positions against screen-space depth. Sampling noise in emissive and indoor regions is mitigated using Next Event Estimation and a custom implementation of ReSTIR GI. Replacing traditional shadow maps with continuous path tracing reduced the visual gap between gameplay and cinematics while enabling real-time light transmission through strand hair.
At checkout, fraud scoring must combine transaction inference with customer-specific rules while keeping latency low enough for an interactive payment. The retail-app sample pairs a FastAPI backend and React frontend in a Databricks App with Model Serving route optimization and Lakebase Postgres, where the model retrieves historical features and the backend reads profile controls. A transaction is scored first, then checked against daily spending, international-transaction, and country rules; pooled OAuth-authenticated connections and token-rotation handling avoid repeated handshakes and stale credentials. In the supplied benchmark, route-optimized calls reached 27.2 ms p50 and 37.3 ms p95 end-to-end, with feature lookup at 8.9 ms p50, CatBoost inference at 0.4 ms, and network overhead at 17.4 ms p50; actual performance varies by model.
Agentic artificial intelligence workflows transform data center infrastructure by turning context management, security enforcement, and inter-service communication into active components of the inference data path. To prevent host central processing units and graphics processing units from stalling on these tasks, NVIDIA introduces BlueField-4 data processing units, Vera BlueField-4 STX storage processors, and the DOCA software platform. The BlueField-4 processor offloads Layer 3 routing, encryption, and telemetry using a sixty-four-core Grace processor with up to eight hundred gigabits per second connectivity. Simultaneously, the Vera BlueField-4 STX powers the NVIDIA CMX context memory platform alongside DOCA Memos to store, protect, and share key-value cache across flash storage tiers. This co-designed infrastructure isolates tenant traffic and accelerates context reuse without degrading host-side agent execution.
Unity Catalog external access to Unity Catalog managed Delta tables is now in Public Preview, allowing external engines to create, read, and write while governance remains centralized. The announcement addresses the previous multi-engine trade-off: external tables enabled access but lacked managed-table performance optimizations and governance guarantees. Catalog commits coordinate writes through Unity Catalog, making it the source of truth for table state and enabling safe external writes, multi-statement transactions, and auditing of external operations. Predictive Optimization cleans storage, collects query statistics, and selects Liquid clustering columns as query patterns change; the post says these capabilities can deliver up to 50% storage cost savings and 20x faster queries. Support includes Spark, Flink, Starburst, DuckDB, and StreamNative, with open APIs and Delta Kernel extending integrations across Databricks UC and UC OSS.
DharmaOCR, a Brazilian Portuguese-focused OCR model, is evaluated against newer Mistral OCR4 and Unlimited-OCR systems on a Portuguese-only benchmark. The article attributes its performance to a two-stage training pipeline: supervised fine-tuning on varied Portuguese documents, followed by Direct Preference Optimization (DPO) using comparative output preferences. Fine-tuning concentrates model capacity on Brazilian Portuguese vocabulary, syntax, and document structures, while DPO targets repetitive or incoherent generation, reducing inference time and cost and improving stability according to the article. DharmaOCR scored 0.925, compared with 0.798 for Mistral OCR4 and 0.7587 for Unlimited-OCR, while examples from handwritten ENEM essays show the competitors misreading Brazilian names and phrases that DharmaOCR handled correctly. The conclusion is that newer architectures can raise absolute performance, but directing finite resources at one domain can preserve a measurable specialist advantage.
Modal rebuilt its sandbox platform to support millions of concurrent sandboxes and tens of thousands of creations per second, removing all central bottlenecks from the control plane. The new architecture eliminates central coordination and durable datastores from the critical path, using horizontally scaled scheduling servers that run a fast load-balancing algorithm against in-memory cached worker state from Redis streams. Workers are contacted directly via RPC to create sandboxes, and sandbox metadata is written asynchronously, avoiding O(sandboxes) and O(nodes) operations that limited previous systems. In a benchmark, Modal created 1 million sandboxes in under a minute with median time-to-interactivity under half a second, though the long tail is attributed to kernel and network contention like the rtnl lock. The system is designed to scale without practical limits, supporting use cases like reinforcement learning and background agents that require massive concurrent sandbox creation.
Checkout Blocks upgraded all five UI extensions from React and Shopify’s legacy Remote UI bridge to remote-dom with Preact and Polaris web components, while moving from JavaScript to TypeScript and API version 2026-01. The migration addressed checkout’s performance sensitivity and Shopify’s requirement that deployments move off versions earlier than 2026-01. A parallel core-next library allowed extensions to transition incrementally, with shared providers, telemetry, hooks, helpers, and rules adapted before legacy core was removed. To meet the remote-dom CLI’s 64KB gzip limit, the team dropped react-reconciler, replaced liquidjs with a smaller parser validated against more than 42,000 lines of merchant configurations, replaced dayjs, and retained markdown-to-jsx through Preact compatibility. Transferred bundles fell 40% to 85%, while weighted ELT improved 8% at P50 and 7% at P90.
The July 2026 newsletter rounds up ClickHouse releases, community projects, benchmarks, engineering work, and upcoming events in real-time data warehousing. Version 26.6 adds hypothetical skip indexes, cascading refreshable materialized-view dependencies, experimental continuous queries, AI embedding functions, PNG output, CLI documentation, and geospatial improvements. A CostBench comparison running the full analytics path over 28 hours found better cost-performance for ClickHouse Cloud than Snowflake Interactive Tables, which required a continuously running refresh warehouse for pre-aggregated data. Other features cover conversational analytics with ClickHouse Cloud, LibreChat, and MCP; Postgres change-data capture through Debezium and Kafka; local agent queries with chDB; high-concurrency tuning; and Silk, whose fiber runtime targets lower I/O tail latency. Quick reads report ingestion, caching, security, and multi-tenant SQL developments.
Stripe describes how its Document Database fleet previously handled degraded MongoDB shards with single-step, hard-coded plugins that struggled with combined failures, intermediate states, and more than 40 shard layouts. The replacement models shard topology and health as a directed graph, with infrastructure operations as edges and safety invariants defining valid states. A simulator uses breadth-first search to find minimum-operation recovery paths, while an abstraction over Temporal contexts lets the same remediation logic run in memory without real infrastructure calls; later, the system also uses Dijkstra's algorithm. The system reduced pager volume by 30%—about 200 pages annually—reclaimed 12 days of fleet health each year, and handled a new layout with zero code changes, according to the post.
Trainy’s Pluto is an open-source experiment tracker that separates analytics metrics in ClickHouse Cloud from relational run metadata in Postgres. Its original metadata store, Amazon RDS Postgres, was too expensive for the volume observed and contributed to slower initial page loads, so Trainy migrated roughly 40–50 GB to ClickHouse Managed Postgres, which uses local NVMe storage and keeps both databases in one account. After migration, initial page loads became 2–3 times faster, while a complex query reported as taking 10–15 minutes previously returned in five seconds; visualization queries over runs with hundreds of thousands of steps typically return in 50–100 milliseconds at the P50. Trainy reports 100% reliable ingestion with zero dropped data points and says the unified stack could support future joins between GPU metrics and run metadata, as well as governed agent access.
Historical market-data systems separate a durable archive from the real-time trading path, but the archive still must support research, backtesting, transaction-cost analysis, and regulatory reconstruction. The worked example uses Binance’s public archive to load trades and quotes for five USD-M perpetual-futures symbols into ClickHouse, with a MergeTree table ordered by (symbol, ts) and codecs including DoubleDelta and ZSTD. SQL queries calculate VWAP, hourly OHLC candles, and as-of joins, while ClickHouse’s sparse primary index skips most granules for symbol-and-time filters. Across ten months, 16.2 billion rows compress to 76 GiB on disk; the post reports a ClickHouse Cloud cost of $1.88 per month and argues that typical symbol-and-time queries retain low latency at that scale, while broader scans read more data.
NVIDIA CUDA 13.3 introduces clmad, a native PTX carryless multiply-accumulate instruction for Ampere and newer GPU architectures. Prior to this release, GPUs lacked dedicated hardware for carryless multiplication, requiring developers to emulate binary extension field arithmetic with ALU operations or bitsliced circuits. The new primitive performs 64-bit by 64-bit carryless multiplication into 128-bit results with accumulation, allowing full 128-bit field multiplications via Karatsuba in six instructions. Benchmarks demonstrate that clmad achieves up to 6.3 TB/s throughput for GHASH on the NVIDIA B200, representing an 18.8x improvement over bitslicing. Additionally, sum-check protocol latency in zero-knowledge proving workloads decreases by up to 13x on the B200 and 4x on the RTX 5090.
Model routing in agentic systems is presented as a systems optimization problem rather than a simple classification task. On 417 AppWorld Test Challenge tasks with the same CodeAct agent, Claude Sonnet 4.6 cost $79 total ($0.19 per task), versus $155 ($0.37 per task) for GPT-4.1, despite GPT-4.1 having lower listed token prices; cache-read pricing and repeated context changed the effective result. The discussion also shows why task difficulty and model speed are incomplete signals: hidden execution work, specialization, governance constraints, routing overhead, cache state, hardware, and endpoint load all affect decisions. The described router optimizes cost, quality, and latency together, producing configurable cost-accuracy operating points; one latency-oriented configuration reached 84% accuracy for $93 and 83 seconds, while adding about 6 ms and 2 kB per task.
Thinking Machines Lab has released Inkling and the smaller Inkling-Small family as open multimodal models that accept image, text, and audio inputs. Inkling is described as a roughly 1T-parameter decoder-only Mixture-of-Experts model, with 975B total and 41B active parameters, a 1M-token context window, and training on 45 trillion multimodal tokens. Its architecture combines relative attention, alternating global and sliding-window attention, short convolution, shared experts, hierarchical image patchification, and discretized mel-spectrogram audio processing. The release includes BF16 and NVFP4 checkpoints, speculative MTP layers, day-zero support in Transformers, SGLang, vLLM, and llama.cpp, plus deployment configurations and interactive voice and image demos. Reported benchmarks show Inkling and Inkling-Small achieving strong but varied results across reasoning, coding, vision, audio, factuality, chat, and safety evaluations.
Existing voice-AI benchmarks increasingly suggest near-human performance, while real-world conversations still expose failures involving speaker consistency, hesitation, accents, noise, emotion, and natural responses. Real World VoiceEQ measures human quality in voice interaction by testing whether systems recognize, produce, and respond to acoustic information omitted by transcripts, including tone, emotion, speaker identity, and background context. It evaluates more than 40 proprietary and open-source models across 15+ dimensions and 60+ metrics covering ASR, TTS, S2S, and Speech Understanding, using more than 1 million human ratings, including 785,000 TTS and 48,000 STS ratings. Findings indicate that capabilities are specialized, speech-to-speech systems often underuse paralinguistic cues, and traditional benchmarks can overestimate performance in noise and other conversational conditions. The research also reports that human agreement remains essential for subjective acoustic and social judgments because SLM evaluators were less reliable outside clear, verifiable tasks.
The best engineering reads, once a week.
