---
title: "Performance"
description: "817 posts about Performance, summarised, each linking to the original."
---

# Performance
> 817 posts about Performance, summarised, each linking to the original.

## Articles

### [Serve Qwen3.8-2.4T-A95B, a 2.4T-Parameter Model, with Configurable Reasoning on NVIDIA GB300 NVL72](https://yomu.fyi/post/serve-qwen3-8-2-4t-a95b-a-2-4t-parameter-model-with-configurable-reaso.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Aug 12, 2026
- Type: Explainer

Alibaba’s Qwen3.8-2.4T-A95B (Qwen3.8-Max) is an open-weight 2.4T-parameter model with 95B activated per token, designed for reasoning and agentic workloads. Its fine-grained mixture-of-experts architecture combines full and linear attention, supporting context of up to one million tokens and output lengths up to 128K while using bounded recurrent state in linear-attention layers. NVIDIA describes multinode serving on the 72-GPU GB300 NVL72, where a 130 TB/s NVLink domain supports expert communication; in FP8, the model reportedly exceeds 4K tokens per second per GPU and 350 tokens per second per user without additional tuning. Developers can control reasoning depth with low, high, or xhigh modes, choose SGLang, vLLM, NVIDIA Dynamo, or NIM for serving, and use NeMo AutoModel for full SFT or LoRA post-training.


### [Introducing OlmoEarth embeddings: Custom embedding exports from OlmoEarth Studio for downstream analysis](https://yomu.fyi/post/introducing-olmoearth-embeddings-custom-embedding-exports-from-olmoear.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Kyle Wiggers
- Published: Aug 12, 2026
- Type: Announcement

OlmoEarth Studio now supports custom computation and export of embedding vectors, compact representations of Earth-observation data generated by open-source OlmoEarth foundation models. Users can select an area, a one-to-twelve-month time span, an encoder variant, spatial resolution, and Sentinel-2 L2A or Sentinel-1 RTC imagery, then receive a Cloud-Optimized GeoTIFF with one int8 band per embedding dimension. The post demonstrates similarity search, few-shot segmentation, change detection, and PCA-based exploration; a logistic regression trained on 60 labeled pixels produced a weighted F1 of 0.84 for three land-cover classes in Ca Mau, Vietnam. Embeddings provide a fast, shareable starting point, while supervised fine-tuning can improve performance for applications needing task-specific results; cloud cover, atmospheric artifacts, and missing observations can affect quality.


### [How to Choose Full-Stack Observability for NVIDIA AI Factories](https://yomu.fyi/post/how-to-choose-full-stack-observability-for-nvidia-ai-factories.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Jorge Cardoso
- Published: Aug 12, 2026
- Type: Explainer

AI infrastructure spans compute, networking, storage, orchestration, and applications, so a symptom at one layer may originate elsewhere. The post proposes selecting observability tools by failure domain, mapping components to telemetry sources, and reducing signals to a top-k alert set in a unified Prometheus/Grafana triage dashboard. For a DGX cluster using InfiniBand, BCM, and Slurm, it recommends Redfish/IPMI, DCGM, NVSM, UFM, and BCM, while deferring NetQ, NMX, Run:ai, and NIM until their corresponding capabilities are needed. The framework emphasizes DCGM for GPU metrics, NVSM for DGX system health, UFM for InfiniBand, and BCM for cluster and job aggregation, with specialized tools retaining low-level telemetry. An elevated InfiniBand bit error rate illustrates how retransmissions can slow one rank during NCCL all-reduce, causing synchronized training throughput to follow the slowest rank; alerts should connect concise signals to owners and remediation actions.


### [How iFood built its agentic security platform on ClickHouse Cloud](https://yomu.fyi/post/how-ifood-built-its-agentic-security-platform-on-clickhouse-cloud.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Aug 12, 2026
- Type: Problem & solution

iFood’s security incident response team needed high-volume log ingestion, long-term retention, and rapid access to months of history, but its Databricks-based platform became costly to scale beyond 130 TB and imposed query timeouts. The team ran a production-data proof of concept and rebuilt its tooling around ClickHouse Cloud on AWS: logs land in S3 and flow through ClickPipes into ClickHouse, with Querybook preserving notebook-based investigations and ClickStack planned for operational metrics. With ClickHouse Cloud, iFood reports 9–16x faster queries, expected cost reductions of 40–50%, and data freshness improving from hourly batches to 2–10 minutes; alerts that previously ran hourly can now run about every 10 minutes. That foundation also supports agentic threat hunting: multiple sub-agents query more than 30 TB in parallel, reducing a hunt from about a week to roughly two hours, while Langfuse is used to inspect and tune agent analyses.


### [NVIDIA JetPack 7.2.1 Adds Agentic Video Skills and T3000 Emulation](https://yomu.fyi/post/nvidia-jetpack-7-2-1-adds-agentic-video-skills-and-t3000-emulation.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Aug 11, 2026
- Type: Announcement

JetPack 7.2.1 adds support for PyNvVideoCodec 2.2, NVIDIA's Python library for hardware-accelerated video encoding and decoding on GPUs, alongside foundational agentic video skills for Jetson workflows. These skills inspect the live target, query codec capabilities, generate Video Codec SDK or PyNvVideoCodec recipes, execute runs, and return measurements, warnings, and reproducible evidence. PyNvVideoCodec keeps frames in GPU-resident device memory through DLPack and CUDA device buffers, with multi-mode frame sampling and a ThreadedDecoder that pre-decodes frames in a background thread. The release also enables emulation of recently announced Jetson T3000 performance on a Jetson T5000 module in the Jetson Thor AGX Developer Kit, while the video skills are limited to Video Codec SDK and PyNvVideoCodec stages.


### [What's new in pg\_clickhouse v0.10.0: Subqueries, TPC-H Speedups, C Driver, and Aggregates](https://yomu.fyi/post/what-s-new-in-pg-clickhouse-v0-10-0-subqueries-tpc-h-speedups-c-driver.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Aug 11, 2026
- Type: Announcement

pg\_clickhouse v0.10.0 expands pushdown for analytic workloads, with full pushdown across the TPC-H suite remaining the stated target. The release moves the scoreboard from 12 of 22 fully pushed-down queries to 16, including Q17, which falls from 32.7 seconds to 37 milliseconds when its correlated aggregate subquery is shipped to ClickHouse. SubPlan handling now turns PostgreSQL subqueries into ClickHouse subqueries, while NOT IN and related operators receive guards that preserve PostgreSQL’s three-valued NULL semantics; compatibility requires ClickHouse 25.8 or later for the correlated-subquery shape. The binary driver was rebuilt on a plain-C client library and hardened against concurrency bugs. The release also broadens pushed-down functions and aggregates, adds clickhouse\_perform() and clickhouse\_server\_version(), and leaves six TPC-H queries plus other roadmap items unresolved.


### [Thinking of ACE? We Can Do It with Fewer Tokens](https://yomu.fyi/post/thinking-of-ace-we-can-do-it-with-fewer-tokens.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Vatche Isahagian, Jayaram Radhakrishnan, Vinod Muthusamy, Gaodan Fang, Punleuk Oum, G Thomas, Ashwath Vaithinathan Aravindan, Evelyn Duesterwald, Merve Unuvar
- Published: Aug 11, 2026
- Type: Benchmark

ALTK-Evolve and ACE are agentic-memory systems that turn an LLM agent’s past trajectories into reusable lessons without weight updates or human labels. Both preserve individual lessons and attach evidence counts rather than compressing experience into a short summary, but ACE injects its full evolving playbook at every step while ALTK-Evolve consolidates typed, provenance-linked guidelines and retrieves a model-appropriate subset or the full set. On AppWorld’s 168-task test\_normal split with the same ReAct setup, ALTK-Evolve reached 89.3 TGC and 80.4 SGC on DeepSeek-V3.2 at 263K tokens per task, versus ACE’s 80.4 and 73.2 at 634K. With gpt-oss-120b, it reached 56.0 TGC and 37.5 SGC at 116K tokens, narrowly exceeding ACE’s 54.8 and 35.7 at 777K. The comparison uses single pass@1 runs and in-house ACE results on matched models, while the authors attribute the cost difference to delivery rather than lesson quality.


### [NVIDIA Nemotron 3.5 Lightning Delivers Fast, Accurate Specialized Task Execution for Long-Running Agents](https://yomu.fyi/post/nvidia-nemotron-3-5-lightning-delivers-fast-accurate-specialized-task.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Aug 11, 2026
- Type: Announcement

NVIDIA Nemotron 3.5 Lightning is an open 30B mixture-of-experts model with 3B active parameters, designed for the high-volume execution layer of long-running autonomous agents. The post positions it alongside frontier reasoning models: larger systems handle planning and orchestration, while Lightning handles tool calls, result validation, subagent delegation, and other routine work. Its design combines harness-optimized training, multi-token prediction for speculative decoding, DFlash and DSpark draft models, and NVFP4 quantization alongside BF16. The model is customizable through LoRA or full SFT, reinforcement learning, and environment-based evaluations, with weights, training data, and recipes released under OpenMDW-1.1. The post reports up to 4x the output speed of similar-sized models, 86% accuracy on PinchBench, and completion of 10,000 tasks 30% faster than Qwen3.6 35B at similar accuracy, while NeMo Switchyard routes work among models.


### [Route AI Agent Workloads Across Models with NVIDIA NeMo Switchyard](https://yomu.fyi/post/route-ai-agent-workloads-across-models-with-nvidia-nemo-switchyard.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Aug 11, 2026
- Type: Explainer

NVIDIA NeMo Switchyard addresses the problem of routing AI-agent work across models with different capabilities, costs, latency, and verbosity profiles rather than sending every request to one model. At runtime, its router evaluates a request, context, policies, and system signals, then directs work to a configured model pool, potentially routing individual agent steps and carrying session state when needed. The provider-agnostic switchyard-libsy SDK separates semantic model targets from provider endpoints, while a reference server accepts OpenAI, Anthropic, and Responses API requests and records routing decisions and outcomes. Available approaches include tuning-free LLM classifier, stage, and escalation routers, alongside tunable and custom algorithms. In a LangChain evaluation, escalation routing cut cost 74% versus a frontier-only baseline with a measured six-point accuracy trade-off, while Cognition’s staged routing reported near-frontier coding performance at approximately 28% lower mean cost.


### [bitdrift brings ClickHouse-powered mobile observability to ClickStack](https://yomu.fyi/post/bitdrift-brings-clickhouse-powered-mobile-observability-to-clickstack.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Aug 11, 2026
- Type: Announcement

bitdrift has joined ClickHouse’s House Mates program as a mobile observability partner, bringing mobile-native telemetry, tracing, and debugging to ClickStack. ClickStack provides an open-source or managed observability stack built on ClickHouse, with HyperDX and an OpenTelemetry Collector for logs, metrics, traces, and OTLP ingestion. bitdrift’s Capture SDK supports iOS, Android, and React Native, stores unsampled telemetry in a fixed-size on-device ring buffer, and uses control-plane configuration to decide what is collected or uploaded. Shared OpenTelemetry trace context lets engineers follow a mobile request’s trace ID from a session timeline into ClickStack, where backend traces, dependencies, logs, metrics, database operations, and infrastructure behavior are available, creating a continuous debugging path across the device and distributed systems.


### [AWS Trainium Frontier competition: Co-design models and kernels on purpose-built AI chips](https://yomu.fyi/post/aws-trainium-frontier-competition-co-design-models-and-kernels-on-purp.md)
- Company: amazon.science
- Author: Louise Ping, John Gray, Emily Webber, Josh Longenecker
- Published: Aug 10, 2026
- Type: Announcement

Modern language model architectures evolved around specific hardware constraints, but AWS Trainium chips alter memory and compute dynamics through on-chip SBUF memory, systolic matrix multiplication, and explicit DMA control. The AWS Trainium Frontier competition invites teams to train language models from scratch under strict wall-clock time constraints to uncover optimal architectures and custom kernels for Trainium silicon. Participants start from a fifty-million-parameter dense baseline and can modify everything from the model architecture and optimizer to custom Neuron Kernel Interface routines. Phase 1 evaluates thirty-minute single-chip training runs by validation bits-per-byte, while Phase 2 grants four-hour multi-chip server access to evaluate both training efficiency and reasoning on the CORE benchmark. Top finalists will present their findings at NeurIPS 2026, compete for cash prizes, and co-publish research with Annapurna Labs.


### [Whats new in ClickStack - June + July](https://yomu.fyi/post/whats-new-in-clickstack-june-july.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Aug 10, 2026
- Type: Announcement

ClickStack’s June–July releases add a broader set of observability features, with work centered on trace navigation, metrics sources, dashboards, and filter performance. The trace waterfall now assigns stable colors by service, keeps span durations outside bars, preserves clickable proportions for very short spans, and adds a minimap plus OpenTelemetry span-link navigation. Metrics support includes quantiles for exponential histograms, a leaner default schema, and an experimental UI connection to external Prometheus-compatible endpoints, while PromQL is proxied to the configured endpoint. Dashboard filters can narrow one another, event patterns can appear as tiles, kiosk mode provides a read-only display, and filter-value discovery prefers ClickHouse text indexes when available. Additional changes cover alert annotations, browser RUM dashboard views, external links from table tiles, grid snapping, and service-map coloring by latency, error rate, or throughput.


### [Build Low-Latency Multilingual Voice Agents: Open Weights & Full Deployment Control with NVIDIA Magpie TTS](https://yomu.fyi/post/build-low-latency-multilingual-voice-agents-open-weights-full-deployme.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Maryam Motamedi, Mikyas Desta, Jason Li, Jason Roche
- Published: Aug 10, 2026
- Type: Announcement

NVIDIA Magpie Multilingual TTS is a 364M-parameter open-weights model for multilingual voice applications, supporting 12 languages, including newly added Modern Standard Arabic, Korean, and Brazilian Portuguese. It is positioned for cascaded voice-agent architectures in which ASR, TTS, and LLM components are independently deployed, tuned, and scaled on controlled infrastructure. Frame stacking predicts two audio frames per decoding step, while a local transformer models dependencies between simultaneously generated codebook tokens; NVIDIA reports 32–79 ms single-stream TTFA across its listed GPUs. Compared with the previous release, French and Spanish CER improved to 1.54% and 0.60%, while SSIM reached 0.747 and 0.793; German SSIM also increased to 0.742. The Hugging Face checkpoint supports research and fine-tuning, while NVIDIA NIM provides optimized production serving and NeMo supports customization.


### [Run Local Agentic AI Workflows with Meta’s Muse Glimmer on NVIDIA](https://yomu.fyi/post/run-local-agentic-ai-workflows-with-meta-s-muse-glimmer-on-nvidia.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Aug 10, 2026
- Type: Announcement

Meta has released Muse Glimmer, a 30B open-weight dense model with a 120K+ context window aimed at local agentic AI workflows. The model uses a dense architecture that activates every parameter per token, avoiding routing overhead and targeting reliable instruction following, long-context coherence, and predictable latency for multi-step agent tasks. On NVIDIA Blackwell Ultra hardware, Muse Glimmer reports over 20K tokens/sec/GPU at BF16/NVF4 precision, with the full model fitting in a single GPU's VRAM. Deployment options span NVIDIA GeForce RTX 5090, DGX Spark, DGX Station, and Jetson edge devices, supported by vLLM, SGLang, and NVIDIA NIM containers. Fine-tuning is available through NVIDIA NeMo AutoModel for SFT and LoRA, with NeMo RL for reinforcement learning, and the NemoClaw harness enables long-running assistants in a sandboxed environment.


### [Serving the most critical missions: Cloudflare for Government achieves FedRAMP Class D (High) Certified status](https://yomu.fyi/post/serving-the-most-critical-missions-cloudflare-for-government-achieves.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Wesley Evans
- Published: Aug 10, 2026
- Type: Announcement

Cloudflare for Government has achieved FedRAMP Class D (High) certification status, following its FedRAMP Moderate authorization in 2022 and sponsorship by the National Institute of Standards and Technology. FedRAMP High covers systems handling the nation’s most sensitive unclassified data, including law enforcement, emergency services, financial systems, and national security information, where a control breach could be catastrophic. Cloudflare says its offering runs the same software stack and services as its commercial platform on one global network, while its Data Localization Suite ensures traffic inspection and processing for FedRAMP High occur exclusively in U.S. data centers. The systems developed for FedRAMP High are also the foundation of Cloudflare’s stated pursuit of DoD Impact Level 4 authorization, with access to its Zero Trust, application performance, and developer features as they are released.


### [Making Knowledge Distillation Cheap Enough to Run at Scale](https://yomu.fyi/post/making-knowledge-distillation-cheap-enough-to-run-at-scale.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Antonio Tiene, Iker García-Ferrero, Ali Hashemi, Bakbergen Ryskulov
- Published: Aug 10, 2026
- Type: Problem & solution

Knowledge distillation trains a smaller student to match a larger teacher's capabilities, but conventional online training keeps both models loaded and materializes full vocabulary distributions at every step, making long-context recovery expensive. The proposed system caches the teacher's top-100 logits once and introduces a fused, chunked KL-divergence loss that avoids storing the full vocabulary-by-sequence grid or the student's full logits. In benchmarks with an 8K-token context on one H200, offline fused chunked KL used 58.3GB peak memory versus 102.8GB for online training, while all four methods reached near-identical training loss. At 32K tokens, the fully chunked loss reduced isolated kernel memory from 85.2 GiB to 5.45 GiB; distilling GPT-OSS 20B at that context cut step time from 57.0 to 12.23 seconds and reduced the setup from four GPU nodes to one.


### [Meta is back with Muse Glimmer: local, agentic, multimodal, and open source](https://yomu.fyi/post/meta-is-back-with-muse-glimmer-local-agentic-multimodal-and-open-sourc.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Pedro Cuenca, merve, ben burtenshaw, Aritra Roy Gosthipaty
- Published: Aug 10, 2026
- Type: Announcement

Meta’s Muse Glimmer is a 30B-parameter multimodal model released under the Apache 2.0 license for local, agentic applications including coding, document analysis, and personal assistants. Its dense architecture combines a 2B ViT-style perception encoder with a 28B text decoder, while an optional DFlash speculative-decoding drafter targets faster generation for structured content. The decoder uses alternating sliding-window and full-attention layers, grouped-query attention, and query-key normalization with extra query scaling. Day-zero support is available in Transformers, llama.cpp, vLLM, Inference Endpoints, and other libraries, with examples for text, images, videos, deployment, and agent workflows. Published benchmarks show varied results: Muse Glimmer leads on MCP Atlas and SWE-Bench Pro, while Qwen3.6 leads on OSWorld-Verified and SWE-Bench Verified.


### [How Musinsa scaled its audience engine with ClickHouse Cloud and reduced TCO by 71.4%](https://yomu.fyi/post/how-musinsa-scaled-its-audience-engine-with-clickhouse-cloud-and-reduc.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Aug 9, 2026
- Type: Problem & solution

Musinsa’s Audience Engine powers a customer data platform that targets audiences across 16.4 million members. It stores each user-to-audience relationship as a separate row, producing roughly 1.1 billion cohort-to-user mappings across 2,500 audiences and potentially 16 million new rows when one audience is added. Its self-hosted ClickHouse deployment coupled storage and compute, forced workloads to share a cluster, and created operational burdens as data and query demands grew. Musinsa migrated to ClickHouse Cloud on AWS, using separated S3-backed storage, workload-specific compute, automatic idling, vertical autoscaling, and ClickPipes for direct real-time ingestion. The company reports an 86.5% reduction in storage costs and a total cost of ownership reduction of up to 71.4%, while shifting engineering effort toward CDP business logic.


### [How and Why Netflix Built a Real-Time Distributed Graph: Part 3 — Querying the graph with gRPC…](https://yomu.fyi/post/how-and-why-netflix-built-a-real-time-distributed-graph-part-3-queryin.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Aug 7, 2026
- Type: Problem & solution

Netflix designed a query execution serving layer for its Real-Time Distributed Graph to power sub-100ms responses across diverse graph traversal workloads spanning billions of nodes and edges. To prevent compounding network delays in multi-hop queries, the engine uses a breadth-first traversal model that batches entity lookups across entire frontier levels rather than tracing paths depth-first. The entire serving architecture relies on asynchronous composition across small thread pools of 16 to 24 threads, ensuring no thread blocks while waiting on remote storage or enrichment I/O. Selective caching via EVCache targets stable properties with volatility-matched TTLs, producing 70 to 80 percent cache hit rates and reducing backend storage calls by three to four times. A layered filtering hierarchy pushes depth and edge limits directly to the storage tier, eliminating bespoke code changes while isolating system resources against excessive fan-out.


### [Managing AI Coding Costs at Scale](https://yomu.fyi/post/managing-ai-coding-costs-at-scale.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Patrick Wendell, Akshat Bhatia, Vinay Gaba, Erich Elsen, Ivan Zhou
- Published: Aug 7, 2026
- Type: Explainer

AI coding tools have improved velocity metrics at Databricks, but broad deployment creates exponentially growing costs that can threaten the efficiency gains they deliver. The post presents a cost-management playbook aimed at preserving broad, low-friction access while keeping aggregate spending within a roughly fixed per-user envelope. It emphasizes the coding-model efficiency frontier, automated evaluations of internal workloads, model-flexible harnesses or meta-harnesses, and request- or task-level routing to the cheapest capable model. Databricks reports that Unity AI Gateway Smart Router reduced average task cost by more than 30% while roughly matching the most expensive model’s quality, and that harness and caching changes cut generated tokens and associated costs by almost 50% without observed quality degradation. The proposed AI Gateway centralizes model access, budget policies, tool configuration, and coding-session traces for ongoing optimization.


[Newer posts](https://yomu.fyi/topic/performance/page/7.md) · [Older posts](https://yomu.fyi/topic/performance/page/9.md)