---
title: "LLMs"
description: "673 posts about LLMs, summarised, each linking to the original."
---

# LLMs
> 673 posts about LLMs, summarised, each linking to the original.

## Articles

### [July 2026 newsletter](https://yomu.fyi/post/july-2026-newsletter.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: Jul 16, 2026
- Type: Announcement

The July 2026 newsletter rounds up ClickHouse releases, community projects, benchmarks, engineering work, and upcoming events in real-time data warehousing. Version 26.6 adds hypothetical skip indexes, cascading refreshable materialized-view dependencies, experimental continuous queries, AI embedding functions, PNG output, CLI documentation, and geospatial improvements. A CostBench comparison running the full analytics path over 28 hours found better cost-performance for ClickHouse Cloud than Snowflake Interactive Tables, which required a continuously running refresh warehouse for pre-aggregated data. Other features cover conversational analytics with ClickHouse Cloud, LibreChat, and MCP; Postgres change-data capture through Debezium and Kafka; local agent queries with chDB; high-concurrency tuning; and Silk, whose fiber runtime targets lower I/O tail latency. Quick reads report ingestion, caching, security, and multi-tenant SQL developments.


### [Develop Lightweight USD Runtimes Faster with AI Agents](https://yomu.fyi/post/develop-lightweight-usd-runtimes-faster-with-ai-agents.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Jul 15, 2026
- Type: Explainer

Building custom Universal Scene Description (USD) implementations traditionally required adapting large existing codebases to meet specific memory footprints, application binary interfaces (ABIs), or performance requirements. The nanousd-labs project introduces an alternative approach by using AI agents to generate lightweight USD runtimes directly from the Alliance for OpenUSD machine-readable USD Core Specification. Agents parse the specification section by section, write conforming code, and validate output against specification-derived tests under engineer guidance. Written in C++ with a public C API, nanousd functions as an independent data layer that handles parsing, composition, queries, and writing without rendering pixels. This methodology enables developers to regenerate runtimes for varying deployment constraints while maintaining standard compliance through reusable workflows and skill graphs.


### [Inkling model from Thinking Machines Lab now on Databricks](https://yomu.fyi/post/inkling-model-from-thinking-machines-lab-now-on-databricks.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Mike Eastham, Yuchen Jin, Preslav Le
- Published: Jul 15, 2026
- Type: Announcement

Databricks announces that Inkling, Thinking Machines Lab’s first open-weights model, is available to enterprise customers through the Unity AI Gateway. The model is positioned for coding and agentic reasoning workflows, supports multi-modal inputs, and can be applied to enterprise data, including proprietary codebases, internal documentation, and domain-specific data. Unity AI Gateway provides centralized security, permissions, audit logging, policy enforcement, cost controls, budgets, and observability, while data remains within the governed environment; Inkling is invoked through a REST API, with SQL query support planned. Teams can try it in AI Playground, deploy a governed endpoint, connect coding agents such as Cursor, OpenCode, or Pi, and build agents with Agent Bricks, with open weights enabling customization and inference-cost optimization without per-token API pricing.


### [Model Routing Is Simple. Until It Isn’t.](https://yomu.fyi/post/model-routing-is-simple-until-it-isn-t.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Yara Rizk, Eyal Shnarch, Jason Tsay, Merve Unuvar
- Published: Jul 15, 2026
- Type: Explainer

Model routing in agentic systems is presented as a systems optimization problem rather than a simple classification task. On 417 AppWorld Test Challenge tasks with the same CodeAct agent, Claude Sonnet 4.6 cost $79 total ($0.19 per task), versus $155 ($0.37 per task) for GPT-4.1, despite GPT-4.1 having lower listed token prices; cache-read pricing and repeated context changed the effective result. The discussion also shows why task difficulty and model speed are incomplete signals: hidden execution work, specialization, governance constraints, routing overhead, cache state, hardware, and endpoint load all affect decisions. The described router optimizes cost, quality, and latency together, producing configurable cost-accuracy operating points; one latency-oriented configuration reached 84% accuracy for $93 and 83 seconds, while adding about 6 ms and 2 kB per task.


### [Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization](https://yomu.fyi/post/exploring-hierarchical-interest-representation-for-meta-ads-deep-funne.md)
- Company: [Meta](https://yomu.fyi/company/meta.md)
- Author: By Yuhui Ouyang, Di Wang, Sreedal Menon, Jie Tian
- Published: Jul 15, 2026
- Type: Explainer

Hierarchical Interest Representation is an upstream representation layer for Meta Ads that addresses sparse deep-funnel feedback by connecting users’ inferred interests with advertisers, products, services, and other ads entities. It models a typed, heterogeneous engagement graph with a transformer architecture, sparse attention for long-range relationships, bias-aware attention, self-supervised cross-view distillation, and multimodal content features processed through vision or language models. The system projects the raw graph into learned interest primitives and cascading super-graphs, producing universal embeddings and Bag-of-Meaning tokens at multiple granularities. Temporal cutoffs and chronological splits prevent future-information leakage, while a pipelined online graph engine reportedly achieved a 30x wall-clock speedup over a synchronous baseline during testing. Trained end-to-end on billions of interactions, the representations are designed for personalization, retrieval, supervision, and ranking integrations including GEM, Andromeda, and the Adaptive Ranking Model.


### [Welcome Inkling by Thinking Machines](https://yomu.fyi/post/welcome-inkling-by-thinking-machines.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: ben burtenshaw, merve, Pedro Cuenca, Aritra Roy Gosthipaty, Andres Marafioti
- Published: Jul 15, 2026
- Type: Announcement

Thinking Machines Lab has released Inkling and the smaller Inkling-Small family as open multimodal models that accept image, text, and audio inputs. Inkling is described as a roughly 1T-parameter decoder-only Mixture-of-Experts model, with 975B total and 41B active parameters, a 1M-token context window, and training on 45 trillion multimodal tokens. Its architecture combines relative attention, alternating global and sliding-window attention, short convolution, shared experts, hierarchical image patchification, and discretized mel-spectrogram audio processing. The release includes BF16 and NVFP4 checkpoints, speculative MTP layers, day-zero support in Transformers, SGLang, vLLM, and llama.cpp, plus deployment configurations and interactive voice and image demos. Reported benchmarks show Inkling and Inkling-Small achieving strong but varied results across reasoning, coding, vision, audio, factuality, chat, and safety evaluations.


### [Introducing Real World VoiceEQ: Measuring the human quality of voice AI](https://yomu.fyi/post/introducing-real-world-voiceeq-measuring-the-human-quality-of-voice-ai.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: David Ayllon, Alice, Jeff Brooks, Franc Camps Febrer, Jakub Piotr Cłapa, Theo Lebryk, Jens Madsen, Olya Ossipova, Sharath Rao, Hoon Shin, Tigran, Rashish Tandon, Panagiotis Tzirakis
- Published: Jul 15, 2026
- Type: Announcement

Existing voice-AI benchmarks increasingly suggest near-human performance, while real-world conversations still expose failures involving speaker consistency, hesitation, accents, noise, emotion, and natural responses. Real World VoiceEQ measures human quality in voice interaction by testing whether systems recognize, produce, and respond to acoustic information omitted by transcripts, including tone, emotion, speaker identity, and background context. It evaluates more than 40 proprietary and open-source models across 15+ dimensions and 60+ metrics covering ASR, TTS, S2S, and Speech Understanding, using more than 1 million human ratings, including 785,000 TTS and 48,000 STS ratings. Findings indicate that capabilities are specialized, speech-to-speech systems often underuse paralinguistic cues, and traditional benchmarks can overestimate performance in noise and other conversational conditions. The research also reports that human agreement remains essential for subjective acoustic and social judgments because SLM evaluators were less reliable outside clear, verifiable tasks.


### [Lessons From the Leaderboard: What 5,000+ Kagglers Taught Us About Improving AI Reasoning](https://yomu.fyi/post/lessons-from-the-leaderboard-what-5-000-kagglers-taught-us-about-impro.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Jul 14, 2026
- Type: Explainer

The NVIDIA Nemotron Model Reasoning Challenge on Kaggle tasked over 5,000 participants with improving reasoning accuracy on Nemotron-3-Nano-30B using restricted LoRA adapters on Google Cloud G4 VMs. Top-performing competitors approached reasoning as an end-to-end engineering workflow by auditing synthetic chain-of-thought traces with programmatic solvers before training. Successful teams also compressed reasoning representations into compact signatures and bit-manipulation formats to prevent long chains from exceeding runtime token generation limits. Additionally, leading approaches separated precomputed reusable structures from live computation and conducted granular per-category validation to monitor regression and non-determinism. These results show that reasoning improvements stem from verifiable training data, compact context encoding, and structured validation rather than unconstrained data scaling.


### [From weeks to a day: how we made LLM evaluation fast enough to iterate on](https://yomu.fyi/post/from-weeks-to-a-day-how-we-made-llm-evaluation-fast-enough-to-iterate.md)
- Company: [Airbnb](https://yomu.fyi/company/airbnb.md)
- Author: Baharak Saberidokht
- Published: Jul 14, 2026

Airbnb built a four-layer LLM infrastructure framework to reduce iteration and evaluation turnaround from weeks to a single day. By using per-sample caching for generated references and judge scores, they established a deterministic evaluation foundation that separates genuine model drift from measurement noise. This setup enables rapid, bounded model hotfixes using micro LoRA adapters and comprehensive end-to-end validation across component boundaries.


### [How to Run an Autoresearch Workflow with RL Agent Skills and NVIDIA NeMo](https://yomu.fyi/post/how-to-run-an-autoresearch-workflow-with-rl-agent-skills-and-nvidia-ne.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Jul 14, 2026
- Type: Tutorial

Setting up reinforcement learning workflows and running repetitive experimentation often consumes significant engineering time before meaningful metrics emerge. To address this, an autonomous autoresearch workflow uses a frontier coding agent running Codex with GPT 5.5 to manage NVIDIA NeMo RL and NeMo Gym on an NVIDIA Brev GPU instance. The agent executes full-stack setup, creates a novel NeMo Gym visual counting environment, tunes Qwen3-VL-2B-Instruct accuracy from 25.0% to 96.9%, and implements an off-policy reinforcement learning algorithm directly from a research paper. Reusable skills, including brev-etiquette, session-memory, and autoresearch, enforce local conventions and prevent context drift during long executions. This setup offloads environment configuration, dependency resolution, and experiment tracking while preserving human oversight for goals and strategy.


### [Post-Train NVIDIA Cosmos 3 in One Day Using Agent Skills](https://yomu.fyi/post/post-train-nvidia-cosmos-3-in-one-day-using-agent-skills.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Jul 14, 2026
- Type: Tutorial

Adapting vision reasoning models for specialized physical AI video tasks often demands days of manual container configuration, training script authoring, and hyperparameter tuning. NVIDIA Cosmos 3 Nano, an omnimodal mixture-of-transformers model, pairs an autoregressive Reasoner tower with an iterative diffusion Generator tower for video question answering. By integrating NVIDIA TAO agent skills with a coding agent like Codex, developers can automate data patching, evaluation, and Low-Rank Adaptation (LoRA) post-training. In experiments using the Woven Traffic Safety dataset, LoRA fine-tuning boosted zero-shot baseline accuracy from 54.41% to 87.14% while using approximately seven times fewer GPU hours than full-parameter supervised fine-tuning. Combining this agent workflow with TAO AutoML further elevated model accuracy to 93.35% before automated deployment via NVIDIA NIM.


### [Reducing High-Bandwidth Memory Bottlenecks in JAX-Based LLM Training with Host Offloading](https://yomu.fyi/post/reducing-high-bandwidth-memory-bottlenecks-in-jax-based-llm-training-w.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Tanya Lenz
- Published: Jul 10, 2026
- Type: Benchmark

Large language model training workloads frequently face GPU high-bandwidth memory (HBM) capacity constraints before compute limits are reached. To alleviate this memory pressure in JAX training workloads, host offloading transfers selected forward-pass activations to pinned host memory over high-speed NVLink-C2C interconnects and reloads them during the backward pass instead of recomputing them. Benchmarks executed on 128 GPUs within NVIDIA GB200 NVL72 systems using MaxText demonstrated significant improvements. For DeepSeek-V3 671B, host offloading combined with the Latency Hiding Scheduler and pipelined offloading reached 908.2 TFLOPs/s/device, surpassing activation rematerialization throughput by 57% and unlocking larger batch configurations. Similarly, QKV activation offloading on Llama 3.1 405B achieved a 2.9% throughput increase when transfer latency was hidden behind computation.


### [AI Model Co-Design: Hardware-Friendly LLM Design](https://yomu.fyi/post/ai-model-co-design-hardware-friendly-llm-design.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Jul 10, 2026
- Type: Explainer

Efficient large language model serving requires co-designing architectures to balance datacenter token throughput and user interactivity within hardware constraints. Roofline analysis shows that linear layers become memory-bound when hidden or intermediate projection dimensions are small, leaving arithmetic hardware underutilized even at large token counts. For prefill workloads, Chunked Pipeline Parallelism reduces first-token latency on long contexts when models use regular, repeatable layer patterns that split into balanced stages. For latency-critical decoding, decoupling attention and feed-forward network parallelization enables independent optimization, applying expert parallelism for feed-forward networks and Helix parallelism to shard the key-value cache across sequences. Following architectural design rules—such as dimension alignment, width prioritization, and low-precision NVFP4 compatibility—allows models to achieve higher utilization on modern accelerator hardware.


### [Better tools made Copilot code review worse. Here’s how we actually improved it.](https://yomu.fyi/post/better-tools-made-copilot-code-review-worse-here-s-how-we-actually-imp.md)
- Company: [Github](https://yomu.fyi/company/github.md)
- Author: Napalys Klicius
- Published: Jul 10, 2026
- Type: Problem & solution

When migrating GitHub Copilot code review from custom navigation tools to shared Unix-style CLI utilities—grep, glob, and view—benchmarks revealed higher review costs and fewer caught issues. Rather than an issue with the underlying tools, trace analysis showed the agent used general-purpose coding assistant instructions that triggered expansive repository browsing loops. In response, the team rewrote the tool guidance to enforce a review-specific workflow anchored to pull request diffs. The updated instructions direct the agent to narrow candidate call sites and files using batched discovery before reading minimal line ranges with view. This workflow tuning reduced average review costs by approximately 20% in production while maintaining review quality.


### [Synthetic Data Generation for Financial AI Research with NVIDIA NeMo](https://yomu.fyi/post/synthetic-data-generation-for-financial-ai-research-with-nvidia-nemo.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Elizabeth Goodman
- Published: Jul 9, 2026
- Type: Problem & solution

Real-world financial natural language processing datasets often overrepresent common events like earnings while underrepresenting rarer occurrences such as credit-rating changes. To address this imbalance, an iterative generation pipeline produced over 500,000 unique headlines across 13 categories using NVIDIA NeMo Data Designer, NeMo Curator, and Nemotron 3 Nano. Instead of relying on a single large batch that yielded 65 percent near-duplicates, the workflow executes iterative cycles of category-weighted generation, rule-based filtering, global semantic deduplication, and dynamic category reweighting. Farthest-from-centroid few-shot example selection guided subsequent rounds toward novel semantic spaces across 82 total iterations. Executed over approximately six days on an eight-way NVIDIA B200 node, the resulting corpus enabled fine-tuning compact student language models to achieve 95 percent of teacher model performance.


### [Capturing token IDs during agentic interactions for better reinforcement learning](https://yomu.fyi/post/capturing-token-ids-during-agentic-interactions-for-better-reinforceme.md)
- Company: amazon.science
- Author: Frederick Robinson
- Published: Jul 9, 2026
- Type: Announcement

Reinforcement learning for agentic tasks requires updating model parameters against the precise token sequences generated during rollouts. Standard agent harnesses frequently modify conversation transcripts through message compaction and tool formatting, causing retokenization drift that degrades policy-gradient training. Turnstile addresses this issue as a Rust-based proxy situated between agent harnesses and inference backends such as SGLang. By intercepting OpenAI-compatible Chat Completions traffic, the proxy records exact token IDs, log probabilities, loss masks, weight versions, and mixture-of-experts routing traces at the moment of generation. Validation across text-based coding and multimodal computer-use agents demonstrated steady reinforcement learning improvements without requiring modifications to agent harnesses.


### [Automating cross-repo documentation with GitHub Agentic Workflows](https://yomu.fyi/post/automating-cross-repo-documentation-with-github-agentic-workflows.md)
- Company: [Github](https://yomu.fyi/company/github.md)
- Author: David Pine
- Published: Jul 8, 2026
- Type: Problem & solution

Maintaining documentation across separate repositories often leads to severe lag because technical writers must reverse-engineer shipped features weeks after release. To address this in the Aspire project, the team implemented an automated pipeline using GitHub Agentic Workflows to bridge the product and documentation repositories. When product pull requests merge, a bash step maps milestones to docs release branches before an LLM agent evaluates the diff, drafts documentation updates, and emits structured pull request intents. A dedicated safe-outputs handler materializes these drafts via a scoped GitHub App and assigns the original code reviewers to verify accuracy. Across 396 product pull requests, the system generated 82 documentation pull requests that all merged with a median turnaround time of 44.8 hours.


### [Create a LangChain Deep Agents Harness Profile for NVIDIA Nemotron 3 Ultra to Improve Performance](https://yomu.fyi/post/create-a-langchain-deep-agents-harness-profile-for-nvidia-nemotron-3-u.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Sean Lopp
- Published: Jul 8, 2026
- Type: Tutorial

The tutorial shows how to create and validate a LangChain Deep Agents harness profile for NVIDIA Nemotron 3 Ultra using an evaluation benchmark rather than fine-tuning the model. It starts with a baseline, analyzes failed trajectories, and iterates on prompts, exclusions, or added middleware while rerunning tests to detect regressions. One failure occurred when read\_file returned the first page of a large file without signaling that more content existed, so the model answered before paginating to the line containing opal-fox-91. The proposed ReadFileContinuationNoticeMiddleware appends a continuation notice when a full page is returned, raising the reported benchmark score from 94/127 to 96/127 and resolving all three read\_file tests. The source also describes automating this propose, verify, and retain-or-reject loop, with safeguards against overfitting and regressions.


### [Open, convenient and predictable: Introducing Provisioned Throughput](https://yomu.fyi/post/open-convenient-and-predictable-introducing-provisioned-throughput.md)
- Company: [Together AI](https://yomu.fyi/company/together-ai.md)
- Published: Jul 8, 2026
- Type: Announcement

Companies adopting open-weight models for production workloads have faced a trade-off between convenient but best-effort serverless inference and powerful but complex dedicated GPU instances. Together AI’s Provisioned Throughput fills that gap with reserved inference capacity sold as Provisioned Throughput Units (PTUs). Each PTU is priced at $0.05 per minute and guarantees a fixed rate of tokens per minute for a specific model, with a 99% uptime SLA. Input, cached input, and output tokens draw down capacity at different rates, letting teams optimize for their traffic shape. Available now for MiniMax M3 and GLM-5.2 with capacity in North America and EMEA and a one-month minimum term, at full utilization costs can run roughly $0.36 per million input tokens and $2.16 per million output tokens, up to 90% below proprietary alternatives.


### [NVIDIA Vera CPU Boosts AI Factory Throughput to Accelerate Agentic Workloads](https://yomu.fyi/post/nvidia-vera-cpu-boosts-ai-factory-throughput-to-accelerate-agentic-wor.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Jul 7, 2026
- Type: Announcement

Agentic AI workflows and reinforcement learning rely heavily on sequential CPU tasks, such as tool calls, sandboxed code execution, data processing, and environment rollouts between GPU steps. When CPU performance degrades under full socket load, training time inflates, serving latency grows, and GPU KV-caches face eviction. The NVIDIA Vera CPU addresses these bottlenecks by featuring 88 Olympus cores on a monolithic compute die, paired with power-efficient LPDDR5x memory and the Scalable Coherency Fabric. These cores incorporate a neural branch predictor, a 10-wide decode front end, deep out-of-order execution, and NVIDIA Spatial Multithreading. Consequently, Vera achieves 1.8x faster core execution, 40% lower peak loaded latency compared to x86 CPUs, and up to 1.2 TB/s total memory bandwidth, preventing costly KV-cache recomputations.


[Newer posts](https://yomu.fyi/topic/llm/page/15.md) · [Older posts](https://yomu.fyi/topic/llm/page/17.md)