---
title: "LLMs"
description: "673 posts about LLMs, summarised, each linking to the original."
---

# LLMs
> 673 posts about LLMs, summarised, each linking to the original.

## Articles

### [Closing the AI agent trust gap with graduated autonomy](https://yomu.fyi/post/closing-the-ai-agent-trust-gap-with-graduated-autonomy.md)
- Company: [AWS](https://yomu.fyi/company/aws.md)
- Author: Dev Arora
- Published: Aug 26, 2026
- Type: Problem & solution

Operating AI agents under binary full-access or read-only permissions creates an operational trust gap due to unpredictable model behavior across updates. The graduated autonomy architectural pattern closes this gap by dynamically granting or revoking tool permissions through sustained reliability scoring. A scoring engine computes a composite score across five weighted dimensions over a rolling 50-action window, mapping agents across four tiers from probation to full autonomy. In-process pre-execution filters check for prompt injection and sensitive targets, while Amazon Bedrock AgentCore Gateway and Cedar policies strictly enforce deny-by-default access at the infrastructure level. Post-execution state capture in Amazon DynamoDB supports reversibility, and AWS CodePipeline delivery gates block releases failing adversarial evaluation benchmarks.


### [How Much Code Do Developers Really Let Agents Write?](https://yomu.fyi/post/how-much-code-do-developers-really-let-agents-write.md)
- Company: [Jetbrains](https://yomu.fyi/company/jetbrains.md)
- Author: Mikhail Bogdanov
- Published: Aug 26, 2026
- Type: Benchmark

JetBrains surveyed over 15,000 professional developers globally in 2026 to measure how much production code is generated by AI agents versus manual authoring. On average, respondents report that agents fully write approximately 47% of their code, while 38% involves AI assistance and 27% is written manually. Adoption varies across ecosystems, with Go, JavaScript, and TypeScript developers reporting the highest agentic code generation at 54% to 55%, while C and C++ developers retain the highest manual share. Senior developers lead adoption over juniors, with about a quarter delegating more than 80% of code generation to agents. Cluster analysis categorizes the developer population into agentic coders (31%), AI-assisted coders (47%), and manual coders (23%).


### [When LLM judges agree, should we believe them?](https://yomu.fyi/post/when-llm-judges-agree-should-we-believe-them.md)
- Company: amazon.science
- Author: Krishna Balasubramanian, Sasha Podkopaev
- Published: Aug 26, 2026
- Type: Problem & solution

Multijudge panels in LLM-as-a-judge evaluations often suffer from correlated errors when models share prompt templates, training lineages, or evaluation rubrics. To account for these interdependencies without human reference labels, a method presented at ICML uses Ising models to simultaneously infer judge reliability and pairwise correlation. The unsupervised algorithm iteratively updates the soft probability of each item's true latent label while estimating both individual judge skill and pairwise dependence structures. Evaluated across retrieval relevance, toxicity classification, and summarization benchmarks with ten zero-temperature judges, the approach achieved test accuracies of 0.912, 0.792, and 0.806. These dependence-aware aggregation scores consistently outperformed weighted majority voting baselines by 9% to 14% on standard metrics across all three tasks.


### [Experiment with Qwen3.8-Flash-Next on NVIDIA GB300 NVL72 for Agentic Coding](https://yomu.fyi/post/experiment-with-qwen3-8-flash-next-on-nvidia-gb300-nvl72-for-agentic-c.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Aug 26, 2026
- Type: Announcement

Alibaba released the model weights for Qwen3.8-Flash-Next as a preview of the upcoming Qwen4 architecture, enabling developers to experiment with a multimodal mixture-of-experts model with a 125B-parameter main model, 51B N-gram embeddings, and 6B parameters activated per token. It supports a native 262,144-token context window, extensible to 1M tokens with YaRN. To reduce long-context attention compute and KV-cache growth, its hybrid design uses Gated DeltaNet in three of every four layers and Qwen Sparse Attention in the remaining layer, where micro-block importance guides retrieval. On NVIDIA GB300 NVL72, it delivers more than 16K tokens per second per GPU and more than 200 tokens per second per user; QSA achieved up to 7.6x prefill and 4.9x decoding speedups versus full attention. Fine-tuning is available through NVIDIA NeMo AutoModel with SFT or LoRA, and reinforcement learning through NVIDIA NeMo RL.


### [Compose Multiplatform 1.12.0 Released](https://yomu.fyi/post/compose-multiplatform-1-12-0-released.md)
- Company: [Jetbrains](https://yomu.fyi/company/jetbrains.md)
- Author: Elvira Mustafina
- Published: Aug 26, 2026
- Type: Announcement

JetBrains has announced the release of Compose Multiplatform 1.12.0 with targeted capabilities for artificial intelligence integrations, web rendering, and desktop application layouts. The update incorporates an experimental Model Context Protocol server inside Compose Hot Reload, allowing AI agents to trigger reloads, capture screenshots, inspect the semantic tree, and simulate user interactions directly. For web deployments, the framework now provides automatic font fallback by downloading necessary Noto font subsets on demand when unresolved characters occur during rendering. Desktop developers receive an experimental v2 window and dialog application programming interface in the androidx.compose.ui.window.v2 package, which distinguishes requested states from actual window states and permits precise control over positioning, sizing constraints, and multi-screen placement. These combined enhancements improve developer workflows and runtime flexibility across multiple target platforms.


### [How AI observability works with MLflow](https://yomu.fyi/post/how-ai-observability-works-with-mlflow.md)
- Company: [Red Hat](https://yomu.fyi/company/red-hat.md)
- Author: Cedric Clyburn
- Published: Aug 26, 2026
- Type: Explainer

Traditional application monitoring treats AI workflows as black boxes, leaving engineering teams unable to determine whether incorrect responses result from model hallucinations or erroneous upstream tool data. MLflow provides AI observability by capturing distributed traces containing model calls, prompts, retrieved context, execution latencies, and tool interactions. In an example mortgage lending application, trace inspection revealed that an AI assistant accurately summarized raw JSON data returned by an internal pipeline tool rather than hallucinating conflicting metrics. Beyond latency attribution and tool debugging, MLflow is compatible with OpenTelemetry and provides automatic tracing integrations for frameworks such as LangChain and Claude Code. Capturing these traces enables teams to evaluate subsequent application versions, detect model provider shifts, and identify tool regressions before reaching users.


### [Try the Ansible playbook generation lab with Gemini and OpenAI](https://yomu.fyi/post/try-the-ansible-playbook-generation-lab-with-gemini-and-openai.md)
- Company: [Red Hat](https://yomu.fyi/company/red-hat.md)
- Author: clobner
- Published: Aug 26, 2026
- Type: Announcement

The official Visual Studio Code extension for Red Hat Ansible Automation Platform includes an automation coding assistant, formerly called Ansible Lightspeed. Initially integrated with IBM watsonx, the extension now supports Google Gemini and OpenAI-compatible endpoints through Red Hat AI. An updated interactive lab demonstrates these capabilities through an end-to-end scenario involving database installation and HTML landing page deployment across two nodes. Learners write natural language prompts to generate Ansible playbooks and roles, evaluate output, and utilize real-time automated linting and integrated documentation. By serving an open-source model through a preconfigured OpenAI-compatible endpoint, the refreshed environment removes external credential setup steps.


### [GRPO fine-tuning on Red Hat OpenShift AI: Reinforcement learning from verifiable rewards with Training Hub](https://yomu.fyi/post/grpo-fine-tuning-on-red-hat-openshift-ai-reinforcement-learning-from-v.md)
- Company: [Red Hat](https://yomu.fyi/company/red-hat.md)
- Author: Fiona Waters
- Published: Aug 26, 2026
- Type: Tutorial

Supervised fine-tuning for structured reasoning tasks like tool calling often demands expensive, labor-intensive dataset curation. Group Relative Policy Optimization (GRPO) resolves this by using programmatic reward functions to score multiple candidate responses without requiring human labels or a dedicated reward model. On Red Hat OpenShift AI, developers can implement GRPO using Training Hub, Kubeflow Trainer, and the Kubeflow SDK to fine-tune models like Qwen3-4B. The setup utilizes vLLM for candidate generation alongside LoRA weight updates, sharing storage via ReadWriteMany persistent volume claims. In a practical tool-calling test, five training iterations raised task accuracy from 33% to 67%, teaching the model to emit parseable function invocations rather than fabricated conversational text.


### [PROOF-Gen: From Optimized Data to Better Distillation](https://yomu.fyi/post/proof-gen-from-optimized-data-to-better-distillation.md)
- Company: [Apple](https://yomu.fyi/company/apple.md)
- Published: Aug 26, 2026
- Type: Problem & solution

Standard distillation pipelines for tool-calling agents use generate-and-filter supervised fine-tuning on teacher trajectories, discarding failed trials and missing training signals on hard scenarios. On τ 2-bench, 57% of teacher trials fail, with two-thirds being near-misses undone by a single error. PROOF-Gen (Per-scenario Reflective Optimization to Overcome Failed Generation) addresses this loss by employing a reflector that reviews execution traces and evaluation feedback to craft corrective guidance for the teacher. Once passing trajectories are generated, this scaffold guidance is stripped so students train on clean demonstrations. Per-scenario prompt optimization recovers 93% of failed scenarios on τ 2-bench, boosting Qwen3-4B-Instruct-2507 Passˆ1 from 0.132 to 0.529 and lifting deployed on-device model performance across all locales.


### [IDEA Prune: An Integrated Enlarge-and-Prune Pipeline in Generative Language Model Pretraining](https://yomu.fyi/post/idea-prune-an-integrated-enlarge-and-prune-pipeline-in-generative-lang.md)
- Company: [Apple](https://yomu.fyi/company/apple.md)
- Published: Aug 26, 2026
- Type: Problem & solution

Recent advancements in large language models necessitate efficient deployment options within constrained inference budgets. While structured pruning offers better token efficiency than training target-sized models from scratch, previous work frequently ignores the pretraining of enlarged models. To resolve this gap, researchers developed an integrated enlarge-and-prune pipeline that unifies enlarged model training, pruning, and recovery under a single cosine annealing learning rate schedule. This system incorporates an iterative structured pruning technique to gradually eliminate parameters, redistributing model capacity across surviving neurons and avoiding the knowledge loss seen in naive approaches. Evaluated on compressing 2.8B parameter models to 1.3B across up to 2T pretraining tokens, the approach yields superior downstream model performance.


### [Restore LLM Inference Capacity in Seconds with Shadow Engine Recovery in NVIDIA Dynamo](https://yomu.fyi/post/restore-llm-inference-capacity-in-seconds-with-shadow-engine-recovery.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Aug 25, 2026
- Type: Problem & solution

LLM engine process failures typically trigger cold restarts that reload weights into HBM, compile kernels, and recapture CUDA graphs, leaving surviving workers to absorb displaced traffic for minutes. Shadow engine recovery, a preview feature in NVIDIA Dynamo, keeps a fully initialized standby on the same GPUs as the active engine. GPU Memory Service (GMS) preserves one physical weight copy across engine processes, while the parked shadow precomputes its CUDA context, graphs, communicators, and mappings but defers KV cache materialization. In a GLM-5.2 test, the shadow resumed service in 7.3 seconds after a worker failure, compared with 283 seconds for a cold restart, reducing post-fault p50 TTFT from 23,815 ms to 1,311 ms and improving decode rate from 12 to 46 tok/s/user. The preview does not cover hardware, node, or multi-node failures and currently starts promoted shadows with empty KV caches.


### [How Capital Markets Finance Protects Balance Sheet Returns](https://yomu.fyi/post/how-capital-markets-finance-protects-balance-sheet-returns.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Andrea DeSosa, Kim Hatton
- Published: Aug 25, 2026
- Type: Problem & solution

Capital markets finance must protect returns on a balance sheet exposed continuously to market, funding, and liquidity risk, while funding costs, capital consumption, and liquidity needs can change intraday. Fragmented desk P&L, treasury, and risk data require shared business context, with ontology and governance linking numbers to desks, products, positions, funding, capital, and permissions. Databricks’ Genie One is presented as a data-smart AI coworker that lets authorized finance users investigate governed data, trace emerging funding or profitability issues, and surface the definitions and source evidence behind capital and valuation figures. One top global bank reduced regulatory data processing for liquidity reporting from 10 hours to 8 minutes, while pricing, funding, risk, and capital decisions remain with accountable people.


### [Granite 4.2 LLMs: How They're Built](https://yomu.fyi/post/granite-4-2-llms-how-they-re-built.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Yousaf Shah, Swanand Kadhe, Riddhiman Moulick, Ashish Sunil Agrawal
- Published: Aug 25, 2026
- Type: Explainer

Granite 4.2 is IBM’s first family of dense, decoder-only reasoning language models, offered in 3B, 8B, and 30B sizes. The models use a shared design built around Grouped Query Attention, Rotary Position Embedding, SwiGLU, RMSNorm, bfloat16 precision, and separate input/output embeddings; each has a 131,072-token sequence length. Pre-training starts from scratch on approximately 15 trillion tokens across five phases, with the final phase extending the context window to 512K tokens. Supervised fine-tuning uses about 7.2 million samples spanning agentic and non-agentic data, followed by multi-stage reinforcement learning; the 8B and 30B models also receive agentic RL in sandboxed environments for tool use, code execution, terminal control, and web search. All models support thinking, non-thinking, and low-effort thinking modes, native tool calling, and Apache 2.0 licensing.


### [Quantization-Aware Healing: a compressed, 4-bit model that outperforms its full-precision original](https://yomu.fyi/post/quantization-aware-healing-a-compressed-4-bit-model-that-outperforms-i.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Antonio Tiene, Iker García-Ferrero, Ali Hashemi, Bakbergen Ryskulov
- Published: Aug 25, 2026
- Type: Benchmark

Structural compression and 4-bit quantization can reduce an LLM’s reasoning, mathematical, and coding capabilities, while conventional healing methods either require costly quantization-aware training or distill from an already degraded checkpoint. Quantization-Aware Healing (QAH) instead distills the original, full-size, full-precision model into a structurally compressed student running in MXFP4, using KL divergence over teacher logits. A chunked KL loss allows healing on documents up to 32k tokens within a fixed GPU memory budget. Applied to GPT-OSS 120B compressed to 60B parameters, QAH’s 4-bit model outperformed its 60B bfloat16 counterpart on 7 of 9 benchmarks, including gains of 7.4 points on AA-LCR and 5.6 on AIME 2025. In a separate comparison, QAH reached its peak about seven times faster than QAT and avoided QAT’s later 19-point collapse.


### [Ideas Worth a Longer Conversation: The JetBrains Research Podcast](https://yomu.fyi/post/ideas-worth-a-longer-conversation-the-jetbrains-research-podcast.md)
- Company: [Jetbrains](https://yomu.fyi/company/jetbrains.md)
- Author: Katie Fraser
- Published: Aug 25, 2026
- Type: Explainer

The JetBrains Research Podcast investigates foundational software development and computer science questions through discussions with industry researchers. In team psychology, Cat Hicks demonstrates that collaborative culture and organizational belonging reduce developer anxiety around automated tooling while curbing the risks of overproduction pressure. Historical analysis by Tomáš Petříček shows that programming spans five competing cultures rather than pure specification implementation, explaining why automated code generation historically encounters limits. Regarding artificial intelligence evaluation, Ibragim Badertdinov details how SWE-rebench isolates model capability through dataset decontamination, leveraging the strict verifiability of software test execution for training feedback. Additionally, conversations with Alexander Kulikov and Anna Kogan examine how foundational mathematical understanding aids AI oversight and address adoption hurdles for open-source computer vision infrastructure.


### [Inside LinkedIn's cognitive memory agent for agentic personalization](https://yomu.fyi/post/inside-linkedin-s-cognitive-memory-agent-for-agentic-personalization.md)
- Company: [Stack Overflow](https://yomu.fyi/company/stack-overflow.md)
- Author: Phoebe Sajor
- Published: Aug 25, 2026
- Type: Explainer

LinkedIn built a cognitive memory agent to deliver personalized experiences across its recruiter workflows and hiring assistants. Interactions between recruiters and agents often reveal recurring preferences, such as candidate criteria, role adjustments, and task habits. To preserve and manage this state throughout the entire lifecycle, the system organizes user data into a multi-tiered memory architecture. This structure comprises conversational, semantic, episodic, and procedural memory tiers that track everything from ongoing sessions to cross-session behaviors and workflow methods. LinkedIn continues to research memory abstractions, dynamic session compaction, virtual file system storage layers, and robust evaluation approaches for attribution across complex multi-source application agents.


### [Auth0, Made for Whatever You Are Building](https://yomu.fyi/post/auth0-made-for-whatever-you-are-building.md)
- Company: [Auth0](https://yomu.fyi/company/auth0.md)
- Author: Chris Sevilleja
- Published: Aug 25, 2026
- Type: Problem & solution

Rapid application development with AI coding tools often produces unreviewed custom authentication logic that handles sensitive Personally Identifiable Information and regulatory requirements like HIPAA. Autonomous AI agents compound these security challenges by significantly widening an application's overall attack surface. Standardizing on Auth0 provides an established identity layer that integrates into agentic workflows via Claude plugins, an Auth0 MCP server, and standardized agent skill packages. This setup exposes Auth0 Management APIs and official documentation directly to AI agents, achieving integration times of three to four minutes according to Agent Experience metrics. Incorporating these pre-built patterns maintains security baselines while supporting scaling requirements across B2B plans for initial user bases up to 1,000 monthly active users.


### [STARFlow2: Bridging Language Models and Normalizing Flows for Unified Multimodal Generation](https://yomu.fyi/post/starflow2-bridging-language-models-and-normalizing-flows-for-unified-m.md)
- Company: [Apple](https://yomu.fyi/company/apple.md)
- Published: Aug 25, 2026
- Type: Problem & solution

Existing multimodal models struggle with structural fragmentation, often sacrificing visual fidelity via discrete tokenization or breaking structural symmetry by pairing causal text generation with diffusion-based denoising. STARFlow2 addresses these limitations by leveraging autoregressive normalizing flows, which share the causal mask, key-value cache mechanism, and left-to-right structure of large language models. Built upon the Pretzel architecture, the system vertically interleaves a frozen pretrained vision-language model stream with a TARFlow stream through residual skip connections under a single causal mask. A deep-shallow flow design combined with a unified FAE latent space allows both text and continuous visual outputs to enter the key-value cache directly without re-encoding. Benchmark experiments validate strong performance across multimodal understanding and continuous image generation tasks.


### [From Prediction to Action: How to Turn AI Outputs Into Decisions](https://yomu.fyi/post/from-prediction-to-action-how-to-turn-ai-outputs-into-decisions.md)
- Company: [Salesforce](https://yomu.fyi/company/salesforce.md)
- Author: Scott Nyberg
- Published: Aug 24, 2026
- Type: Problem & solution

Salesforce addressed an operational challenge where sellers faced roughly 12,000 dashboards and over 20 applications outputting machine learning predictions without clear next steps. The engineering team reframed machine learning outputs as raw signals rather than standalone answers. To bridge the gap between assessment and action, they built a Next Best Action layer that combines model signals, business logic, and contextual institutional knowledge into actionable recommendations. They integrated this layer with an AI agent using Model Context Protocol (MCP) tool contracts, enabling dynamic discovery and explicit handling of missing data. Finally, rather than introducing a separate dashboard destination, the agent serves on-demand recommendations directly inside Slack where sellers already collaborate.


### [Your alt text passes automated checks. That doesn’t mean it’s any good.](https://yomu.fyi/post/your-alt-text-passes-automated-checks-that-doesn-t-mean-it-s-any-good.md)
- Company: [Github](https://yomu.fyi/company/github.md)
- Author: Taarik Ashenafi
- Published: Aug 24, 2026
- Type: Explainer

Automated accessibility checkers reliably flag missing alt text attributes but frequently miss unhelpful descriptions like raw filenames or repetitive labels. To evaluate image description quality without generating excessive false positives, GitHub built an alt text plugin for the GitHub Accessibility Scanner. The tool combines five deterministic, zero-credential rules that evaluate strings and visual layout spacing with an opt-in vision model check for subjective context. Page context including headings and surrounding prose is extracted alongside images to guide the model using structured outputs and explicit anti-nitpick instructions. While deterministic checks catch unwritten text, the model-driven rule serves as an opt-in prompt for human review rather than an absolute verdict.


[Newer posts](https://yomu.fyi/topic/llm/page/6.md) · [Older posts](https://yomu.fyi/topic/llm/page/8.md)