---
title: "LLMs"
description: "673 posts about LLMs, summarised, each linking to the original."
---

# LLMs
> 673 posts about LLMs, summarised, each linking to the original.

## Articles

### [How Databricks Uses AI to Accelerate Incident Investigation](https://yomu.fyi/post/how-databricks-uses-ai-to-accelerate-incident-investigation.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Avijeet Gupta, Bhuban Seth, Kusum Madarasu
- Published: Aug 24, 2026
- Type: Explainer

Databricks describes AI SRE, an AI-powered debugging agent for operating hundreds of microservices across more than 1,500 Kubernetes clusters, 70+ regions, and three clouds. It addresses incident investigations by assembling context from platform health, service telemetry, changes, dependencies, and team-specific runbooks, with automatic triage followed by interactive, natural-language investigation. The platform uses layered primitives, purpose-built APIs, and a core orchestration engine; team-owned agentic runbooks encode domain procedures, while controlled access and evidence links support auditing. The post reports support for more than 150 teams, 250+ weekly active users, and over 2,000 daily investigations, with users saving several hours of debugging time.


### [Help AI Coding Agents Write Up-To-Date Code With Modern Golang Skills](https://yomu.fyi/post/help-ai-coding-agents-write-up-to-date-code-with-modern-golang-skills.md)
- Company: [Jetbrains](https://yomu.fyi/company/jetbrains.md)
- Author: Artem Pronichev
- Published: Aug 24, 2026
- Type: Announcement

AI coding agents often generate outdated Go patterns because older syntax dominates training data and new features fall past model cutoffs. To address this gap, the GoLand team released Modern Go Guidelines, an open-source set of skills and CLI tools covering Go 1.0 through Go 1.27. The tool inspects a project's go.mod file or a specific version flag to supply only the language features and standard library additions compatible with that environment. Using a progressive disclosure approach, the CLI provides concise rule identifiers via a list command and expands into before-and-after examples through an explain command. This mechanism minimizes token consumption while helping agents produce up-to-date, compilable code without modifying project files.


### [How We Optimized the Qwen 3.6 Model for Our Junie Agent](https://yomu.fyi/post/how-we-optimized-the-qwen-3-6-model-for-our-junie-agent.md)
- Company: [Jetbrains](https://yomu.fyi/company/jetbrains.md)
- Author: Stanislav Erokhin
- Published: Aug 24, 2026
- Type: Problem & solution

Deploying the Junie coding agent locally on Apple MacBook M5 hardware requires overcoming steep performance bottlenecks during model execution. Engineers optimized the entire stack using Qwen3.6-27B rather than Qwen3.8-27B, which severely degrades without token-heavy reasoning enabled. At the agent layer, Junie appends requests directly into a rolling context and caches initial prompt prefixes to maximize KV-cache reuse across tasks. For model inference, the team disabled reasoning, selected 4-bit quantization, and patched MLX-VLM to execute self-attention prefill matrix operations using faster 8-bit instructions on the M5 processor. Combining these prefill improvements with simultaneous multi-token prediction and n-gram speculative decoding delivered up to a 2x generation speedup.


### [Junie Can Now Run Entirely on Your Mac – No Credits, No Cloud](https://yomu.fyi/post/junie-can-now-run-entirely-on-your-mac-no-credits-no-cloud.md)
- Company: [Jetbrains](https://yomu.fyi/company/jetbrains.md)
- Author: Dmitry Savelev
- Published: Aug 24, 2026
- Type: Announcement

JetBrains has introduced Junie Local, a feature that runs the Junie coding agent entirely on local hardware without cloud dependencies, subscriptions, or credit meters. Users execute a single command inside Junie to download a 4-bit quantized Qwen3.6-27B model, requiring an Apple M5 Mac with 64 GB of RAM. The team prioritized prefill throughput over pure generation speed by utilizing 8-bit arithmetic instructions on the M5 Neural Accelerator, which increased prefill throughput by approximately 40%. Performance optimizations also include KV-cache reuse, speculative decoding to double generation speed, and disabling reasoning to preserve performance. In JetBrains' evaluations, the local configuration performed comparably to cloud-based Sonnet 4.5 on everyday tasks while ensuring source code, diffs, and prompts never leave the local environment.


### [Responsible AI adoption needs developer workflow design](https://yomu.fyi/post/responsible-ai-adoption-needs-developer-workflow-design.md)
- Company: [Stack Overflow](https://yomu.fyi/company/stack-overflow.md)
- Author: Dr. Gleb Tsipursky
- Published: Aug 24, 2026
- Type: Explainer

Responsible AI adoption struggles when organizations rely on static policy documents instead of integrating guardrails directly into engineering workflows. Developers under delivery pressure inevitably turn to unapproved shadow AI when sanctioned routes introduce friction. Stack Overflow survey data reveals that 84% of respondents use or plan to use AI tools, yet more developers distrust AI accuracy than trust it. Operational controls must be embedded inside repositories, build pipelines, and pull request reviews to match specific failure modes like those listed by OWASP. Organizations achieve dependable adoption by pairing automated controls with clear role ownership, psychological safety, role-specific training, and outcome-based engineering metrics.


### [Run LoRA fine-tuning on Red Hat OpenShift AI with Ray](https://yomu.fyi/post/run-lora-fine-tuning-on-red-hat-openshift-ai-with-ray.md)
- Company: [Red Hat](https://yomu.fyi/company/red-hat.md)
- Author: Fiona Waters
- Published: Aug 24, 2026
- Type: Tutorial

Red Hat OpenShift AI 3.5 integrates Ray cluster runtime images with preinstalled Training Hub packages to support distributed fine-tuning algorithms including LoRA, SFT, OSFT, and GRPO. Using the CodeFlare software development kit within a Jupyter workbench, developers can provision an elastic single-GPU Ray cluster attached to shared ReadWriteMany persistent storage containing model weights and training datasets. The tutorial demonstrates fine-tuning the Qwen2.5-1.5B-Instruct model on the public sql-create-context dataset to produce direct, bare SQL output without explanatory conversational text. After training completes, the workbench loads and merges the resulting LoRA adapter checkpoints with the base model to evaluate output format correctness against sample database schema questions. The RayCluster automatically tears down resources once the job concludes, while the shared pipeline architecture allows switching between supported fine-tuning algorithms by modifying dataset inputs and configuration parameters.


### [Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning](https://yomu.fyi/post/beyond-visual-cot-internalized-visual-thinking-for-proactive-video-rea.md)
- Company: [Apple](https://yomu.fyi/company/apple.md)
- Published: Aug 24, 2026
- Type: Problem & solution

Visual chain-of-thought methods allow multimodal large language models to reason through intermediate image generation, but synthesizing explicit visual steps creates major inference overhead during proactive video reasoning. To eliminate runtime synthesis, Internalized Visual Thinking introduces a post-training framework that jointly optimizes textual prediction and next-embedding prediction across unlabeled videos. By predicting latent representations of future frames alongside target text answers during training, the model captures motion, object transitions, interactions, and latent intent. At inference time, the model produces answers directly without generating or re-encoding future frames. IVT outperforms text-only post-training across six evaluation settings while matching or exceeding Visual CoT performance and cutting end-to-end latency by more than 5x.


### [Say it once: introducing Bot Preference Sync](https://yomu.fyi/post/say-it-once-introducing-bot-preference-sync.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Jin-Hee Lee
- Published: Aug 21, 2026
- Type: Announcement

Cloudflare announced Bot Preference Sync, a feature across all customer tiers that automatically reflects dashboard AI bot configurations directly in a site's robots.txt file. Website operators previously faced challenges synchronizing static robots.txt directives with edge enforcement rules, which led some crawlers to ignore stated preferences or attempt bypasses. Under this system, Cloudflare generates and prepends directives for Search, Agent, and Training categories to existing robots.txt files using crawler lists tracked in BotBase. Mixed-use crawlers performing both search indexing and training must meet strict transparency criteria, including honoring opt-outs and providing page-level visibility, to avoid edge blocks when training is disallowed. Additionally, onboarding workflows allow ad-supported publishers to automatically set AI model training to Disallow by default while retaining search visibility.


### [Build a unified AI agent architecture with DynamoDB and Bedrock](https://yomu.fyi/post/build-a-unified-ai-agent-architecture-with-dynamodb-and-bedrock.md)
- Company: [AWS](https://yomu.fyi/company/aws.md)
- Author: Dhananjay Karanjkar
- Published: Aug 21, 2026
- Type: Problem & solution

AI agent architectures on AWS frequently divide operational records and vector embeddings across separate databases, creating synchronization lag and infrastructure overhead. To resolve this fragmentation, a single DynamoDB table can store operational document attributes alongside embeddings by leveraging native vector search. An Amazon Bedrock agent directs incoming queries to AWS Lambda action groups, which run standard CRUD operations or semantic searches against the DynamoDB SearchVectors API. When items change, a DynamoDB Streams pipeline uses Amazon Titan Text Embeddings V2 to generate new embeddings and write them back to the table. This unified approach eliminates standalone vector databases while keeping semantic indexes synchronized with operational updates.


### [Connecting retail demand planning to campaign and store execution](https://yomu.fyi/post/connecting-retail-demand-planning-to-campaign-and-store-execution.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Jack Yallop, Pavi Singh
- Published: Aug 21, 2026
- Type: Explainer

Retail and consumer goods teams often split performance review, demand planning, marketing activation, store execution, and measurement across disconnected systems, making it difficult to connect analysis with results. The Databricks demo presents a unified application built on data unification, governance, and intelligence, combining point-of-sale, loyalty, supply chain, media spend, inventory, and store-operations signals with role-specific access. Users can move from KPI and forecast-versus-actual analysis to SKU or store what-if planning, audience selection, campaign activation, replenishment, price-tag updates, assigned tasks, and post-campaign measurement. Genie supports natural-language querying, while specialized agents and a coordinating supervisor connect sales insights, demand planning, audience building, in-store operations, and measurement. The stated design goal is a governed, shared workflow that lets teams test decisions, execute them across stores, and evaluate outcomes without losing operational context.


### [SOP-Bench: A new benchmark for evaluating AI agents on real business procedures](https://yomu.fyi/post/sop-bench-a-new-benchmark-for-evaluating-ai-agents-on-real-business-pr.md)
- Company: amazon.science
- Author: Rohith Nama, Nandi Subhrangshu
- Published: Aug 21, 2026
- Type: Benchmark

AI agents often struggle to execute standard operating procedures because real-world business workflows contain implicit domain knowledge and procedural ambiguity that standard benchmarks overlook. To evaluate agent reliability, researchers introduced SOP-Bench, an open framework featuring more than 2,000 runnable tasks across 12 industrial domains paired with functioning tool interfaces and ground-truth answers. Domain experts authored the workflows, while an Anthropic Claude 3.5 Sonnet v2 model generated mock APIs, tool code, and edge-case datasets before expert verification. Experiments evaluating function-calling and reasoning agents across 11 frontier models revealed that newer models did not consistently outperform older versions, and providing 20 extraneous tools nearly halved task success. The benchmark, baseline agents, and evaluation code are released on GitHub and HuggingFace for reproducible testing.


### [Dispatches from O'Reilly: The right amount of spec for agentic development](https://yomu.fyi/post/dispatches-from-o-reilly-the-right-amount-of-spec-for-agentic-developm.md)
- Company: [Stack Overflow](https://yomu.fyi/company/stack-overflow.md)
- Author: Markus Eisele
- Published: Aug 21, 2026
- Type: Explainer

As software agents make implementation faster and cheaper, engineering difficulty shifts away from typing code toward defining correctness and verifying behavior reliably. Omitting specifications forces human reviewers into expensive correction loops, while overspecification clutters model context windows with stale prose and competing instructions. To find the optimal middle ground, teams must validate specifications before coding, using one agent to draft minimal constraints and another to attack ambiguities, hidden dependencies, and missing failure modes. Multi-agent architectures demand formal interface contracts, schemas, and executable checks because interpretive drift compounds rapidly across unbounded agent handoffs. Core Extreme Programming and Agile practices such as continuous integration, small release batches, and rapid feedback loops remain essential error-detection mechanisms during agentic development.


### [Where Security Fits in an AI Agent Stack](https://yomu.fyi/post/where-security-fits-in-an-ai-agent-stack.md)
- Company: [NVIDIA](https://yomu.fyi/company/nvidia-developer-blog.md)
- Author: Michelle Horton
- Published: Aug 21, 2026
- Type: Explainer

The post explains where security controls should sit in an emerging AI agent stack as agents gain capability and operate over longer horizons. It distinguishes behavioral controls in models and harnesses, which guide proposed actions, from authoritative infrastructure controls that bind identity, enforce policy, contain failures, and record activity. The stack is presented as layers for models, harnesses, meta-harnesses, secure runtimes such as NVIDIA OpenShell, and inference infrastructure, with the runtime boundary established before launch so tools, plugins, MCP processes, and delegated runtimes operate under policy. It recommends least privilege, isolation, explicit authorization, auditability, short-lived and task-scoped access, independent checks, and safer failure states, while describing four security profiles from isolated to adversarial workloads. The conclusion is that every high-impact external effect must pass through enforcement layers the agent cannot bypass.


### [Running AI agents in GitHub Actions with Docker Sandboxes](https://yomu.fyi/post/running-ai-agents-in-github-actions-with-docker-sandboxes.md)
- Company: [Docker](https://yomu.fyi/company/docker.md)
- Author: Jennifer Kohl
- Published: Aug 21, 2026
- Type: Tutorial

GitHub Agentic Workflows added support for Docker Sandboxes as an agent runtime, allowing CI coding agents to operate within microVM isolation. Useful AI agents require broad capabilities like executing arbitrary shell commands and launching containers, which elevates risk on shared CI runners. With the docker-sbx runtime, agents receive root privileges and a private Docker daemon inside a dedicated microVM while external repository access and network traffic remain restricted. In a demonstration, an agent ran on an Ubuntu runner, executed Java 21 integration tests with PostgreSQL via Testcontainers, identified an email case-sensitivity defect, applied a code fix, and opened a draft pull request. The complete workflow executed in eleven minutes and sixteen seconds while confining modifications strictly to source files.


### [How AgentFlo built AI sales agents with Amazon Bedrock AgentCore – Part 2](https://yomu.fyi/post/how-agentflo-built-ai-sales-agents-with-amazon-bedrock-agentcore-part-2.md)
- Company: [AWS](https://yomu.fyi/company/aws.md)
- Author: Muhammad Musab Iqbal
- Published: Aug 21, 2026
- Type: Problem & solution

AgentFlo designed an architecture using Amazon Bedrock AgentCore and AWS serverless infrastructure to address traffic spikes and build trust in autonomous commerce agents. Security operates across three stages: AWS Fargate filters prompt injections and opt-outs, AgentCore Gateway enforces deterministic Cedar authorization policies during tool execution, and post-turn privacy filters screen outputs. Session isolation is maintained across merchants through AgentCore runtime microVMs, while AgentCore Observability and Amazon CloudWatch provide end-to-end tracing. Data reliability and long conversation journeys rely on Amazon DynamoDB context storage, Amazon Bedrock Knowledge Bases, and Amazon S3 Vector embeddings. Early deployment data showed a 12 percent net revenue uplift.


### [GLM-5.3 vs. GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing](https://yomu.fyi/post/glm-5-3-vs-gpt-5-6-sol-on-deepswe-cost-coding-and-routing.md)
- Company: [Together AI](https://yomu.fyi/company/together-ai.md)
- Published: Aug 21, 2026
- Type: Benchmark

GLM-5.3 and GPT-5.6 Sol were evaluated on 113 DeepSWE tasks with 904 rollouts, comparing accuracy, cost, and failure patterns. Sol edges ahead at pass@1 (72.7% vs 69.0%), but GLM-5.3 leads at pass@4 (87.6% vs 85.8%) at $3.99 per rollout, half Sol’s $8.37 cost and yielding 17 solves per $100 versus Sol’s 9. Their low per-task correlation of 0.43 makes a cascade effective: GLM-5.3 first, then Sol on test failure, achieving 85.9% solved at $6.61 per task. Sol runs faster (19 minutes, 61 steps vs 35 minutes, 124 steps) and achieves higher reliability (61 tasks solved all four tries vs 48), but it breaks passing tests in 20% of failures versus GLM-5.3’s 11%.


### [GLM-5.3 vs. Claude Fable 5 on DeepSWE: Cost, Coding, and Routing](https://yomu.fyi/post/glm-5-3-vs-claude-fable-5-on-deepswe-cost-coding-and-routing.md)
- Company: [Together AI](https://yomu.fyi/company/together-ai.md)
- Published: Aug 21, 2026
- Type: Benchmark

GLM-5.3 and Claude Fable 5 were compared on the DeepSWE benchmark across 113 tasks with four trials each, totaling 904 rollouts. On first-attempt accuracy, Fable 5 scored 69.7% and GLM-5.3 scored 69.0%, a statistical tie inside error bars. GLM-5.3 led on pass@2 (81.1% vs. 77.1%) and pass@4 (87.6% vs. 84.1%), and cost $3.99 per rollout versus Fable’s $21.63, a 5.4x price gap. Per $100, GLM-5.3 solved 17 tasks while Fable solved 3. Both models regress test suites in only 11% of failures, and GLM-5.3 averages 35 minutes per rollout with 80k output tokens versus Fable’s 34 minutes and 114k tokens. The models are near-substitutes with 0.65 per-task correlation; GLM-5.3 won five of eight domains including JavaScript and concurrency, while Fable excelled only in Rust and serialization, making GLM-5.3 the practical default.


### [Building Jarvis Pro: Route first, answer later](https://yomu.fyi/post/building-jarvis-pro-route-first-answer-later.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Christian Coffrant
- Published: Aug 21, 2026
- Type: Problem & solution

Grab designed Jarvis Pro as an AI assistant for account managers turning complex account data into actionable merchant conversations. Early prototypes generated polished but operationally unsound advice, such as pushing promotions when order drops were caused by outlet pauses and fulfillment issues. To address this, the team implemented a "route first, answer later" design that uses a prompted LLM classification step constrained by a routing guide to dictate context, tool paths, answer structure, and guardrails before generation. The pipeline also isolates narrow memory decisions, reconciles metric freshness across data sources, and caps tool calls to maintain usability. In offline evaluations across 351 realistic prompts, routing matched safe routes 99.4% of the time while answer quality scores in a portfolio subset improved from 78.5 to 91.0.


### [Why AI-Generated Code Is Easy but Engineering Trust Is Hard](https://yomu.fyi/post/why-ai-generated-code-is-easy-but-engineering-trust-is-hard.md)
- Company: [Salesforce](https://yomu.fyi/company/salesforce.md)
- Author: Scott Nyberg
- Published: Aug 20, 2026
- Type: Problem & solution

At Salesforce, engineering teams discovered that AI coding agents could produce internally consistent code and passing test suites that still failed to meet actual requirements. To ensure correctness before writing code, the team developed an agentic Spec-Driven Development workflow centered on explicit specifications and gated verification. The system separates evidence-based repository lookups handled by agents from subjective judgment calls escalated to human engineers. Implementation plans must cite repository evidence reviewed by a Skeptic Agent, while a Compliance Matrix traces each success criterion directly to executable test proof. Finally, a multi-agent review system evaluates the implementation under an asymmetric rule allowing an independent judge to downgrade passing checks but never override failed gates.


### [AI Won't Replace Project Managers, But It is Reshaping How Work Gets Done](https://yomu.fyi/post/ai-won-t-replace-project-managers-but-it-is-reshaping-how-work-gets-do.md)
- Company: [Stack Overflow](https://yomu.fyi/company/stack-overflow.md)
- Author: Emim Vigin Lumina
- Published: Aug 20, 2026
- Type: Explainer

Technical project managers historically spend 60 to 70 percent of their working time managing routine coordination tasks, including manual spreadsheet updates and obsolete status reporting. Engineering organizations are addressing this overhead by integrating generative artificial intelligence, large language models, and agentic workflows into delivery pipelines. Automated agents ingest telemetry directly from Git commits, pull request comments, and CI/CD logs to assess system state while machine learning models execute Monte Carlo simulations to forecast delivery risks. Although this automation reduces administrative overhead below 30 percent, implementations face practical obstacles like inconsistent ticket definitions across teams and algorithmic over-reliance on uncontextualized historical patterns. Consequently, technical project managers are transitioning into strategic roles that coordinate hybrid human-agent workforces while providing critical judgment and organizational context.


[Newer posts](https://yomu.fyi/topic/llm/page/7.md) · [Older posts](https://yomu.fyi/topic/llm/page/9.md)