---
title: "LLMs"
description: "673 posts about LLMs, summarised, each linking to the original."
---

# LLMs
> 673 posts about LLMs, summarised, each linking to the original.

## Articles

### [Bringing Gemma 4 12B to your Laptop: Unlocking Local, Agentic Workflows with Google AI Edge](https://yomu.fyi/post/bringing-gemma-4-12b-to-your-laptop-unlocking-local-agentic-workflows.md)
- Company: [Google](https://yomu.fyi/company/google.md)
- Author: Google AI Edge Team
- Published: Jun 3, 2026
- Type: Announcement

Google DeepMind’s Gemma 4 12B is presented as an open model for running agentic, multimodal capabilities locally on everyday laptops through the Google AI Edge stack. The release covers Google AI Edge Gallery for local data analysis and coding, including Python generation, execution, chart rendering, and a reported 3D rendering workflow that can generate dependencies, code, and self-corrections in one turn. Google AI Edge Eloquent adds fully on-device dictation, transcription, and Voice Edit commands for rewriting or translating selected text, while LiteRT-LM’s new serve command exposes the model through a local, OpenAI-compatible endpoint for tools and frameworks. The post says Gemma 4 12B delivers a 60%+ overall quality improvement over prior models for Eloquent’s use case and keeps data on-device, with performance and memory benchmarks provided in the model card.


### [RAG vs Fine Tuning: Enterprise Decisions for AI Models and AI Systems](https://yomu.fyi/post/rag-vs-fine-tuning-enterprise-decisions-for-ai-models-and-ai-systems.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Databricks Staff
- Published: Jun 2, 2026
- Type: Explainer

The guide frames enterprise choice between retrieval-augmented generation (RAG) and fine tuning as a trade-off between injecting knowledge at inference time and changing model weights before deployment. RAG connects an LLM to external stores by embedding a query, searching vector databases for similar chunks, and adding retrieved context to the prompt, making current information and citations possible. Fine tuning uses curated input-output examples and supervised training to update parameters; full-model training is expensive, while LoRA trains a smaller set of added weights and can support a single-GPU pilot. The decision depends on whether the priority is stable behavior and output format or fresh, traceable knowledge, alongside data quality, latency, compute, and maintenance requirements. For many mature deployments, the guide recommends combining both: fine tuning for domain understanding and consistency, and RAG for real-time data.


### [Introducing Mellum2: A 12B Mixture-of-Experts Model by JetBrains](https://yomu.fyi/post/introducing-mellum2-a-12b-mixture-of-experts-model-by-jetbrains.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Nikita Pavlichenko
- Published: Jun 1, 2026
- Type: Announcement

JetBrains is releasing Mellum2, a 12B-parameter Mixture-of-Experts model trained from scratch on natural language and code. It targets latency-sensitive, high-throughput workloads including routing, retrieval-augmented generation (RAG), summarization, sub-agents, and coding features. The model has 12B total parameters but activates 2.5B per token, with its architecture keeping total capacity high while making inference more efficient for real-time use. Released under the Apache 2.0 license, Mellum2 is described as competitive with similarly sized open models on code generation, reasoning, science, and math benchmarks while delivering more than 2x faster inference. Its text-and-code focus supports high-throughput production workloads and private deployments where proprietary code or internal data may be involved.


### [Beyond LLMs: Why Scalable Enterprise AI Adoption Depends on Agent Logic](https://yomu.fyi/post/beyond-llms-why-scalable-enterprise-ai-adoption-depends-on-agent-logic.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Nicholas Fuller
- Published: Jun 1, 2026
- Type: Explainer

Enterprise workflows are dynamic, long-running, API- and data-rich, and constrained by policies or regulations, creating challenges that an LLM alone may not handle efficiently. The article defines agent logic as software primitives—including knowledge graphs, algorithms, and program-analysis libraries—within an agent harness that steer models toward relevant workflow context. Examples span legacy COBOL and PL/1 understanding, automated test generation, incident investigation and code remediation, and compliance modernization; reported results include ~30× lower token consumption for application understanding, 20–45% higher test coverage, and up to 4.0× better incident-investigation performance than a ReAct baseline. A Condition Insights example reports 57% fewer unsupported claims and 77% lower average token usage, supporting the conclusion that structured guidance can improve quality, cost, and rule adherence.


### [Reinforcement learning is an infrastructure problem](https://yomu.fyi/post/reinforcement-learning-is-an-infrastructure-problem.md)
- Company: [Modal](https://yomu.fyi/company/modal.md)
- Published: Jun 1, 2026
- Type: Announcement

Reinforcement learning for post-training large language models has become infrastructure-constrained as teams move to multi-node configurations for larger models like Qwen 3 30B and Kimi K2.6. The RL loop divides into three independent infrastructure challenges: training with reliable forward and backward passes, high-performance inference rollouts, and thousands of concurrent sandbox environments for policy execution. Weight synchronization between trainer and rollout engine becomes the critical bottleneck in multi-node setups, with RDMA offering up to 100x speedup over TCP and significant cost savings per step. Teams commonly get stuck maintaining glue code, queueing for cluster time, and under-utilizing GPUs when sandbox buffers are not correctly sized. Modal addresses these with RDMA-connected clusters via a single rdma=True keyword, fast container boot, and the new open-source Modal Training Gym library that abstracts common scaffolding into under 100 lines of code.


### [From Rigid Rules to Empathetic Agents: Rebuilding Swiggy’s Order Cancellation Experience with LLMs](https://yomu.fyi/post/from-rigid-rules-to-empathetic-agents-rebuilding-swiggy-s-order-cancel.md)
- Company: [Swiggy](https://yomu.fyi/company/swiggy.md)
- Author: Aishwary Yadav
- Published: May 29, 2026
- Type: Problem & solution

Swiggy rebuilt its “I Want To Cancel” (IWTC) support journey because a roughly 150K-line deterministic decision tree treated cancellation keywords alike, repeated canned deterrence, and could leave users stuck without handoff. The replacement is a stateful, multi-turn framework orchestrated with LangGraph: an intent layer identifies needs such as delay help, coupon support, direct cancellation, or human access, while live order signals and deterministic policy checks determine the next action. Full graph state is checkpointed in Redis by order- and session-level thread IDs, allowing conversations to resume across turns. Swiggy evaluated intent reliability using more than 100k historical chats and a pre-deployment suite of about 500 representative queries, then operated the system with reporting and alerts for behavioral and service metrics. The conclusion favors a hybrid design in which LLMs handle language while deterministic systems anchor truth and business decisions.


### [Explaining Every Rupee: How We Built Reliable LLM Support Bots for Delivery Partners](https://yomu.fyi/post/explaining-every-rupee-how-we-built-reliable-llm-support-bots-for-deli.md)
- Company: [Swiggy](https://yomu.fyi/company/swiggy.md)
- Author: Yash Khandelwal
- Published: May 29, 2026
- Type: Problem & solution

Swiggy faced close to 9 lakh monthly payout- and incentive-related tickets, where delivery executives needed accurate explanations of earnings, deductions, missed incentives, and credited amounts. The system separates financial truth from language generation: deterministic preprocessing and internal APIs resolve incentive eligibility, payout breakdowns, deductions, and conditions before an LLM explains the result. Two multilingual bots—the Daily Incentive Bot and Payout Bot—use a LangGraph StateGraph with LLM, tool, and validator nodes, Redis checkpointing for multi-turn context, and post-processing translation into the executive’s preferred language. Azure Translator preserve-list controls help keep business-critical terminology consistent, while structured escalation sends out-of-scope cases and context to human agents. Moving retrieval from Databricks SQL queries to an aggregated internal API reduced latency from 8–10 seconds to milliseconds; the bots now resolve about 90% end to end, cut resolution from roughly 20 minutes to seconds, and improved CSAT by 40%.


### [Beyond code generation: rethinking engineering productivity in the age of AI agents](https://yomu.fyi/post/beyond-code-generation-rethinking-engineering-productivity-in-the-age.md)
- Company: [Dropbox](https://yomu.fyi/company/dropbox.md)
- Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra,Kazuaki Okumura
- Published: May 28, 2026

Dropbox shares how widespread AI code generation shifts software development bottlenecks downstream into code review, CI infrastructure, and validation pipelines. To adapt, they built Nova, an internal coding agent platform that safely automates scoped tasks such as migrations and flaky test remediation. They also evolved their developer productivity framework to measure end-to-end customer impact and code quality rather than simple pull request throughput.


### [Slack AI: The Path to Multi-Cloud](https://yomu.fyi/post/slack-ai-the-path-to-multi-cloud.md)
- Company: [Slack](https://yomu.fyi/company/slack.md)
- Author: Shaurya Kethireddy
- Published: May 28, 2026
- Type: Problem & solution

Slack evolved its Slack AI serving infrastructure across multiple phases to handle enterprise LLM workloads reliably and securely. The initial deployment on AWS SageMaker provided zero-knowledge escrow VPC isolation and FedRAMP compliance, but engineers faced scaling latency, GPU scarcity, and significant operational overhead. Slack then migrated live traffic to Amazon Bedrock to leverage managed Model Units and eliminate model release lag without customer-facing incidents. However, fixed Provisioned Throughput commitments and regional peak traffic variations created persistent underutilization challenges. Consequently, Slack expanded into a multi-cloud orchestration architecture that normalizes disparate provider APIs, integrates unified cross-cloud telemetry, and routes traffic dynamically around latency spikes and outages.


### [How we cut build times by two-thirds by deleting our CMS](https://yomu.fyi/post/how-we-cut-build-times-by-two-thirds-by-deleting-our-cms.md)
- Company: [Sentry](https://yomu.fyi/company/sentry.md)
- Author: Eli Lennox
- Published: May 28, 2026
- Type: Problem & solution

Sentry rebuilt its marketing site and blog of roughly 2,500 pages, replacing a legacy Gatsby setup and traditional headless CMS with Astro, Markdown, Frontmatter, and AI-driven automation. The previous architecture suffered from 14-minute build times, restrictive CMS schema limits, and frequent build failures caused by external API dependencies. A team of 2.5 developers executed the migration in two months using Claude Code agents for data extraction, template translation, and testing. To manage content without a CMS dashboard, Sentry built Claude Skills to guide updates, run local previews, and draft GitHub pull requests. The migration reduced average build times to under 4 minutes, cut broken staging builds by 95 percent, and improved Web Vitals scores from 89 to 97.


### [Under the River](https://yomu.fyi/post/under-the-river.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: May 28, 2026
- Type: Explainer

Shopify describes how its AI agent, River, grew from an infrastructure bet aimed at making a large, fragmented codebase usable by AI. In early 2024, the company adopted a monorepo called World and Nix as a reproducible substrate for development, CI, and production, despite substantial migration costs. River operates in public Slack channels, where it reads code, runs tests, queries data, inspects production traces, and opens pull requests; in a recent 30-day period it handled 59,918 sessions and contributed to 3,536 merged pull requests. Underneath it, Aquifer provides durable sessions, sandboxes, gateways, event logs, credentials handling, and observability, allowing agent profiles to share one platform across interactive, automated, and batch modes. The central design principle is that the session must survive process and machine failure, while public transcripts create a compounding knowledge corpus.


### [Open House 2026 Day 2: 10 years of open source and what the ecosystem is building next](https://yomu.fyi/post/open-house-2026-day-2-10-years-of-open-source-and-what-the-ecosystem-i.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: May 28, 2026
- Type: Announcement

Open House 2026 Day 2 recaps ClickHouse’s tenth year as an open-source project and the ecosystem forming around its data platform. The event reported more than 2,600 contributors, nearly 48,000 GitHub stars, over 200 integrations, and the launch of the House Mates partner program. Announcements included Fivetran’s generally available ClickHouse Cloud destination, a Google Cloud Dataflow Pub/Sub template, Sigma’s public-beta connector and ETL Cache Layer, and a ClickHouse dbt Fusion adapter available in alpha through the CLI. The recap also describes MCP and agent-oriented observability work, including a ClickStack MCP server that translates investigations into optimized ClickHouse queries. Internal testing reported up to 25% fewer tool calls, 2.5 times greater consistency, and nearly 20% better evaluation results.


### [Open House observability announcements: MCP server, AI Notebooks, and ClickStack Cloud](https://yomu.fyi/post/open-house-observability-announcements-mcp-server-ai-notebooks-and-cli.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: May 27, 2026
- Type: Announcement

At Open House, ClickHouse announced three observability updates: ClickStack Cloud entered private preview, AI Notebooks entered beta for Managed ClickStack, and a ClickStack MCP server was introduced in open-source ClickStack. ClickStack Cloud is a fully managed, serverless platform that accepts OpenTelemetry data through a managed endpoint, while Managed ClickStack is generally available for teams controlling collectors, ingestion, compute sizing, schema design, and tuning. AI Notebooks provide a persistent, shareable investigation workspace in which prompts, queries, charts, reasoning steps, findings, branches, intermediate results, and generated queries remain visible and editable. The MCP server exposes the investigative tools used by Notebooks to external agents, routing requests through optimized searches, aggregations, and visualizations rather than relying entirely on ad hoc SQL generation; it can also persist findings in dashboards. Together, the announcements present a choice between managed convenience, operational control, and programmable observability workflows.


### [Reliable LLM Inference at Scale](https://yomu.fyi/post/reliable-llm-inference-at-scale.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Ying Chen, Wendy Hu, Ankit Mathur, Mike Eastham, Pei-Lun Liao, Wai Wu, Arjun DCunha
- Published: May 27, 2026
- Type: Explainer

Databricks describes an inference platform serving open-source and proprietary frontier models at more than 125T tokens per month, with reliability as the central challenge under spiky, multi-tenant demand. It identifies failures and latency risks from expensive GPU configurations, variable input and output lengths, long-context requests, multimodal preprocessing, and engine-level hangs. Its architecture combines rate limiting, capacity management, model units, Dicer load-aware sticky routing, and autoscaling; model units estimate multidimensional request cost and allocate capacity more predictably. For runtime reliability, prioritized black-box health checks trigger Kubernetes liveness restarts for silent hangs, completing detection and recovery in less than five minutes while eliminating false probe failures. Image-processing changes, including Torchvision processors and OMP\_NUM\_THREADS configuration, removed CPU throttling and increased completed requests per second by more than threefold with the same replicas and load; bursty workloads also achieved over 80% GPU savings versus static peak provisioning.


### [Thank you for building with us](https://yomu.fyi/post/thank-you-for-building-with-us.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: May 27, 2026
- Type: Announcement

At its 2026 user conference in San Francisco, ClickHouse reports surpassing 4,000 customers and $250 million in annual run-rate revenue, while marking nearly ten years since Alexey Milovidov open-sourced the database. The company attributes its growth to a community of contributors and describes use across AI products, financial services, observability, analytics, and infrastructure. New offerings include ClickHouse Agents, a fully managed agentic analytics service powered by Anthropic’s Claude; CostBench, an open benchmark using each warehouse’s real billing model; and House Mates, a partner program launching with more than 50 partners across six continents. Additional updates include Managed Postgres in public beta, Managed ClickStack and Langfuse for AI-era observability, generally available full-text search, and automatic JOIN optimization, with cost-performance identified as the central constraint for high-concurrency, low-latency workloads.


### [ClickHouse tops $250M ARR and 4,000 customers, launches Claude-powered agents at Open House 2026](https://yomu.fyi/post/clickhouse-tops-250m-arr-and-4-000-customers-launches-claude-powered-a.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: May 27, 2026
- Type: Announcement

At Open House 2026, ClickHouse announced that its serverless cloud offering exceeded $250 million in annual run-rate revenue and reached 4,000 customers, with ARR more than tripling year-over-year and more than 1,000 net new customers added since January. The company launched ClickHouse Agents, a fully managed, no-code agentic analytics service powered by Anthropic’s Claude, with a chat interface, sandboxed code interpreter, artifacts, skills, memory, multi-agent workflows, and MCP-compatible integrations. It also published CostBench, an open benchmark applying each vendor’s real compute billing model to the same workload; ClickHouse Cloud was reported to have 23x better cost-performance than the nearest cloud data warehouse. Other announcements included Managed Postgres in public beta, managed ClickStack, generally available full-text search, cross-region replication, and the House Mates partner program.


### [Amazon Research Awards recipients announced](https://yomu.fyi/post/amazon-research-awards-recipients-announced.md)
- Company: amazon.science
- Author: Amazon Research Awards team
- Published: May 27, 2026
- Type: Announcement

Amazon Research Awards has announced 70 award recipients across 49 universities in 11 countries for its fall 2025 funding cycle. The funded research spans six distinct tracks: AI for Information Security, Agentic AI, Automated Reasoning, AWS Cryptography, Cybersecurity and Anti-Abuse Technologies, and Sustainability. Each selected academic team receives unrestricted financial grants, AWS Promotional Credits to access specialized cloud AI and machine learning services, and access to more than 700 Amazon public datasets. Awardees are paired with an Amazon research contact who provides advisory support, consultation, and access to company training sessions. Furthermore, the program explicitly encourages recipients to publish their discoveries, deliver presentations at Amazon offices worldwide, and release associated software implementations under open-source licenses.


### [Reachy Mini goes fully local](https://yomu.fyi/post/reachy-mini-goes-fully-local.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Amir Mahla, Andres Marafioti
- Published: May 27, 2026
- Type: Tutorial

Reachy Mini’s conversation app can now run its speech stack locally instead of sending audio to a server, avoiding cloud, API keys, and data leaving the machine. The tutorial uses speech-to-speech as a cascaded VAD → STT → LLM → TTS pipeline exposed through a Realtime API-compatible /v1/realtime WebSocket, with Silero VAD v5, Parakeet-TDT 0.6B v3, Qwen3-TTS, and a selectable LLM backend. It shows llama.cpp serving Gemma 4 E4B with parallel slots, 64k context, flash attention, and full sliding-window attention, then covers vLLM, MLX, Transformers, hosted Responses API endpoints, and network connection to a wireless robot. Once the backend and app are connected, conversations run through the local loop, while individual stages can be exchanged to trade latency, quality, language coverage, and hardware choices.


### [Open House 2026 Day 1: Real-time data without lock-in and what teams can build next](https://yomu.fyi/post/open-house-2026-day-1-real-time-data-without-lock-in-and-what-teams-ca.md)
- Company: [Clickhouse](https://yomu.fyi/company/clickhouse.md)
- Published: May 27, 2026
- Type: Announcement

ClickHouse’s Open House 2026 Day 1 announced updates spanning ClickHouse Cloud, Postgres, distributed query execution, observability, AI agents, and lakehouse interoperability. ClickHouse Postgres entered public beta with local NVMe storage, high availability, S3-backed point-in-time recovery, and a native CDC path into ClickHouse; early benchmarks reported more than 5x the transactions per second of AWS RDS. Multi-stage distributed queries repartition intermediate data between stages to scale joins and high-cardinality aggregations, reducing TPC-H SF100 runtime from 117.6 seconds on one node to 54.7 seconds on eight nodes. Additional launches included a Claude-powered ClickHouse Agents public beta, an AI-enhanced query builder, Monitoring v2, and Langfuse V4’s reported 200x query performance improvement, while the wrap-up framed the releases around real-time systems without lock-in.


### [Beyond the Menu Tree: How Yelp Built a Smarter Customer Success Chatbot with AI](https://yomu.fyi/post/beyond-the-menu-tree-how-yelp-built-a-smarter-customer-success-chatbot.md)
- Company: [Yelp](https://yomu.fyi/company/yelp.md)
- Author: Lina Lee, Machine Learning Engineer; Nelson Lee, Engineering Manager
- Published: May 27, 2026
- Type: Problem & solution

Yelp transitioned from a legacy customer support chatbot that relied on rigid phrase matching and static menus to a retrieval-augmented generation system. The updated architecture classifies incoming queries using a large language model to route users into one of five specialized workflows, including QA, Billing, Refund, Cancel, and Review. For general QA, the system uses FAISS to search an in-memory vectorstore built exclusively from article metadata such as titles, summaries, and headers rather than raw article text. Retrieved articles are supplied to the language model as prompt context, followed by automated validations for safety, character limits, and hallucinated hyperlinks. In production A/B testing, this metadata-driven design doubled the chatbot resolution rate while achieving approximately 94% recall@5.


[Newer posts](https://yomu.fyi/topic/llm/page/20.md) · [Older posts](https://yomu.fyi/topic/llm/page/22.md)