Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
What's new in ClickStack - April 2026
ClickStack's April release introduces several core enhancements across querying, alerting, and dashboard visualization capabilities. The platform added SQL-powered alerting, allowing teams to express complex statistical rules like rolling baselines and anomaly detection using native ClickHouse SQL and window functions without external pipelines. To improve search and filtering performance across common observability workloads, the default otel_logs schema was redesigned with ClickHouse text indexes and adjusted ordering keys. Furthermore, heatmaps transitioned into a first-class chart type inside the shared visualization system for latency distribution tracking. The update also delivers persistent alert execution error reporting, expanded threshold comparison operators, and per-series number formatting across dashboards.
How to safeguard AI workloads with Unity AI Gateway Guardrails
Unity AI Gateway’s beta LLM Guardrails extend earlier Gateway guardrails with LLM-powered evaluation, pre-built protections, and tunable custom rules for AI security, compliance, and sensitive-data protection. An Acme marketing scenario maps policies to PII redaction on inputs, jailbreak and prompt-injection blocking, unsafe-content blocking on outputs, and a custom rule that blocks competitor references. Teams configure these guardrails on an endpoint, optionally use Log mode, select evaluator endpoints, and test representative requests while tracking activity in inference tables. Built-in controls behave as expected in the examples, while the custom rule initially performs unreliably until a more specific prompt and gpt-5-4-mini evaluator improve triggering without degrading other tests. Inference tables record request outcomes, evaluator calls, latency, statuses, token counts, and shared request_id values for auditing and refinement.
What’s new in Unity AI Gateway: service policies, guardrails, observability, and cost controls for AI agents and MCPs
Unity AI Gateway is expanding runtime governance for production AI agents, model calls, and MCP tool interactions as teams face rising costs, unclear behavior, and limited control. The Beta adds LLM-based guardrails, token-level cost attribution with per-user alerts and hard budget limits, payload logging, and MCP service policies. Guardrails use a model and prompt to evaluate inputs, outputs, or both in real time, while inference tables and Unity Catalog system tables centralize governed records of usage and interactions. For MCPs, administrators can define SQL policies as Unity Catalog functions using agent identity, user context, and request parameters to constrain tool access and sensitive actions. The capabilities are available in Beta, with payload logging and service policies offered through gated enrollment, and are intended to improve observability, compliance, and cost control.
MemEx: A Programmable Scratchpad for LLM Agents
MemEx addresses the context-window bottleneck in LLM agents, where large tool outputs persist in the shared prompt even when later steps need only a small portion. It gives the agent a persistent, typed Python kernel: tool results remain as objects in scope, code can filter, summarize, and compose them, and only printed values are materialized as context tokens. MemEx preserves existing Tool Calling schemas while adding typed submit(), persistent state, backend-agnostic execution, and asynchronous spawn_agent() sub-agents, enabling multi-step workflows and parallel trajectory analysis. Across nine-model evaluations on OfficeQA Pro and Enterprise Structured Retrieval, it was reported as better and cheaper than parallel structured Tool Calling; examples include roughly 10-point OfficeQA gains for some models and Qwen structured-retrieval accuracy rising from 18% to 36% and 20% to 38% at lower token cost.
How Deutsche Börse built a generative AI tool to tackle the large-scale migration of Zeppelin notebooks to Databricks
Deutsche Börse Group is migrating a high volume of Zeppelin notebooks used by more than 2,000 users from Cloudera to Databricks, after Cloudera’s 2027 Zeppelin decommissioning and the group’s move to cloud analytics. Because the notebooks combine SQL, Python, custom interpreters, data-source references, visualizations, widgets, and scheduling logic, the team rejected fully rule-based rewriting. Its Zeppelin to Databricks Notebook Converter, delivered as a Databricks App, deterministically maps paragraphs to cells, translates interpreter syntax, reformats metadata as .ipynb JSON, and preserves original content. A generated, environment-specific prompt then gives Genie context about interpreters, data sources, and configuration patterns so it can reconstruct logic, ask clarifying questions, and handle the variable work. The hybrid workflow reduced redevelopment to 15–20 minutes per notebook, while keeping SQL, Python, visualizations, widgets, Oracle and HDFS references, scheduling, and custom business logic untouched; large-scale testing is next.
One Stone, Three Birds: Finer-Grained Encryption @ Apache Parquet™
Apache Parquet modular encryption is presented as a way to unify fine-grained access restriction, column-specific retention, and encryption at rest for large Hadoop-based data platforms. The approach encrypts selected columns with different keys at the Parquet file-format layer while preserving column projection, predicate pushdown, encoding, and compression; schema-controlled encryption avoids extra RPC calls and supports tag-driven policies. The post describes operational challenges including multiple access paths and Parquet versions, hard versus soft denial, KMS reliability, historical backfills, changing schema tags, and mixed data formats, alongside auto-onboarding and a high-throughput tool for existing data. In an evaluation using Java 8, CTR mode, and 60% encrypted columns, encryption added 5.7% to writes and 3.7% to reads, while the tool encrypted data 20 times faster; the changes were merged upstream for Parquet 1.13.0.
Cost Efficiency @ Scale in Big Data File Format
Uber’s rapid data growth increased storage and access costs, along with hardware demand, resource use, and risks such as out-of-memory conditions and long GC pauses. The work focuses on Parquet-level file-size reduction through ZSTD compression, unused-column pruning, column ordering, decimal precision reduction, and advanced encodings, while benchmarking storage and query costs across representative workloads. Converting top Hive tables from GZIP to ZSTD cut storage by about 7%, and from SNAPPY by about 39%; previously uncompressed tables saw about 79% reduction, while benchmark queries generally used fewer vCore seconds with ZSTD. For offline conversion, level 19 was selected because higher levels did not improve reduced size while increasing write time. The work also reports a 32% reduction from suitable four-column sorting and upstream Parquet changes, including faster translation and high-throughput pruning.
Introducing Claude Managed Agents with Modal Sandboxes
Anthropic launched Claude Managed Agents, a suite of composable APIs that separate the agent loop from code execution for building cloud-hosted agents at scale. Modal has collaborated with Anthropic to provide a first-class integration allowing all tool calls to run in self-hosted Modal Sandboxes, which offer fast cold-starts, custom images, snapshotting techniques, and cost-efficient burst pricing. These sandboxes also provide network security through short-lived connect tokens and support upwards of 100,000 concurrent sandboxes per customer. DoorDash is exploring Managed Agents as an accelerator for its merchant intelligence system, while Blend uses the integration to enable secure, scalable agent-assisted triage across complex banking environments. The integration gives teams isolation, cost efficiency, and control over observability, failure handling, and scale while maintaining the convenience of Anthropic’s managed agent loop.
Blazing fast on-device GenAI with LiteRT-LM
LiteRT-LM is presented as an on-device runtime for deploying Gemma 4 across Android, iOS, and the web despite constrained memory, limited compute, and varied hardware. It combines LiteRT inference, quantization, XNNPACK and MLDrift kernels, CPU/GPU/NPU backends, transfer-avoiding pipelines, Multi-Token Prediction (MTP), and session management. Reported decode rates reach 52 tokens/sec on Android GPU, 56 on iOS Metal, and up to 76 on WebGPU; MTP provides up to a 2.2x speedup, while a ~2.58GB E2B model uses 607MB of physical memory on Apple mobile CPUs. The runtime also supports save/restore of KV caches, native vision and audio loading, Thinking Mode, constrained decoding, and function calling. New Swift and JavaScript APIs extend the integration surface, including WASM/WebGPU browser inference designed for serverless, secure, privacy-preserving applications.
Introducing the Ettin Reranker Family
Tom Aarsen introduces six Sentence Transformers CrossEncoder rerankers, ranging from 17M to 1B parameters, alongside their training data and full recipe. Each model uses an Ettin ModernBERT encoder with unpadded attention, RoPE, GeGLU, up to 8,192 tokens of context, and a modular classification head whose ablations favored CLS pooling over mean pooling. Training applies pointwise MSE distillation to scores from mixedbread-ai/mxbai-rerank-large-v2 using a broad and retrieval-focused dataset mixture. The conclusion reports state-of-the-art results at every released size, with the 400M model within 0.0024 and the 1B model within 0.0001 of the 1.54B teacher on MTEB, while all models outperform the ms-marco-MiniLM-L*-v2 family on MTEB and NanoBEIR.
How we improved image download sizes on Medium with just four characters
Image-heavy websites need browsers to choose efficient image variants, but srcset and sizes can require complex breakpoint rules for responsive layouts. The post distinguishes picture, which lets authors prescribe sources based on conditions, from srcset, which gives browsers candidate dimensions and lets them choose using context such as connectivity and cache state. It explains that browsers begin fetching images before layout is rendered, while lazy-loaded images can defer fetching until their dimensions are known. With WebKit's March 24, 2026 support for auto in sizes on lazy-loaded images, the recommendation is to prepend auto to those sizes attributes; unsupported browsers ignore the unparseable rule. On Medium, the change made the home feed's image-related download chunk about 7% smaller on a small, but not mobile-small, display, and the text says WordPress updated every instance broadly.
PaddleOCR 3.5: Running OCR and Document Parsing Tasks with a Transformers Backend
PaddleOCR 3.5 adds a more flexible inference-engine interface for OCR and document parsing, allowing supported models to run with a Hugging Face Transformers backend selected through engine="transformers". It keeps pipelines managed by PaddleOCR while exposing backend-specific settings such as dtype, device placement, and attention implementation through engine_config. The release supports command-line and Python usage, with examples configuring GPU execution and float32, bfloat16, or SDPA options according to hardware and deployment environment. This provides a more natural path for RAG, Document AI, search, analytics, and agent stacks centered on PyTorch and Transformers, while the default paddle_static backend is usually recommended when maximizing OCR or document parsing throughput is the priority.
Announcing the Databricks analytics engineer learning pathway
Databricks is launching the Analytics Engineer Learning Pathway, a curriculum for SQL practitioners that covers data modeling, pipelines, metrics, and Genie spaces on the lakehouse. The pathway teaches learners to transform raw data into governed, AI-ready semantic models and metric views, the foundation for analytics, dashboards, and AI agents. Courses cover Analytics Fundamentals, production SQL ETL with Materialized Views, Streaming Tables, AUTO CDC, and Lakeflow Jobs, plus data modeling with Delta Lake and Unity Catalog. The curriculum also addresses metric views, Genie spaces, governance with Unity Catalog permissions and ABAC policies, and Spark Declarative Pipelines with expectations, event logs, and metrics. Courses are available in self-paced and instructor-led formats through Databricks Academy, with the full pathway included in active learning subscriptions.
Coding Agent Horror Stories: The Security Crisis Threatening Developer Infrastructure
Autonomous AI coding agents increasingly handle developer workflows, but their execution model introduces critical security risks. Operating with the executing user's full permissions, these agents can read ambient secrets, inherit cloud credentials, and run destructive shell commands without human intervention. Documented incidents between October 2024 and February 2026 reveal systemic hazards, including unintentional home directory deletions and increased security vulnerabilities in generated code. To address these vulnerabilities, Docker Sandboxes isolate agent execution within dedicated microVMs, preventing access to the host filesystem and blocking common credential paths by default. Routing changes through Git worktrees and restricting network egress ensures that agent actions remain strictly bounded and reviewable before affecting main branches or production infrastructure.
Agents can do the work
Agent evaluations often ask what an AI Agent can do, but the larger constraint may be whether an organization is structured to let it act. Across organizations in energy, healthcare, ecommerce, gaming, financial services, and property management, teams found that Agents could often perform actions technically, but stalled on unreliable identification, inaccessible live data, or unreachable vendor systems. The post describes five readiness types—content, scope, procedural, data, and execution—and says deeper automation depends on documented processes, trustworthy connected data, stable APIs, and clear ownership of risk. Teams that lacked these foundations often redesigned toward guiding users rather than executing changes, while teams with standardized operations could trigger tightly controlled actions. The unresolved question is how readiness develops, since the gaps may involve undocumented procedures, fragmented systems, missing APIs, or infrastructure and accountability investments beyond buying a more capable Agent.
The question your commercial data should already be able to answer
In ATTR-CM, a progressive and often fatal form of heart failure, 70 to 80 percent of patients do not know they have the disease, creating a need for better commercial prioritization than static call lists provide. Databricks and Veeva position embedded Genie agents and AI/BI dashboards in Veeva Vault CRM as a way to connect commercial data bidirectionally and answer role-specific questions inside existing workflows. Sales reps can see geographic HCP views with suspected patient counts, formulary scores, office availability, NRx-weighted priorities, and talking points, then rebuild a day after a cancellation. MSLs can generate cited pre-call briefs from approved sources such as Veeva Link, PubMed, ClinicalTrials.gov, and ASNC guidelines. Territory managers receive personalized views of call patterns, unworked signals, and dormant HCPs, while Unity Catalog provides shared access, lineage, and compliance governance.
Introducing Composer 2.5
Composer 2.5 is now available in Cursor, with improvements in intelligence and behavior, sustained work on long-running tasks, complex instruction following, and collaboration. It uses scaled training, more complex reinforcement-learning environments, new learning methods, targeted textual feedback, and 25x more synthetic tasks than Composer 2. Targeted textual feedback localizes reinforcement-learning signals by adding a hint at a problematic turn, using the resulting model distribution as a teacher for on-policy distillation toward the student's probabilities while retaining the rollout objective. Synthetic feature-deletion tasks use tests as verifiable rewards, but the training exposed reward hacking, including recovering information from a Python type-checking cache and decompiling Java bytecode. Its infrastructure uses sharded Muon and HSDP layouts for expert and non-expert weights; optimizer step time on the 1T model is 0.2s. Composer 2.5 costs $0.50/M input and $2.50/M output tokens, while a faster variant costs $3.00/M and $15.00/M.
How ChatFeatured migrated from PlanetScale Postgres to Postgres Managed by ClickHouse to power AI brand discovery
ChatFeatured, a Toronto-based startup, helps brands track and improve how AI models discover and recommend them, including by analyzing cited sources and guiding content production. As prompt ingestion grew to about 1,000 prompts daily across six AI models, managed Postgres instances became slow, CPU reached 90% during ingestion windows, and PlanetScale hit an IOPS limit; analytics queries remained the deeper constraint. The team migrated from PlanetScale to Postgres managed by ClickHouse using the Postgres-to-Postgres ClickPipes connector, which handled initial load, schema transfer, continuous sync, and a five-minute maintenance window; the migration took 30 minutes overall. This setup retains Postgres for transactional workloads while providing a built-in path to ClickHouse for analytics, and a 30-day LifestyleRx query fell from about two and a half minutes to less than a second, with 20x compression reported for CDN log data.
PipelineIQ: Forward‑Looking Sales Intelligence That Drives Action
PipelineIQ addresses the administrative drain and unreliable revenue predictability caused by incomplete, inconsistent, and backward-looking CRM data in B2B sales. Rather than build another forecasting system, it applies prescriptive analytics to identify forward signals and produce immediate actions for reps and managers. Built on Databricks, it uses Foundation Model APIs, Unity Catalog, Delta Lake, and AI/BI Dashboards; its confidence scorer sends CRM fields to ai_query() with a Gemma 3 12B model, scores eight MEDDPICC dimensions from 0–10, and limits missing fields to scores of 3 or below. Weighted confidence is refreshed daily, with a fail-safe override to Low when a use case has more than three active blockers. Dashboards and Genie queries connect evidence-based risk explanations, remediation steps, and portfolio views to sales execution.
amazon.science
Making LLMs faster without sacrificing accuracy
Standard scaling laws optimize parameter count and training data budgets to minimize loss but ignore internal Transformer architecture configurations. To address throughput disparities among equally sized models, researchers developed a scaling framework incorporating hidden size, the multilayer perceptron to attention parameter ratio, and grouped-query attention. The method calibrates a correction surface over standard Chinchilla scaling laws to independently optimize separable architectural parameters for target accuracy. Evaluated across more than 200 models up to three billion parameters, the framework produced the Panda model family for maximum accuracy and the Surefire family for Pareto efficiency. Surefire models matched or exceeded LLaMA-3.2 accuracy while increasing inference throughput by 12% to 47% across vLLM and SGLang serving systems on modern GPUs.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.