Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Building Menu Vision: Real-Time Dish Recognition
Yelp developed Menu Vision to help diners recognize menu items and view corresponding user-generated photos and reviews in real time using their phone cameras. The feature evolved from a two-day hackathon Android prototype into a production system deployed across iOS and Android. Client devices perform all optical character recognition, text cleanup, and dish matching locally using native machine learning frameworks, while dish metadata is prefetched upon opening a restaurant page. Backend pipelines combine partner data, reviews, and photo captions into a Cassandra database, filtering exclusively for dishes with photos. To handle optical character recognition errors and name variations, the matching pipeline runs exact matching, bidirectional substring matching, and Jaro-Winkler similarity matching.
Developing NVIDIA Holoscan Applications with CLI, Skills, and AI Coding Agents
NVIDIA Holoscan is a platform for real-time AI applications at the edge, and HoloHub provides reference applications and components. This post describes an engineer-guided workflow in which a coding agent builds a separate Python endoscopic tool segmentation application from the existing MONAI model, sample video, and Holoscan patterns. The Holoscan CLI, invoked through ./holohub, exposes shared commands for scaffolding, running, modes, tests, and benchmarking, while development skills and documentation guide the agent through project-specific operations. The application connects video replay, preprocessing, TensorRT inference, segmentation postprocessing, telemetry, and HoloViz visualization, with visual, smoke, and benchmark modes; the post reports that the CLI-plus-skills-plus-docs/examples configuration produced a separate application in 40 minutes using 11M tokens, and concludes that this combination offered the lowest resource overhead among the compared configurations.
How we think about text classification in the LLM era
Medium revisited its aging NSFW text-classification model, which had outdated code, infrequent retraining, and poorly documented training and performance history. The replacement uses an LLM prompt containing NSFW guidelines and story content, requesting a 0-to-100 score, structured output, and a short reason rather than requiring a training set. To evaluate it, the team created a curator-labeled dataset with deliberate oversampling of NSFW stories, then compared several LLMs with the legacy model using false-positive and false-negative metrics. Some models matched or exceeded the legacy model; the team selected Mixtral 8×7b based on cost and performance and ran inference in Snowflake. The post presents this as a strong fit where training data is costly, trends are relatively stable, and only distribution-eligible stories need scoring, while topic and spam models still benefit from continuously updated datasets.
Building Federated Multimodal AI Workflows with NVIDIA FLARE
Federating vision-language models (VLMs) across institutions requires keeping images, text, and prompts local while coordinating training and combining updates from sites with different task and modality mixes. NVIDIA FLARE addresses this workflow through server-scheduled rounds, client-local execution, and an explicit update contract, while supporting large-object externalization, pull-based tensor streaming, and disk-backed aggregation for large payloads. The post presents FedUMM as an adapter-based design in which clients train LoRA adapters over a frozen BLIP3o backbone and send only adapter updates, while full-model training remains possible when required. In reported eight-client experiments, adapter-only federation reduced per-client communication from 28.6 GB to 0.094 GB per round, improved VQA v2 by 0.7 points, and retained about 97% of centralized performance on VQA v2 and GenEval; the evaluation used simulated sites and does not establish clinical performance or formal privacy guarantees.
How we knew COVID was over (and what our models had to unlearn)
Airbnb's forecasting team outlines their framework for managing production models that experience drift or external shocks. Rather than treating model updates as a generic retraining task, they distinguish between refitting with new data, respecifying the underlying model structure, and intentionally holding without changes. This discipline prevents teams from chasing temporary noise, carrying obsolete crisis assumptions, or over-engineering solutions during disruptions.
A revisit of remote Spectre attacks on Cloudflare Workers
Cloudflare reassessed remote Spectre attack feasibility within Cloudflare Workers production workloads to evaluate defenses against modern stabilization techniques. The experimental proof of concept bypassed Dynamic Process Isolation (DyPrIs) by keeping invocations alive using WebSocket traffic, which also diluted hardware performance counter ratios below detection thresholds. By combining branch mistraining, cache line eviction, and speculative type confusion, researchers transiently accessed out-of-bounds memory via raw 64-bit TypedArray backing store pointers. This attack demonstrated cross-tenant data leakage reaching up to 12 bit/s with over 99% accuracy in production. In response, Cloudflare mitigated the vulnerability by integrating the V8 memory sandbox, deploying Memory Protection Keys for hardware-assisted in-process isolation, and redesigning DyPrIs to monitor active long-lived executions.
Designing effective Genie Agents from a single prompt
Generic agents may select the first relevant-looking revenue table rather than the finance-maintained source because they lack business context. Genie Agents address this by using a single prompt to assemble a domain-specific agent from governed Unity Catalog context, including structured data, documents, and files. The prompt identifies the desired outcome and relevant sources, while the agent can retrieve content from PDFs, Word documents, presentations, images, and tables under the requesting user’s permissions. The recommended process is to begin with one focused, repetitive job, test it against known questions or incidents, and expand only after measuring performance. Built-in benchmarks provide accuracy scores, while monitoring surfaces user questions and feedback that can reveal gaps in the curated context.
Post-Train NVIDIA Cosmos 3 Edge for On-Device Robot Control
Deploying large world models on robots is constrained by onboard memory and the control latency needed for continuous manipulation. The tutorial post-trains NVIDIA Cosmos 3 Edge, a 4B omni-model with a 2B NVIDIA Nemotron-based reasoner, into a DROID action policy using 76,000 successful teleoperated trajectories, then serves it through an OpenPI-compatible WebSocket server on Jetson Thor. The configuration predicts 32 future 8-dimensional joint-position actions at 15 Hz, while a receding-horizon loop executes a prefix and replans from fresh camera and robot-state observations. On an NVIDIA Jetson AGX Thor T5000, inference takes about 1.53 seconds per action chunk covering roughly 2.13 seconds of motion, and the policy reaches 22.9% success in closed-loop RoboLab evaluation; the source contrasts this with Cosmos 3 Nano’s 36.8%.
Evaluating AI Agent Skill Performance with NVIDIA SkillEvaluator
NVIDIA SkillEvaluator measures whether packaged agent skills improve agent performance by comparing task runs with and without a skill installed. It combines three evaluation tiers: static safety and structure checks, embedding-based distinctiveness analysis, and live evaluations in isolated Harbor sandboxes. In the August 12, 2026 benchmark snapshot, more than 300 verified skills spanning over 30 NVIDIA products were tested across Claude Code and Codex, with macro-averaged Skill Lift of +31 points across all dimensions and +39 excluding Security. Correctness rose from 46 to 87 and Effectiveness from 39 to 78, while Security moved from 97 to 98; token and runtime effects varied by skill, with one reducing both and another increasing them.
How Clario technology detects PHI/PII in DICOM images using Amazon Bedrock
Clario developed an automated PHI and PII detection solution on AWS to process thousands of DICOM image slices and PDF files in clinical trials. Medical imaging workflows require strict privacy compliance under HIPAA, GDPR, and ICH E6 guidelines across standard tags, custom vendor metadata, and pixel-burned text. To address this, the pipeline retrieves files from Amazon S3 and runs memory-intensive detection workloads on Amazon EKS backed by Amazon RDS for PostgreSQL. Amazon Textract handles optical character recognition, while Anthropic's Claude Sonnet on Amazon Bedrock classifies sensitive data using domain-optimized prompt engineering. The system identifies precise coordinates and sensitive entity types rather than directly altering files, enabling human-in-the-loop review before downstream redaction occurs.
AI-powered clinical trial eligibility and safety using Amazon Bedrock AgentCore
Manual chart review across fragmented clinical sources slows clinical trial enrollment, contributing to timeline delays and high screen failure rates. To address this bottleneck, an architecture on AWS automates eligibility and safety assessments while preserving human clinician authority. AWS HealthLake ingests and normalizes records into FHIR R4 resources, while Amazon Bedrock AgentCore orchestrates pre-screening, detailed screening, and site enrollment agents. Amazon Bedrock AgentCore Evaluations scores screening determinations using an LLM-as-a-judge approach for clinical accuracy, operational effectiveness, and safety compliance. This human-in-the-loop workflow produces structured recommendations with source citations, reducing patient matching time from days to minutes while maintaining compliance audit trails.
Accelerate automation with AI and the Ansible development tools MCP servers
Ansible automation development often involves frequent context switching and struggles with AI assistants generating syntactically valid code that hallucinates module parameters or misses local dependencies. To address this gap, the Ansible development tools Model Context Protocol server connects AI assistants to external development tooling through structured interfaces. The server wraps command-line utilities, enabling AI models to deterministically run ansible-lint with automatic fixes, scaffold projects via ansible-creator, manage environments, and build execution environment container images. By surfacing local workspace schemas, documentation, and organizational linting standards, the system grounds AI responses in real environment configurations. The integration operates across multiple MCP-compatible clients without requiring a specific model provider.
Building E-commerce AI Agents on MongoDB with CrewAI
A 34% year-over-year increase in consumers using generative AI for online shopping, especially among Gen Z and Millennials, is motivating e-commerce teams to adopt AI shopping agents. This tutorial demonstrates building a chat-based shopping assistant on MongoDB Atlas with CrewAI that supports zero-click buying from product discovery to post-purchase follow-up. The agent interprets natural-language queries, retrieves semantically relevant products via MongoDB Vector Search with Voyage AI embeddings using price and inventory pre-filters before similarity ranking, and uses Change Streams for atomic inventory checkout. Atlas Triggers enable proactive order notifications, while Atlas Stream Processing detects delivery patterns by monitoring updates in two-minute tumbling windows, writing confirmed signals to agent_triggers when at least 10 orders per carrier-region are delayed by 45 minutes. MongoDB consolidates operational data, semantic search, and event-driven capabilities into a single platform, reducing latency and complexity.
The Merchant Identity Gap: The Hidden Payment Data Problem
Banks face a merchant identity gap: transaction descriptors from acquirers (e.g., SBUX) often don’t match customer search terms (e.g., “Starbucks” or “星巴克”) due to script differences, colloquial aliases, and cross-subsidiary naming inconsistencies. This gap inflates support costs, degrades spend categorization, and undermines personalized financial management and AI advisors. The solution uses a semantic resolution architecture on MongoDB storing a master merchant catalog with canonical names, aliases, and vector embeddings. An escalating pipeline applies fuzzy matching, semantic vector search with Voyage AI embeddings and MongoDB Vector Search, and gen AI write-back to resolve unknown descriptors and enrich the catalog. With canonical identities indexed, customers search in their own language, and institutions power conversational AI that accurately answers queries like “How much did I spend at Starbucks last month?” across markets.
Cloud Agents and Cursor Harness Improvements
Cursor’s release expands cloud agents and the Cursor harness for longer-running, event-driven software work. Cloud agents can subscribe to pull requests, Slack threads, or scheduled tasks, wake on events, keep goals until completion, and automatically address CI failures and bot comments on pull requests they create. Custom modes pin a selected skill in chat, while /goal supplies a long-lived objective that can be paired with a custom mode or /loop. Subagents can run on separate virtual machines with isolated project copies and clean context, supporting fresh testing or parallel fixes without collisions. Steering improvements let users send follow-ups that wait for the next tool call instead of interrupting an action.
Welcoming Laura Beatris, our new Software Engineer
Resend announces that Laura Beatris is joining its team as a Software Engineer. She previously worked at Clerk and WorkOS, where she shipped authentication products with an emphasis on developer experience, and she has spoken on React, TypeScript, and learning in public. Laura describes a path that began with learning Python from free tutorials, continued through a technical degree and early project-building after repeated internship rejections, and eventually led to Amsterdam in 2021. At Resend, she says she was drawn to products where details matter, a culture of learning in public, and a new problem space for improving her developer-experience judgment; her advice is to learn, create, then teach, because teaching helps knowledge stick.
How to Build Passwordless Auth With Twilio Verify in Rust
Passwordless authentication replaces passwords with one-time codes delivered by SMS, and this tutorial builds a Rust app around that flow. The app uses Twilio Verify for OTP generation and validation, with GET and POST routes at "/" for collecting a phone number and at "/verify" for entering the received code. It sets up a Cargo project with Axum, Handlebars templates, sessions, dotenvy configuration, and the rustlio Twilio client, while environment variables hold the Account SID, Auth Token, and Verify Service SID. The implementation also serves HTML forms and static assets, with verification status rendered as a success or error message. Running cargo run starts the server on port 8080, where users can submit a number, receive an SMS, and see whether the verification code is valid.
How to Build Passwordless Auth With Twilio Verify in PHP
Passwordless authentication with PHP and Twilio Verify replaces password entry with a phone-based one-time passcode flow, intended to reduce password-related risk and simplify sign-in. The tutorial uses a Slim base project, Composer dependencies, Dotenv configuration, a Twilio Verify Service SID, and PHP session state to support the flow. Users submit a phone number through the root route; the server asks Twilio Verify to send an SMS OTP, redirects to /verify, and checks the submitted code through the verification endpoint. The example’s verification form restricts input to a six-digit numeric value, and the application reports whether verification succeeded or failed after testing locally with composer serve on port 8080.
How to Build Passwordless Auth With Twilio Verify in Go
The tutorial builds a passwordless authentication flow in Go that uses Twilio Verify to send and validate one-time passwords (OTPs) by SMS, avoiding the need for users to manage passwords. It defines a small net/http API with GET and POST handlers for "/" and "/verify": the first collects a mobile number and starts verification, while the second accepts the received code and displays the result. The setup covers Go 1.23 or later, module dependencies, dotenv configuration for the Twilio Account SID, Auth Token, and Verify Service SID, plus session handling for the phone number. The verification check calls Twilio's CreateVerificationCheck() and renders success or failure using templates; running go run main.go starts the server on port 8080 for testing.
Gisting: Compressing LLM Agent context to ↑ throughput and ↓ cost
Long system prompts improve agent behavior but increase inference time, cost, and the GPU capacity needed to serve a given traffic level. Shopify compresses the Sidekick GraphQL agent’s roughly 6,000-token system prompt into about 1,500 gist tokens, using special vocabulary tokens whose embeddings are learned by knowledge distillation while the model weights remain frozen. Training compares teacher logits from the full prompt with student logits from the gist-token prompt using KL divergence; at inference, the prompt is replaced with the gist-token string, requiring no custom serving path. In load tests at 350 requests per minute, compression reduced median time to first token from 438ms to 354ms and end-to-end latency from 6.8s to 4.2s, while raising throughput from 20.2 to 23.4 QPS and enabling 14% fewer GPUs in production without losing prediction quality.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.