Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Agent Memory Inside the Harness
The post distinguishes agent state, the run-bound working data of a single execution, from agent memory, knowledge that outlives the run and is read across sessions. Context compaction, which decides what to keep when the window fills, is lossy by construction and sets both token cost and reliability; Chroma’s study found models become unreliable well before their advertised 200k-token window, and teams compact at 95% or proactively at 60%. System prompts and skills must be versioned, governed infrastructure, not text files, with prompt registries and the SKILL.md standard for procedural memory, and skills are executable so they require sandboxing. Enterprise data enters memory through pipelines like change data capture, not migrations, with chunking, embedding, and provenance stamps. The database choice caps filtered recall under selective scoping, the consistency window, and temporal modeling, setting the ceiling for later decisions.
DeepSeek V4 Pro 0813 vs GPT-5.6 Sol on DeepSWE: Cost, Coding, and Routing
DeepSeek V4 Pro 0813 costs $0.24 per rollout and achieves 62.8% pass@1 on DeepSWE, while GPT-5.6 Sol reaches 72.7% pass@1 at $8.37 per rollout—a 35x price gap. At four attempts, DeepSeek V4 Pro 0813 surpasses Sol, with 88.5% pass@4 versus 85.8%. Sol is faster and more reliable, completing a rollout in 17 minutes and 53 steps, and breaking tests in only 11% of its failures. A cascade strategy—running DeepSeek V4 Pro 0813 first and escalating to Sol on test failure—solves 83.0% of tasks at $3.35 each, outperforming either model alone. Per $100, DeepSeek V4 Pro 0813 solves 260 tasks versus Sol’s 9, making it the value leader for budget-constrained workloads.
Hello, Ireland! Sending SMS in Twilio’s Ireland Region
Twilio’s Ireland region, IE1, provides EU data residency for Programmable SMS, keeping personal data such as recipient phone numbers and message bodies in the EU through the supplier edge. Adopting it requires separate IE1 credentials and an IE1 Messaging Service, while US1 Messaging Services cannot be reused; SMS to and from +1 numbers, RCS, and WhatsApp are not currently supported. For API calls, the main code change is replacing api.twilio.com with api.dublin.ie1.twilio.com, while client SDK users switch the region parameter from us1 to ie1 and can retain the Dublin edge. Twilio’s logs include a region filter to verify where messages were processed, and the post notes that Studio and TaskRouter support in IE1 is still an initiative.
Add an AI Voice Assistant to a Twilio Video Room with Conversation Relay
The tutorial addresses wasted patient time in telehealth waiting rooms by adding an AI voice assistant to a Twilio Video room for intake collection. It combines Twilio Programmable Video, Conversation Relay, and an LLM, with the assistant joining as an audio-only participant and handling questions about visit reasons, symptoms, and medications. A two-leg Voice call bridges the room to Conversation Relay: one leg participates in the room, while the other sends transcribed audio to a WebSocket server and returns spoken LLM responses. The tutorial uses Node.js, Express, the Twilio helper library, OpenAI, ws, dotenv, and ngrok, and explains setup, browser interaction, and troubleshooting. Its stated result is a working application in which a patient converses naturally with the AI before a provider joins, with GPT-4o-mini processing speech.
How to Build Passwordless Auth Using TOTP With Twilio Verify in PHP
Passwordless authentication can replace passwords with time-based one-time passwords (TOTP), but implementing the setup and verification flow requires coordinating an authenticator app with a verification service. This tutorial builds a web-based PHP application using Twilio Verify, Slim, Twig, and the Twilio PHP Helper Library. Users submit a username, create an initially unverified Factor resource, scan a QR code or enter its setup code, and submit the first six-digit code; later, they enter codes generated by the authenticator app for validation. The project configures Twilio credentials and a Verify Service SID through environment variables, adds session and flash-message support, generates QR codes, defines routes and templates, and runs locally with composer serve on port 8080. The conclusion presents TOTP passwordless authentication as more convenient and potentially more secure than passwords, while acknowledging added application complexity.
Multi-Vector (Late Interaction) Embedding Models with Sentence Transformers
Sentence Transformers v6.0 adds MultiVectorEncoder, bringing ColBERT-style late-interaction retrieval into the library’s existing API for dense, sparse, and reranker models. The encoder retains one projected, typically 128-dimensional vector per token instead of pooling a document into one vector, allowing documents to be encoded independently and indexed offline. At scoring time, MaxSim finds each query token’s highest dot-product match among document tokens and sums those maxima; contextual embeddings support both paraphrase matching and exact-token evidence. The guide covers checkpoint loading, encoding, scoring, semantic search, retrieval and reranking, indexing, token pooling, inference speed, evaluation, and retrieval from page images, audio, and video. It also quantifies the storage trade-off: 4,874 passages produced 608,414 token vectors and a 311.5 MB float32 index, while fast-plaid compression reduced that example to 92 MB.
Device Authorization Flow for a Rust CLI and an Axum API
Command-line tools lack browser redirect capabilities, preventing standard OAuth login flows. To resolve this limitation, the OAuth 2.0 Device Authorization Flow issues a temporary device code, presents a verification URL to the user, and waits for authentication in an external browser. A Rust workspace demonstrates this end-to-end pattern across a CLI, an Axum web API, and a shared authentication library. The shared library relies on jsonwebtoken and reqwest to fetch JSON Web Key Sets and validate token signatures, issuers, and audiences against typed claims. Axum secures protected routes through custom FromRequestParts extractors that decode Bearer headers, enabling verified CLI access.
How Databricks Feature Store serves features with sub-second freshness
Databricks Feature Store addresses the lag between batch-computed machine-learning features and decisions that depend on events from seconds earlier. It lets a feature definition drive both offline batch processing and online streaming pipelines, orchestrating Kafka ingestion, Spark Real-Time Mode (RTM), Lakebase, and Model Serving. For a rolling 10-minute transaction sum, RTM updates per entity state in local RocksDB, expires contributions per event, and writes revised values to Lakebase through a streaming JDBC sink. The described path reports end-to-end p99 200ms latency from Kafka arrival to online feature availability, while Model Serving retrieves declared dependencies automatically at inference. Tumbling and sliding windows remain options when fewer, cheaper updates are acceptable.
How to Evaluate Production AI Agents: Measure System Outcomes, Not Conversations
Evaluating AI agents solely on conversational transcripts fails when agents must invoke tools to modify external production systems. Language models can convincingly narrate that an action occurred while completely omitting the underlying tool call, leaving database or billing records unchanged. Outcome-based evaluation addresses this divergence by treating external system mutations rather than generated dialogue as the actual source of truth. Frameworks such as CRMAgentBench implement persistent, stateful environments with strict all-or-nothing validation, verifying tool arguments, execution sequence, final record modifications, and the total absence of unauthorized actions. Furthermore, measuring repeated success across independent executions using pass^k metrics exposes operational variance and inconsistency that standard single-shot scores and pass@k metrics conceal.
Same Cluster, 33 Points More Utilization: What Changed Was the Order
The post presents a constraint-aware GPU allocator as a response to utilization losses caused by fixed real-time reservations and FIFO ordering under contention. It compares the allocator with FIFO on identical hardware and workloads across seven benchmark scenarios, modeling batch-like jobs as contiguous GPU blocks and real-time inference as a timestep-by-timestep demand curve with bounded GPU swaps. Across five contention scenarios, utilization improved from 52–85% to 72–88%, while priority-weighted value increased 24.6–105.1%; the strongest case gained 33 percentage points of utilization and 105% value, and the scale test gained 15.9% value at identical utilization and throughput. The scheduler optimizes a 24-hour horizon but commits only the current timestep, re-running every 30–60 minutes so fresh observations absorb forecast error while preserving running work.
The prototyping tax is killing your AI roadmap
The post argues that the “prototyping tax”—delay between an AI-powered idea and a working MVP—causes initiatives to lose momentum, often because organizational context is fragmented across teams, tools, and codebases. Traditional encapsulation and siloed domain knowledge can leave agents with syntax and contracts but not business intent, making production codebases harder to navigate than personal projects. The proposed response is agentic development grounded in business semantics and governed data, with alignment occurring through working builds while CI/CD, code review, and production rigor remain unchanged. The post recommends tracking time-to-prototype, first-pass acceptance rate, and PoC-to-production rate to distinguish faster demos from durable delivery. It cites a benchmark of 401 data tasks and Abacus Insights results: 77% accuracy at roughly half the cost for a platform-native agent, plus about 40% less manual effort and onboarding to first value in roughly half the time.
Developing Nemotron 3.5 Lightning NVFP4 with QAD Using NVIDIA Model Optimizer
NVIDIA’s post presents a workflow for developing an NVFP4 checkpoint of the Nemotron 3.5 Lightning 30B-A3B model, targeting lower memory use and higher inference throughput without sacrificing accuracy. The method starts with post-training quantization (PTQ), producing a W4A16 student from the BF16 teacher, then applies quantization-aware distillation (QAD), where simulated quantization and KL-divergence loss align student logits with the frozen teacher. For the selected four_over_six recipe, 32K calibration performed best, and the recipe using W4A16 Mamba linears offered the best reported trade-off between accuracy degradation and inference performance; the checkpoint is 22 GB versus 66 GB and delivers up to 4x faster throughput. The workflow includes Model Optimizer examples and launchers for Megatron-LM and Megatron-Bridge, covering calibration, distillation, evaluation, and export.
How Palo Alto Networks Built a Zero
Palo Alto Networks built a new SASE Health Portal to provide real-time service health visibility for its enterprise customers. The previous architecture relied on relational databases and cloud data warehousing, which caused schema migration bottlenecks and regional lock-in. To solve this, the team deployed containerized services across multiple cloud zones in an active-active configuration. They used MongoDB Atlas as a globally replicated database, enabling automated cross-region failover and flexible document modeling for customer subscriptions. The new portal eliminates alert fatigue by allowing granular subscriptions and achieves zero relational joins for fast incident matching.
Origin Code Hosting
Cursor is rolling out Origin, an early-beta code-hosting service for all paid plans, except enterprise organizations whose admins opt out, with repositories, pull requests, code browsing, and GitHub synchronization. Origin-hosted repositories treat Origin as the source of truth: users push through the CLI, while GitHub is not involved; GitHub-synced repositories instead keep GitHub authoritative and mirror changes to Origin in real time. Each repository includes pull-request timelines, commits, checks, file diffs, comments, reviews, and merging, with synced pull requests updating between Cursor and GitHub within seconds. Cursor also places agents alongside code and PRs, allowing them to answer questions, make changes, update pull requests, or push branches, while Vercel, Depot, and Buildkite integrations add preview deployments and CI options.
DeepSeek V4 Pro 0813 vs Claude Fable 5 on DeepSWE: Cost, Coding, and Routing
The DeepSWE benchmark comparison between DeepSeek V4 Pro 0813 and Claude Fable 5 reveals a trade-off between cost and accuracy. Claude Fable 5 leads on the first attempt with 69.7% pass@1 versus DeepSeek V4 Pro’s 62.8%, but costs 90 times more per rollout ($21.63 vs $0.24). Under repeated attempts, DeepSeek V4 Pro surpasses Fable, achieving 88.5% pass@4 compared to Fable’s 84.1%. The two models show unusually low per-task correlation (0.39), meaning they fail on different tasks. A cascade strategy that runs DeepSeek V4 Pro first and escalates to Claude Fable 5 only on failures solves 82.7% of tasks at $8.28 per task, outperforming either model used alone.
A/B test models in production
A/B testing model changes in production by sending fixed shares of live endpoint traffic to one control and up to 20 variants, allowing teams to compare user-facing outcomes rather than relying only on shadow-traffic checks. The endpoint router keeps variants out of the base traffic split, resamples requests that reach the control, and treats experiment percentages as absolute shares independent of replica counts or autoscaling. A 95/5 experiment can be updated to 80/20 and then 50/50 by resending the complete member list; an etag rejects stale concurrent updates. Measurement joins per-deployment platform metrics with product signals using the deployment ID in response metadata. To finish, a winning variant is promoted through a blue-green rollout before deleting the experiment, while deletion alone returns traffic to the control and can unwind a losing test.
Build zero-trust AI agents with Google's Agent Development Kit
Connecting an autonomous Agent Development Kit (ADK) agent to databases, internal APIs, and code execution turns model decisions into production-state changes that system prompts cannot securely constrain. The post presents an open-source Customer Support & Returns Agent built with ADK and Gemini, using an example where prompt injection could produce an unauthorized refund or leak environment secrets. Its zero-trust design combines hardware-backed signing for database mutations, gVisor isolation with no network egress and resource limits for generated Python, and deterministic semantic gateways for prompts, tool calls, and database updates. Local equivalents use HMAC keys, a database ingress verifier, a constrained Docker runner, and Python validation tests, while production mapping uses Cloud KMS, HSM, and VPC Service Controls. The conclusion is that infrastructure-enforced boundaries let the model reason dynamically within strict limits.
Auth0 Learning: The New Skill-Building Platform for Developers
Following the February 2025 launch of Okta Learning, user feedback revealed that mixing Auth0 materials into the shared catalog created friction, buried relevant content, and obscured certification progression. In response, Auth0 launched Auth0 Learning, a dedicated platform tailored specifically to developer workflows. The restructured catalog focuses on three distinct product domains: Auth0 as an identity provider, Auth0 for AI Agents (A4AA), and Auth0 Fine-Grained Authorization (FGA). To reduce onboarding friction, developers can access and interact with free, on-demand courses without creating an account or logging in. Users who choose to authenticate gain access to redesigned learning paths that track individual progress and award shareable digital skill badges upon completion.
Introducing Husker Share, the University of Nebraska-Lincoln’s latest program with Uber
The University of Nebraska-Lincoln (UNL) has partnered with Uber to launch Husker Share, a campus-exclusive shared-rides program for students, faculty, and staff. Powered by Uber’s Closed Loop Share feature, the program matches verified UNL riders traveling in the same direction, supports sharing with up to two other riders, and limits each booking to one person. Eligible riders can save up to 20% on standard UberX fares, while the first 10,000 people to enroll receive $5 off their first trip. Enrollment requires an Uber account, the Uber app, and UNL credential verification; once approved, the option appears in the app, with trips required to start and end inside the designated service zone.
Using AI_Functions in Your Data Warehouse: Top Use Cases
Databricks AI Functions bring model inference into SQL queries, addressing the need to combine warehouse-resident structured data with unstructured reviews, tickets, PDFs, and transcripts without exporting rows to a separate service. The post presents six Lakehouse use cases: document parsing and extraction, sentiment classification, translation, ticket routing, sales-call extraction, and generative drafting with ai_query. Task-specific functions such as ai_parse_document, ai_extract, ai_classify, and ai_translate handle specialized work, while ai_query sends prompts to an accessible Databricks-hosted Foundation Model serving endpoint. Unity Catalog permissions, existing pipelines, and system.billing.usage keep governance and billing within Databricks, while SQL execution manages planning, parallelization, and retries. Production guidance includes tagging jobs, sampling at least 10,000 rows, choosing models deliberately, requesting structured output, and versioning prompts.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.