Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Your sandbox can now serve requests
Sandboxes have traditionally been outbound-only machines, unable to receive inbound connections because they sit behind NAT with no public IP or open ports. BoxCompute now enables sandboxes to serve requests by using Tailcat, an open-source library from Tailscale that performs NAT traversal. Tailcat establishes a direct peer-to-peer WireGuard-encrypted path between the client and sandbox, using a DERP relay only for the initial handshake. This approach avoids the latency and bandwidth bottlenecks of relay-based tools like ngrok or Cloudflare Tunnel, and keeps the relay out of the data path. The feature is rolling out as an early preview with scoped, short-lived grants for selected TCP ports.
ReadyOn’s Four Walls of tenant isolation on Amazon EKS
ReadyOn describes a four-layer tenant-isolation architecture for Harmony, its multi-tenant workforce-intelligence platform on Amazon EKS, which handles payroll data, organizational hierarchies, and operational analytics. The design combines dedicated Kubernetes namespaces, Karpenter-managed tenant node pools, per-tenant Amazon VPC security groups, and dedicated Amazon Aurora databases, with RBAC, admission control, default-deny network policies, IAM Roles for Service Accounts, and tenant-scoped secrets. GitOps through Argo CD ApplicationSets provisions identical controls and continuously reverts drift, while dual taints and admission validation prevent cross-tenant scheduling. ReadyOn says adversarial exercises have blocked all six tested cross-tenant paths to date, and reports defenses across 10 threat-model stages, though shared control-plane dependencies make the layers defense in depth rather than fully independent probabilities.
Partnering with Accenture on embedded evaluation
Anthropic is partnering with Accenture’s specialist AI business, Faculty, on independent evaluation of frontier AI, advancing a commitment to embed evaluators within Anthropic. The work will cover model evaluation and red-teaming, alignment assessments, and testing safeguards, informed by Accenture’s experience deploying AI across industries. Anthropic and Accenture each expect to invest at least $1 billion in building capacity over five years, although operating details, access standards, reporting practices, and funding models remain unsettled. Embedded evaluators would work inside AI companies with employee-comparable access to observe training, follow development and deployment decisions, speak with staff, verify safety commitments, identify blind spots, and report incidents; Anthropic says its partnership is non-exclusive and that model safety remains its responsibility.
Benchmarking LLM Inference at Scale with AIPerf
AIPerf is NVIDIA’s successor to GenAI-Perf, intended to benchmark large-scale LLM inference without letting the load client become the bottleneck. Its multiprocess architecture combines worker processes, separate record-processor services, and ZMQ coordination, while supporting 15+ endpoint types, public datasets, and trace-replay formats. The walkthrough serves Qwen3-0.6B through vLLM, using streaming synthetic requests with fixed 128-token inputs and outputs; AIPerf reports TTFT, ITL, request latency, output token throughput, percentile distributions, and optional GPU telemetry in CSV and JSON. A second profile uses Poisson arrivals averaging 10 requests per second, variable input and output lengths, and a fixed random seed, producing wider latency distributions than the static baseline as concurrency and workload variability increase.
Enabling secure, productive work on personal devices
Personal devices let employees use Slack, approve requests, and access internal apps, but they combine corporate data with private content that Databricks cannot inspect or control. The proposed mobile security model layers Mobile Device Management (MDM), identity and context-aware authentication, Zero Trust Network Access (ZTNA), and application management. On iOS, Account-Driven User Enrollment creates an encrypted workspace without full device management, while Android Work Profile provides comparable separation; per-app VPN keeps work traffic apart from personal traffic. Access requires strong authentication, a managed device, and a trusted tunnel, with continuous posture checks able to block unhealthy devices. Managed apps, configurations, tenant controls, and sensitivity tiers contain data where supported, while transparency about privacy encourages enrollment and supports mobile apps such as Genie.
Saving another 100TB of RAM with math (and Rust)
Cloudflare describes how excessive memory use in pingora-ketama, the consistent-hashing component of its Pingora Backend Router, was investigated as a resource-efficiency problem at global scale. Consistent hashing maps servers and cache keys onto a circular hash space, but a single hash per server creates uneven ranges, while multiple hashes improve balance; with 160 points per server, the coefficient of variation falls to about 8% in the 100-server example. The work produced a v2 ring with compacted storage, faster sorting, and a configurable base number of hashes per node, while retaining the existing v1 ring and supporting simultaneous use during migration. After the old ring was decommissioned, measured memory usage for PBR fell by 100TB globally, and the changes are available in pingora-ketama behind an unadvertised Cargo feature.
How Notion handles concurrent editing with CRDTs
Notion replaced its text-editing system because its former last-write-wins storage could overwrite concurrent edits to the same block, with Offline Mode increasing the risk of lost changes. The new system uses a Conflict-free Replicated Data Type based on Replicated Growable Array, representing inserted text as uniquely identified items, retaining deletions as tombstones, and deterministically ordering concurrent inserts. Peritext-based annotation operations merge rich-text formatting such as bold, italics, and hyperlinks while preserving extendability rules. To keep edits correctly positioned when Enter splits a block, Notion introduced text slices, text slice trees, text instances, and compact search labels for locating slices across blocks. Deployed in July 2025, the system processes millions of CRDT operations per minute and supports Offline Mode and agent collaboration.
How CSIRO built scalable, cost-optimized genomic variant querying on AWS
CSIRO built Serverless Beacon (sBeacon), a production-ready implementation of the Global Alliance for Genomics and Health (GA4GH) Beacon protocol for secure genomic variant querying. Its serverless architecture combines Amazon S3, AWS Lambda, Amazon DynamoDB, and Amazon Athena: genomic data remains in its original S3 location, while metadata is indexed in ORC tables and ontology indexes for query processing. Standard VCF files can be queried through API Gateway, with modular Lambda functions resolving ontology terms, querying Athena, and, when needed, issuing HTTP byte-range requests to source buckets so raw VCF bytes do not leave their owning organization. The post reports support for hundreds of millions of individuals and billions of genomic locations, approximately USD 0.40 monthly operation for a 1000 Genomes-scale dataset, roughly five-second real-world queries, and about 18-second generation-to-use onboarding, while noting that whole-genome range queries can time out.
The AIDEs Framework: How We Built a “Theory of Everything” for AI Development Tools
JetBrains presents the AIDEs Framework as a model for understanding how AI development tools evolve, rather than trying to track every market change. The post describes a progression from single-line completion through multiline completion, IDE chat, AI code editors, agentic development environments, and cloud-based agents. It defines an Artificial Intelligent System as a system that demonstrates intelligence in helping users achieve goals, and an Artificial Intelligent Development Environment (AIDE) as an AIS for creating software; intelligence may come from deterministic heuristics as well as LLMs. The framework treats principals and agents as potentially human or AI actors and views software creation as a broader delegation process than coding alone. It proposes examining assumptions, building a model, and validating it against ongoing change, while suggesting a possible future of vertically integrated Agentic Software Production Platforms that use AI as an “intelligence” resource.
Making Local AI Smarter and Faster
Junie Local’s update introduces Qwen3.8-3.6-27B-blend, a merged model that balances speed and capability for local coding agents. The blend, created by merging Qwen3.6 and Qwen3.8 weights in equal proportions, completed 37 tasks in a 100-task internal benchmark, compared to 34 for Qwen3.6 and 39 for Qwen3.8, while generating 71% fewer output tokens than Qwen3.8. The runtime uses multi-token prediction (MTP), with a two-token proposal on M5 MacBook Pro making decoding 60% faster. A four-bit MTP head was kept over an eight-bit one due to similar acceptance rates and memory savings. The update also brings experimental NVIDIA RTX support on Windows for machines with at least 24 GB of VRAM.
Red Hat OpenShift networking: Default OVN-Kubernetes vs. Cilium operator
The article compares Red Hat OpenShift’s default OVN-Kubernetes CNI with the Cilium operator for teams managing microservices, AI workloads, and declarative multi-cluster infrastructure with Cluster API (CAPI). It describes OVN-Kubernetes as natively managed by the OpenShift Cluster Network Operator, with Open vSwitch foundations, hardware offloading, Multus and OpenShift Virtualization compatibility, and OVN Interconnect for large-scale deployments. Cilium uses kernel-level eBPF programs and offers identity-based policies, sidecarless service mesh capabilities, and Hubble Layer 7 observability, while CAPI manifests and MachineSets support declarative worker-node provisioning. The source recommends Cilium for new clusters when eBPF capabilities and CAPI integration justify added operational complexity, while positioning OVN-Kubernetes as the simpler choice for integrated lifecycle management and hardware acceleration.
Matching OpenShift edge topologies to your physical footprint
Red Hat OpenShift topologies are mapped to edge-site constraints such as hardware footprint, availability, rack space, power, and workload needs. It is presented as a micro cloud that brings full Kubernetes functionality to server-class edge hardware, with optional virtualization and software-defined storage. Cluster capabilities can be minimized with baselineCapabilitySet: none and selectively enabled later, although added capabilities cannot be removed without reinstalling. Options span single-node OpenShift, which requires 4 vCPUs, 16 GB RAM, and 120 GB storage, two-node designs using fencing or an arbiter, and three-node compact clusters for resilient high availability. Each topology retains shared management and security practices, so selection depends on the physical footprint, failure tolerance, and scale required at the site.
What did AI cost you this quarter?
AI infrastructure costs are often billed to a platform team as one lump sum, leaving Finance without a departmental breakdown and teams without feedback on their usage. The post presents a working demo on Red Hat OpenShift AI that attributes requests and token consumption through department-specific MaaS API keys, shared model servers, Prometheus metrics, Perses dashboards, and MLflow traces. In its example, three departments generate 79,420 inference requests and consume 634 million tokens against a $14,200 quarterly bill, while Marketing’s 32% GPU utilization reveals roughly $34,000 in annual waste. The system combines aggregate usage filters with request-level details such as prompts, token counts, latency, and calculated cost, and the post argues that showback helps identify inefficient allocations and model mismatches rather than police teams.
Your diff has a demo now with Junie /demo
Junie /demo adds a mode to Junie CLI that builds and launches an application, drives its UI, and records a requested check. It produces an HTML report, screenshots, and video, while leaving reviewers able to inspect steps, outcomes, and evidence. In the example, a prompt selects two issues, marks them Done, checks counter changes, reloads the page, and verifies persistence. Initial setup analyzes the repository and generates configuration after a confirmed build and launch plan; Docker-based environments can use custom templates for complex repositories. The mode also supports GitHub Actions workflows for pull-request demonstrations and release smoke tests, but an explicit PASS verdict is required for a scenario to pass.
Ktor 3.6.0 Is Now Available!
Ktor 3.6.0 introduces experimental typed authentication with full type safety, including role-based access and anonymous users, plus a dedicated OpenID Connect plugin that supports sessions and auto-refreshing tokens, and experimental HTTP/3 support for the Netty engine via QUIC. Request-parameter conversion now supports Kotlin’s Uuid, Byte, and unsigned numeric types, while receive() accepts nullable types, deprecating receiveNullable(), and more flexible routing syntax. The ContentNegotiation plugin gains a ContentTypeMergeStrategy.SkipIfPresent to preserve explicit Accept headers. Kotlin Multiplatform projects benefit from the ktor-client-engine-defaults artifact and file-based HttpCache storage using kotlinx-io Path, simplifying cross-platform configuration. Additional improvements include respondHtmlPartial for partial HTML output, asynchronous DNS resolution for the CIO engine, and WebRTC support for JVM.
Classic Engine Feature EOL: SSR, Desktop SSO, Device Trust
Okta will end support for Self-Service Registration, Desktop Single Sign-On using Integrated Windows Authentication (IWA), and Device Trust on Classic Engine on March 5, 2027. After that deadline, the capabilities will be turned off unless organizations upgrade to Okta Identity Engine, and Okta will no longer address related bugs, security vulnerabilities, or broken flows. The migration paths are Profile Enrollment Policy for self-service registration, Agentless Desktop Single Sign-On or Okta FastPass for desktop authentication, and management attestation with Okta Verify for mobile device trust. Identity Engine adds passwordless authentication, device assurance, phishing-resistant multifactor authentication, identity verification, zero-trust capabilities, and a newer policy framework, with the upgrade included at no additional licensing cost. Organizations are advised to assess feature usage, test in non-production, and schedule production migration before the hard deadline.
Modernizing the Trade Lifecycle With Governed Data and AI
Capital-markets firms are revisiting the trade lifecycle as data volumes grow, AI moves toward production, settlement cycles shorten, and scrutiny increases. Fragmented data across research, trading, risk, operations, and surveillance can slow decision-making, complicate execution analysis, limit risk visibility, and hinder reconstruction of events. The proposed response starts with decisions such as execution-cost divergence, shock impacts, and exception or settlement-fail rates, then identifies the data domains needed for a few high-value workflows rather than migrating everything at once. It emphasizes a governed, reusable foundation connecting real-time and historical data, analytics, and AI, with access controls, lineage, traceability, and evaluation. Databricks components named include Unity Catalog for governance, discovery, and lineage, Agent Bricks for domain-specific agents, and AI/BI for governed exploration; the stated conclusion is that this operating-model approach can support faster decisions, resilience, and scalable use cases.
Database for AI Agents: 5 Evaluation Criteria
The post defines a database for AI agents as infrastructure for short-term, episodic, procedural, and operational memory, rather than merely a place to retrieve documents. It proposes five evaluation criteria for moving agent workloads from prototypes to production: per-agent branch isolation, serverless scale-to-zero, hybrid search, ACID guarantees, and unified platform access. The technical rationale is that agents perform repeated reads and writes, resume tasks, share state concurrently, and need current data across vector, keyword, and metadata retrieval; zero-copy branches, sub-second wakeups, and single-query hybrid search address specific operational constraints. It concludes that a database missing more than one minimum bar becomes a production risk, while standard Postgres alone supplies ACID but not branching, scale-to-zero, or unified operational and analytical access.
Building a knowledge assistant over code
Building a knowledge assistant over code requires more careful chunking than indexing ordinary business documents, because code has nested structure, cross-file dependencies, and multiple file formats. The comparison built three Databricks Knowledge Assistants over the Casper’s Kitchens repository: fixed-size character chunks, LangChain language-aware splitting, and AST-based splitting with structure and metadata headers. The AST approach used ASTChunk to preserve units such as functions, classes, or statement blocks and prepend file paths plus class/function hierarchies to chunks. MLflow GenAI evaluation ran 46 questions with retrieval sufficiency, groundedness, and custom answer-correctness judges; AST chunking achieved 89% retrieval sufficiency and 70% fully correct answers, versus 85% and 59% for the naive baseline. The advantage was strongest for disambiguating structurally similar code, while value lookups and complete definition retrieval were comparable across methods.
Google Cloud Use Cases: How Enterprises Run Data & AI on GCP with Databricks
The guide presents Google Cloud as a unified platform for enterprise data engineering, analytics, machine learning, and generative AI, with managed services intended to reduce infrastructure and integration work. It focuses on practical architectures, including medallion ETL pipelines that store Bronze, Silver, and Gold Delta Lake tables in Google Cloud Storage and expose Gold data to BigQuery through an optimized connector. For streaming workloads, Cloud Pub/Sub supplies ingestion while Apache Spark Structured Streaming processes continuous micro-batches; together with Delta Lake ACID transactions, this supports exactly-once results despite duplicates or out-of-order messages. The text also describes Cloud Run and Cloud Functions for containerized or event-driven workloads, and identifies retrieval-augmented generation applications on Vertex AI as a common starting point for generative AI. It concludes that integrated service combinations, rather than isolated tools, are the central strategic advantage for organizations building evolving data and AI platforms on Google Cloud.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.