Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
llm-d flow control: Priority queuing for shared GPU inference
Shared GPU model pools allow platform teams to maximize utilization, but mixed workloads often contend for serving capacity with differing service requirements. Generally available in Red Hat AI Inference 3.5, llm-d flow control introduces explicit admission control prior to backend routing. An Endpoint Picker sits in the request path and evaluates active priority bands from highest to lowest integer values whenever pool saturation is detected. Saturation metrics combine per-replica queue depth and key-value cache pressure to determine when dispatch gates should engage. Once a priority band is admitted, tenant fairness policies distribute dispatch turns, and ordering rules select the next request to forward to a vLLM replica via Envoy.
Beyond the Cloud: Building an Offline
A digital-first economy assumes constant high-speed internet, leaving mobile wallet apps unusable when connectivity drops, such as on subways or in rural areas. A demo wallet solution combines MongoDB Atlas as the cloud source of truth with ObjectBox as an on-device operational store that mirrors the same schema. The ObjectBox Sync Server continuously bridges the two, enabling local reads and writes that sync automatically when connectivity returns. For offline sends, the wallet checks the requested amount against the last-synced balance to prevent overspending, ensuring safe local operations. The architecture treats connectivity as an enhancement rather than a requirement, keeping the wallet responsive in airplane mode with full data access.
The Cheapest Token You Never Send: Agentic AI Search on MongoDB + Voyage AI
The post reframes cost, latency, and answer quality in agentic AI search as a single retrieval precision problem. Its pipeline embeds with flexible Voyage AI dimensionality and automatic quantization in MongoDB index, with binary reducing RAM by roughly 24x. Hybrid search via $rankFusion combines vector and full-text search with metadata filters in one server-side query, eliminating separate systems. A Voyage reranker then reorders the broad candidate set by true relevance, allowing the LLM to receive only the top few high-signal chunks and drastically shrinking input context. The agentic loop over consolidated Atlas where data, vectors, embeddings, and reranking coexist minimizes network hops per iteration, and an illustrative example shows an 80% reduction in retrieved-context input tokens.
Start from scratch, without a repo
Cloud Agents can now start without a connected GitHub or other third-party source-control provider. Selecting Start from scratch in the repo picker lets someone prompt the agent immediately, while Cursor creates an Origin repo in the background. When the build is ready, Create repo saves it as a fully scaffolded Origin repo with a custom or suggested name and private or internal visibility; the Codebase tab provides access to the project. Cursor also port-forwards the cloud agent’s live environment to the browser for previews and design mode, and connecting a Vercel account enables publishing to a live URL for what was built.
Decoding cosmic signals with deep learning and Keras
Astroparticle physics must reconstruct cosmic particles from indirect measurements because high-energy particles create extensive air showers before reaching distributed detectors. Modern observatories such as Pierre Auger, IceCube, and CTAO record spatially distributed sensor data and nanosecond-resolution waveforms, producing image-like spatiotemporal structures at enormous scale. The post explains how deep learning can improve instrument sensitivity, reveal hidden patterns, search for anomalies, and support event reconstruction across cosmic-ray, neutrino, and gamma-ray observations. It also frames fast prototyping of physics models in Keras as a practical entry point, while noting that more data and efficient scaling can sometimes outperform physics-inspired architectures. The resulting tension is between predictive performance from machine-learned black boxes and the physical insight, causal understanding, and interpretability scientists seek.
How to Approve Real Users and Block Fake Accounts at Sign Up with Lookup and Verify in Python
This Python tutorial builds a signup flow that screens phone numbers for fake accounts while aiming to keep legitimate onboarding smooth. It combines Twilio Lookup’s Line Type Intelligence, optional Line Status, and Identity Match checks before using Twilio Verify to send an OTP and confirm possession of the number. The provided Flask example collects a phone number and first and last names, rejects landlines, nonfixed VoIP, toll-free numbers, and pagers, and accepts Identity Match only when both name fields are exact_match or high_partial_match. The post shows how to configure the project, run it locally, inspect outcomes such as LINE_TYPE_BLOCKED or identity-match failure, and consider pricing because the four API calls have different costs.
A universal interface: How QuintoAndar made ClickHouse plug-and-play with managed Postgres
QuintoAndar’s customer data platform was split between a Postgres-based hot layer for low-latency lookups and a Databricks-based cold layer for analytical processing, creating duplicate code and expensive MERGE INTO operations. The team consolidated the pipeline by having its event gateway write directly to ClickHouse Cloud, which also serves API requests and analytical queries while remaining accessible to Trino, Spark, and Superset. Hightouch’s Customer Studio did not support ClickHouse, so QuintoAndar used ClickHouse Managed Postgres and the pg_clickhouse extension to expose ClickHouse tables as Postgres tables. Queries are pushed down to ClickHouse, while Hightouch’s planner and audit tables remain in NVMe-backed Postgres, allowing the existing activation workflow to continue with negligible round-trip impact for that use case.
From Preferences to Principles: Rubric-Based Alignment for Grounded Knowledge Answers
Designing reward signals for open-domain question answering remains challenging because high-quality answers must satisfy multiple distinct criteria that holistic scalar objectives fail to capture. To address this limitation, researchers developed a rubric-based reward framework that produces query-specific rubrics grounded in retrieved evidence and decomposed into specific quality dimensions. This structured reward setup provides fine-grained supervision during the post-training phase. Across the evaluation axes of composition, grounding, and instruction-following, the multi-dimensional grounded rubric method outperforms instruction-tuned baselines by 6.5% and flat rubric variants by 4%. Grounding rubrics in retrieved evidence bolsters factual support, while dimension decomposition enhances organization, coherence, and requirement adherence.
NVIDIA NVLink Fusion Brings NVHBM to Next-Generation AI Infrastructure
AI factories need custom XPUs to handle larger models and complex reasoning workloads, but scaling them requires HBM bandwidth, package and silicon area, power efficiency, supply-chain resilience, and rack-scale integration. NVIDIA NVLink Fusion addresses this by connecting custom XPUs and CPUs to NVIDIA’s AI infrastructure platform, using its scale-up and scale-out stack, ecosystem, and MGX rack-scale architecture. Its package-level NVHBM custom HBM base die, designed and validated with leading memory vendors, provides up to 30% more bandwidth and 15% lower HBM power usage than standard HBM4e, while enabling up to 25% more compute die area. The combined NVLink Fusion and NVHBM design is described as delivering a 30% overall end-to-end performance increase per XPU and simplifying integration into heterogeneous, rack-scale deployments.
Anthropic joins White House pledge for AI education
Anthropic announced three commitments at the White House’s AI Education Taskforce event, following its August signing of the Pledge to America’s Youth: Investing in AI Education. It will invest $1 million over three years in Carnegie Mellon University’s PicoCTF, support the Presidential AI Challenge, and develop a Creative Commons-licensed AI Fluency curriculum for K–12 and higher-education instructors that can work with any AI system. The announcement also reports adoption of Claude across education, including MagicSchool’s reach to 3 million educators, Claude for Education serving more than 100,000 students and faculty, and Claude Builder Clubs launching at over 60 U.S. universities. Anthropic says analysis of more than one million university-student conversations and 74,000 educator conversations informed features such as Learning Mode and the curriculum, with the stated aim of enhancing rather than compromising learning.
Claude for Enterprise powers LLNL research
Lawrence Livermore National Laboratory (LLNL) is expanding Claude for Enterprise access to its entire laboratory, reaching about 10,000 scientists, researchers, and staff. The deployment is intended to support work in nuclear deterrence, energy, materials science, and energy security, with applications spanning emergency response, fusion research, advanced manufacturing, computational biology, and high-performance computing. Claude’s government-oriented security features include single sign-on, audit logging, role-based access controls, end-to-end encryption, and an expanded context window that can process hundreds of documents, codebases exceeding 100,000 lines, or complex datasets in one query. The expansion follows a pilot, an AI Jam with U.S. National Labs, and a March aiEDGE for Innovation Day attended by approximately 3,200 LLNL scientists and operational staff.
How to Train a Cross-Embodiment Robot Navigation Policy with AI Agents
The tutorial presents an agent-driven workflow for adapting robot navigation across embodiments and scenes, addressing the repeated cost of collecting data, preparing simulation assets, training, diagnosing, and evaluating each robot-scene pair. Using Boston Dynamics Spot, it applies COMPASS to a built-in warehouse and a SAGE-10K scene, with NVIDIA Omniverse NuRec described as an option for captured environments. COMPASS reuses the pretrained NVIDIA X-Mobility navigation policy and trains a residual reinforcement-learning specialist that corrects base actions for a selected robot and environment; specialists can later be distilled into a shared policy. Repository skills guide dependency checks, asset preparation, smoke testing, training, checkpoint comparison, and packaging, while human gates control scene acceptance, smoke-test continuation, and checkpoint promotion. Runtime integration converts camera images, goals or routes, and speed from odometry into velocity commands, with cuVSLAM available separately when compatible odometry and transforms are unavailable.
Understanding and addressing AI harms
An evolving framework addresses AI harms beyond catastrophic risks, covering concerns such as biological threats, child safety, disinformation, and fraud. It complements the Responsible Scaling Policy by examining physical, psychological, economic, societal, and individual-autonomy impacts, then assessing likelihood, scale, affected populations, duration, causality, technology contribution, and mitigation feasibility. Policies and practices include a Usage Policy, pre- and post-launch evaluations, red teaming, adversarial testing, misuse detection, and enforcement ranging from prompt modifications to account blocking. Examples include computer-use safeguards for banking platforms and communications tools, plus hierarchical summarization intended to detect harms while maintaining privacy standards, alongside model-response evaluations balancing helpfulness with harmlessness. For Claude 3.7 Sonnet, encouraging safe, helpful responses to ambiguous prompts produced a reported 45% reduction in unnecessary refusals while maintaining safeguards against truly harmful content; the broader approach remains under development.
Challenges in red teaming AI systems
AI red teaming is presented as adversarial testing intended to identify vulnerabilities and risks in AI systems, but inconsistent practices make safety comparisons difficult. The post surveys domain-specific expert testing, policy vulnerability testing, frontier-threat assessments, multilingual and multicultural work, language-model-assisted attacks, automated red teaming, multimodal testing, and crowdsourced or community-based approaches. It describes an iterative process in which experts articulate threat models and probe systems qualitatively, then standardize inputs and use language models to generate many variations for more quantitative automated evaluation. In automated red teaming, a red team model generates attacks and a blue team model is fine-tuned on those outputs to improve robustness. The conclusion calls for technical standards, independent testing capacity, professional services, third-party access, and links between red teaming and model release conditions.
Accenture, AWS, and Anthropic collaboration
Anthropic, AWS, and Accenture announced a collaboration to help enterprises move generative AI ideas from concept to production, with an emphasis on regulated sectors where accuracy, reliability, and data security are paramount. More than 1,400 Accenture engineers will train as specialists in Anthropic models on AWS, supporting end-to-end delivery, customer-specific fine-tuning with private data, and prompt and platform engineering through Amazon Bedrock and Amazon SageMaker. In public health, the initiative has already produced Knowledge Assist, a Claude-powered chatbot built with the District of Columbia Department of Health that answers questions about health programs and services in English and Spanish. The collaboration combines Anthropic’s AI expertise, AWS’s security and reliability approach, and Accenture’s industry knowledge to develop tailored solutions for trust-driven sectors and streamline adoption of AI systems.
SKT partnership announcement
SK Telecom (SKT), Korea’s largest mobile operator, has become a commercial partner and strategic investor in Anthropic. The companies will develop a multilingual large language model customized for telecommunications applications such as customer service, marketing, sales, and interactive consumer experiences, with support planned for Korean, English, Japanese, Spanish, and more. Anthropic will fine-tune Claude using SKT’s telecommunications expertise: SKT experts will provide feedback on responses, and that feedback will help train industry-specific solutions. SKT has invested an additional $100 million in Anthropic, following an earlier investment by SK Telecom Venture Capital, while the companies describe the partnership as a way to promote AI innovation and develop safer, more reliable technology for telcos worldwide.
Frontier model security
Frontier AI models and their research are presented as strategic assets requiring protections beyond ordinary commercial technology because theft or misuse could affect economic and national security. The post recommends treating advanced AI as akin to critical infrastructure, with public-private cooperation and possible procurement or regulatory requirements. Its central control is multi-party authorization: no individual retains persistent access to production-critical environments, and time-limited access requires a coworker’s approval and business justification. It also argues that NIST’s SSDF and SLSA can be adapted to model development and coupled software, creating a chain of custody that supports provenance for deployed systems. Anthropic says it is implementing these controls and other cybersecurity practices through an iterative process, while acknowledging protections must evolve as model capabilities scale.
Zoom partnership and investment in Anthropic
Anthropic announces a partnership with Zoom under which Zoom will use Claude, Anthropic’s AI assistant built with Constitutional AI, to develop customer-facing products centered on reliability, productivity, and safety. Zoom’s federated approach will combine its own technology with models including Claude to address diverse customer needs, with the first Claude integration planned for the Zoom Contact Center portfolio. There, Claude is intended to improve the end-user experience and support contact-center agent performance. Zoom Ventures has also invested in Anthropic, aligning the companies around customer-centric AI products grounded in trust and security and designed for real-world use. The announcement presents the collaboration as a way to bring robust, steerable AI into workplace applications.
Introducing 100K context windows
Claude’s context window has expanded from 9K to 100K tokens, or about 75,000 words, enabling businesses to submit hundreds of pages and sustain conversations for hours or days. The announcement describes using the larger window to digest documents, synthesize information across multiple files, answer questions, and work with full codebases, with the claim that complex questions may perform substantially better than vector-search approaches. In a test, Claude-Instant processed the 72K-token text of The Great Gatsby after one line was altered, identified the change correctly, and responded in 22 seconds; 100K tokens is also described as roughly six hours of audio. The capability is available through the API, with examples including financial analysis, legal review, developer-documentation search, podcast summarization, and rapid prototyping.
Anthropic partners with Google Cloud
Anthropic, an AI safety and research company, has selected Google Cloud as its cloud provider in a partnership intended to support the next phase of its work. The companies plan to co-develop AI computing systems, while Anthropic will use Google Cloud’s GPU and TPU clusters to train, scale, and deploy its AI systems to a larger set of people. The work centers on Claude, Anthropic’s AI assistant, and on building safe, steerable systems using techniques including Constitutional AI, which is intended to make AI technologies easier to rely on and understand. Anthropic says the infrastructure’s performance and scale will support a more robust platform for reliable, interpretable, and steerable AI systems.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.