Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Insights and lessons from some of the best people in product - The Intercom Blog
The post compiles insights from Intercom’s 50th podcast episode, featuring product leaders Bob Moesta, Julie Zhuo, Michael Pryor, and Jason Fried on topics from customer jobs to pricing and product overhaul. Bob Moesta explains that new products always replace existing consumption; understanding what customers stop doing reveals the job to be done, as there is no new time in the world. Julie Zhuo describes a hierarchy where utility is foundational, then ease of use, and finally emotional appeal differentiates products when utility is equal. Michael Pryor shares that Trello initially avoided charging but user fear of shutdown forced them to introduce a simple pricing model, and they debated per-user versus flat fees. Jason Fried advocates for periodic ground-up rebuilds when new ideas cannot be retrofitted, supporting old versions indefinitely and not forcing upgrades.
From post-training to production: How TinyFish runs its web-native model on Databricks AI Runtime
TinyFish builds infrastructure for enterprises that need agents to work across the live web and is developing its own specialized web-native model. The company uses Databricks AI Runtime to manage GPU compute for post-training with supervised fine-tuning and moving toward Direct Preference Optimization, MLflow for evaluation, and Model Serving for production deployment. TinyFish evaluates its model by measuring full agent task completion rather than individual responses, achieving a 77.6% success rate on its internal go/no-go benchmark where 1,691 of 2,178 tasks were completed successfully. Over a 15-day period, the web-native model handled 294,781 agent runs across 6.4 million tool steps at an estimated serving cost of $0.088 per run or $0.0040 per tool step. By consolidating training, evaluation, serving, and operational data on Databricks, TinyFish reduces infrastructure overhead and maintains a continuous development cycle from agent behavior to deployment.
Filter, reshape, and redact logs before export with Transformers
Cloudflare Logpush now supports Transformers, allowing teams to filter, redact, and reshape log streams using SQL before export to SIEMs, analytics platforms, or object storage. This eliminates the need for a separate processing service that teams would otherwise have to deploy, scale, and maintain. Transformers are SQL queries applied per batch within Logpush’s high-throughput pipeline, using Apache DataFusion and Arrow for columnar execution. The implementation embeds DataFusion in Bufstream, the Go-based delivery service, using cgo and Arrow IPC to pass typed batches across the language boundary without introducing network hops. Customers can create, preview, and version Transformers via the dashboard or API, and attach them to NDJSON Logpush jobs to reduce downstream data volume and cost.
Agentic antimicrobial resistance surveillance with AWS HealthOmics and Amazon Bedrock AgentCore
Antimicrobial resistance, estimated to cause 1.27 million direct deaths in 2019, requires surveillance that normally demands bioinformatics skills to turn raw reads into answers. It ingests public SRA data via a Step Functions workflow and Fargate task, then runs a HealthOmics Nextflow pipeline that trims reads, assembles them, and detects resistance genes with AMRFinderPlus using a baked-in database. Results are normalized into the hAMRonization schema and merged idempotently into Iceberg tables on S3 Tables, so reprocessing an isolate updates rows rather than duplicating them. A Strands agent on Amazon Bedrock AgentCore answers plain-language questions by generating Athena queries, with Guardrails keeping responses grounded. The solution is defined entirely in AWS CDK, deploys with a single command, and avoids manual bioinformatics steps, remaining inexpensive when idle.
Beyond the AI noise: How Stack Overflow is rethinking tech hiring
The post explains that hiring has become exponentially harder due to a shift to an employer-led market, new roles in machine learning, and AI-generated resumes that erode trust on both sides. A recent Staff Data Scientist posting received over a thousand applications in 24 hours, many with identical formatting and links to nonexistent profiles. Traditional resumes no longer provide trusted information, and AI tools like ChatGPT produce generic answers that obscure genuine candidates, forcing recruiters to add more stages to hiring processes. Stack Overflow is resurrecting its Developer Story feature to let technologists showcase their work, expertise, and impact with verifiable details, including personal projects and career achievements. Developer Story aims to give recruiters objectively verifiable insight and help candidates stand out from the crowd, fixing a broken part of the hiring process.
Build an AI matching engine with Docling, Granite & OpenShift AI
PeopleMesh uses Docling and Granite4:3B LLM on Red Hat OpenShift to extract structured data from PDF resumes and match people across multiple dimensions. Docling converts documents into a DoclingDocument with semantic labels using an AI model trained on 150,000 annotated samples to recover reading order from PDF’s print-oriented encoding, then exports to Markdown for LLM-driven JSON extraction via a system prompt. For matching, PeopleMesh decomposes a user query with an LLM to extract intent, entity type, country, and language, then applies structured filtering followed by semantic vector search via cosine similarity on distilled query embeddings. To mitigate embedding dilution from concatenated concepts, PeopleMesh uses a weighted scoring algorithm that reranks top 100 candidates using 7 information channels including nice-to-have skills and penalty multipliers for missing must-have and negative skills. The scoring provides relative ordering only, not normalized probabilities, and default weights are adjustable via the SearchOptions Java record.
Build Your Game Challenge: Join in and Compete for Prizes
JetBrains Academy announces the Build Your Game Challenge for October, where participants complete the Build a Game With an AI Coding Agent course, add custom features, and submit their projects for prizes. The challenge runs from October 12 to October 23, with community voting and a JetBrains jury evaluating submissions on creativity, originality, gameplay, and player experience. Prizes include $400 and $250 for Community Choice, $150 for a Jury Award, and four $50 Jury Awards for standout projects, all as digital gift cards. Participants can use Junie Lite in JetBrains Air for free to build their game, or switch to a more capable model for complex mechanics. Submissions and voting open on October 19 via Discord, and winners will be announced on October 28.
Now in GA: Power Live Conversations with Segment Conversation Memory Integration
Segment Conversation Memory Integration is now generally available, bridging Segment CDP profiles with Twilio’s low-latency conversational recall layer. The integration streams customer traits into live voice, SMS, WhatsApp, and Flex interactions in under 300 milliseconds. It eliminates the need for custom webhooks or middleware pipelines by providing a click-to-configure connector within Segment’s Destinations Catalog. Key benefits include sub-300ms latency, 10x to 15x lower LLM token costs by selectively syncing factual traits instead of raw JSON histories, and faster agent deployment. CarFinance247 used the integration to build an AI assistant that handles approximately 288,000 conversations and drives 1.6x faster lead conversions.
Enriching Product Catalogs For Agentic Commerce Discovery with MongoDB
AI agents are transforming retail discovery, requiring product catalogs to be machine-readable and context-rich for systematic retrieval, with McKinsey estimating AI agents could orchestrate $3 trillion to $5 trillion globally by 2030. Retailers need a single enriched catalog that projects consistently to three surfaces: outbound product feeds, agentic commerce endpoints via REST APIs or MCP, and product detail pages with schema.org JSON-LD. MongoDB’s document model supports three tiers of enrichment—identity and transactability, filterable attributes, and agentic context such as use cases and semantic embeddings—to help AI agents reason over products. Change streams and aggregation pipelines keep projections current across surfaces, ensuring consistency that AI agents cross-check. MongoDB Search and Vector Search enable hybrid retrieval for intent-based queries, helping retailers improve discoverability and gain a competitive advantage in the agentic commerce era.
MongoDB Extension for Hibernate ORM is Now GA
The MongoDB Extension for Hibernate ORM is now generally available as a first-party, open-source library, giving Java teams a production-ready way to use familiar Hibernate APIs with MongoDB’s document model. Developers can continue using @Entity annotations, HQL, Criteria Queries, and Session APIs while building new applications or migrating existing Hibernate workloads from relational databases. The GA release adds JOIN support (INNER and LEFT OUTER) via standard HQL/JPQL syntax, optimistic locking with @Version, Spring Boot auto-configuration that auto-detects dialect and connection provider from the MongoDB JDBC URL, and clearer exception messaging and logging. The extension transparently translates Hibernate operations into MongoDB aggregation pipelines and requires Hibernate ORM 7.4 LTS. It also supports native embedded documents for faster reads and allows dropping into MongoDB’s native Query API for advanced operations like vector search and geospatial queries.
Large Language Models for Machine Translation Quality Annotation: Humans and Models Are Both Challenged
The paper evaluates LLMs as alternatives to human judgment for machine translation quality annotation, comparing agreement with human annotators on two schemes: Multidimensional Quality Metrics (MQM) and Error Span Annotation (ESA). Tests cover a long-context set of 70 language pairs plus WMT23 and WMT25 data, examining both score and error span annotation agreement across language pairs and domains. Results show LLM-human agreement sometimes exceeds human-human agreement, but both vary substantially across annotation schemes, language pairs, and domains and remain unreliable for most settings. Humans are particularly challenged by fine-grained MQM annotations and low-resource language pairs, while LLMs struggle with minor errors, wrong language variants, and error span annotation. The authors suggest human-LLM collaborative annotation pipelines as a way to address the reliability issues identified in this work.
Minions: A Dynamic and Distributed Virtual GPU Cloud Built on Idle Apple Silicon Devices
Minions is a dynamic, distributed virtual GPU cloud that aggregates idle Apple Silicon GPUs across macOS devices to run large-scale machine learning inference. The work responds to limits in centralized GPU infrastructure, including high cost, limited elasticity, and long wait times when resources are constrained. Demand for GPU-accelerated machine learning inference keeps rising, while modern Apple Silicon macOS devices carry powerful GPUs that stay largely idle during normal usage. Minions offers flexible GPU inference task execution and provides fault tolerance and efficient resource utilization, letting teams use existing hardware in place of additional GPU procurement. The authors report that idle macOS devices can become a practical distributed GPU inference platform in environments where centralized GPU resources are constrained.
Self-Compacting Language Model Agents
Long agent traces of chains of thought and tool calls accumulate stale content that anchors later generations and outgrow context window. SELFCOMPACT lets the model itself decide when and how to compact rather than using fixed-interval compaction triggered at a token threshold. It pairs a compaction tool that summarizes accumulated context with a lightweight rubric for when to fire, such as resolved sub-task or converging trajectory, and when to suppress, such as mid-derivation or stuck. Both are needed: the tool alone is unevenly used across open-weight models, while a rubric closes a meta-cognitive gap without training. On six math and agentic-search benchmarks across seven models, SELFCOMPACT matches or exceeds fixed-interval summarization at a fraction of the token cost, improving over no summarization by up to 18.1 points on math and 5–9 points on agentic search at 30–70% lower per-question cost.
Building A Culture Of Growth: MongoDB's Massive Opportunity In APAC
MongoDB’s go-to-market organization in APAC has nearly tripled in size over the past three years, driven by the region’s massive database opportunity. IDC forecasts the global database market will grow to $137 billion by 2027, and MongoDB has reached $1 billion in annual recurring revenue, with APAC containing five of the world’s top 13 economies by GDP. To serve diverse markets like India, Australia, and Singapore, MongoDB deploys regionalized business leaders who tailor go-to-market strategies while collaborating on shared best practices at the leadership level. The company’s culture, shaped by six core values including “Embrace the Power of Differences,” emphasizes psychological safety and accountability, encouraging teams to learn from setbacks. MongoDB’s APAC teams prioritize three goals: ensuring customer success, building technical champions, and driving new workloads, as exemplified by Cathay Pacific’s Flight Folder application powered by MongoDB.
Endpoint Candidates: evaluate and promote inference endpoints
Modal introduced Endpoint Candidates to stage alternative inference deployments for Dedicated Endpoints, allowing evaluation of quality, latency, throughput, and cost without affecting production traffic. Users create a Candidate by selecting a model from Modal’s catalog or importing custom weights, then set a percentage of production traffic to mirror. The Pingora-based proxy forks each request stream into separate bounded buffers per Candidate and shares body chunks across mirrors with a per-process memory budget; if a buffer fills or the budget is exceeded, the proxy cancels that mirrored request while production continues. Each Candidate receives its own URL for running existing evaluations on output quality and performance benchmarks under load, and all mirrored compute uses standard Modal compute pricing. When ready, users promote from the dashboard to roll out to the production endpoint, leaving other Candidates available for continued independent testing and enabling Auto Endpoints to validate automated inference optimizations.
How California built its Behavioral Health Public County Profile on Databricks
The California Department of Health Care Services built the Behavioral Health Public County Profile on Databricks, a public dashboard consolidating behavioral health data from all 58 counties for nearly 40 million Californians. Launched on October 1, 2026, following the 2024 passage of Proposition 1, the dashboard supports the new Behavioral Health Services Act (BHSA) tying funding to statewide goals for outcomes, equity, and transparency. The platform uses Spark Declarative Pipelines and Lakeflow Connect for unified data ingestion, Semantic Views for standardized metrics across counties, and serverless compute to process data at scale without infrastructure management. It operates on a FedRAMP-authorized environment meeting federal security standards for health data, with data quality and lineage ensuring every number can be traced to its source. The County Profile enables public comparison of county plans, funding, service delivery, and performance measures, creating a feedback loop between policy intent and measurable outcomes for democratic oversight.
How iSAM Funds built an options research platform for a 10,000x data problem with ClickHouse Cloud
iSAM Funds, a UK-based systematic asset manager, built an options research platform on ClickHouse Cloud to handle the 10,000x data fan-out from underlying instruments to related options. Migrating from Postgres, the team achieved 35-50x compression and increased single-core ingestion from 8,000 to 800,000 rows per second, while ClickHouse’s native Apache Arrow support eliminated encoding reshuffling. ClickHouse’s columnar storage and vectorized execution made queries fast enough to run on demand, allowing the team to drop pre-computed rollups and materialized views and remove a layer of pipeline complexity. The managed service integrates with iSAM’s AWS footprint and enterprise authentication, pushes backups to iSAM’s own buckets for developer use, and lets them scale instances up for large backfills then down to control costs. The platform gives researchers fast, interactive access to full options history, shortening the time from hypothesis to tested strategy.
Introducing Clef-omni with full multimodality, plus a faster Clef and a cheaper Clef-flash
Cloudflare introduced Clef-omni, a decision model that natively handles audio, video, image, and text inputs in a single pipeline, built on a Qwen3-Omni mixture-of-experts backbone with text-to-speech components removed. Unlike cascading pipelines, it directly embeds media into a unified sequence and uses two-stage attention routing with lexical grammar to compute confidence scores, trained by freezing the backbone and applying LoRA adapters with label-smoothed cross-entropy and Brier score calibration. The company also cut Clef-flash’s price to $0.038 per million input tokens while reducing its context window from 64k to 24k for the hosted version, based on usage data showing only 0.24% of requests exceed that limit. Clef’s hosted version received speed optimizations at the infrastructure layer without new model weights, and Clef-omni is available open-weight on HuggingFace. Benchmarks show Clef-omni scoring 98.2 on BFCL case exact and 92.7 on API-Bank accuracy, with median text-only decision latency around 130 milliseconds.
Impactful scheduling for GPU clusters
Ai2 replaced its priority-based GPU cluster scheduler with a system combining GPU time budgets, hierarchical fair-share allocation, and a time-slicing contract to prioritize high-impact research while maintaining full occupancy. The previous priority scheduler induced GPU squatting, priority inflation, and constant negotiation over non-preemptable workloads, exemplifying a tragedy of the commons. In the new model, managers allocate proportional GPU time budgets to projects, making resource allocation a transparent administrative process rather than an operational debate. The scheduler tracks occupancy over a time window with weights set by budgets, and time-slicing enforces minimum runtime protection before workloads become preemptible. However, interactive sessions suffered from the 8-hour cap, prompting investment in a CPU-only cluster and restorable sessions, while potential capacity fragmentation for large workloads remains under investigation.
Solving defense supply chain visibility through governed data sharing
The Department of War cannot obtain cost transparency across its supply chain of over 200,000 suppliers due to reliance on manual data calls and fragmented spreadsheets. The proposed solution is a federated data exchange where each contractor publishes validated cost and pricing evidence from existing records to named government recipients using open-source sharing protocols. Databricks provides the platform, enforcing governance, access controls, CUI markings, and audit lineage without storing raw records centrally or requiring contractors to adopt Databricks. Small and mid-tier suppliers submit through validated intake forms that produce the same governed data contracts as large primes, requiring no new accounting systems. The reporting obligation already exists through CSDR, FlexFile, and DD Form 1921, so a pilot exchange can launch in months by defining two data contracts on a single program.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.