---
title: "The AMR Surveillance Stack: From Raw Reads to Governed Answers"
description: "Antimicrobial resistance surveillance is a layered data architecture: raw reads and demand signals flow in, get normalized into governed tables, and answer plain-language queries. The diagram maps each boundary."
---

# The AMR Surveillance Stack: From Raw Reads to Governed Answers

**Type:** [Architecture](https://yomu.fyi/visual-explainers/category/architecture.md)

![Layered architecture diagram for antimicrobial resistance surveillance: an ingest layer, a bioinformatics pipeline, a normalization step, governed open tables, and a semantic plus agent query layer, connected by downward arrows with a privacy boundary running alongside.](https://yomufyi.s3.us-east-1.amazonaws.com/visual-explainers/f21f19f4-e7d7-4254-b108-98105c3286da/image-9093e658-5438-4e3e-a6ee-fca329f79d1b.webp)

> Antimicrobial resistance surveillance is a layered data architecture: raw reads and demand signals flow in, get normalized into governed tables, and answer plain-language queries. The diagram maps each boundary.

## Explanation

AMR surveillance turns raw sequencing reads into resistance signals, but the hard part is the architecture around the science. Public read archives and retail demand signals enter through an ingest layer, a bioinformatics pipeline trims, assembles, and detects resistance genes, and results are normalized into a shared schema before being merged idempotently into open table formats. A governed semantic layer standardizes metrics so different sources stay comparable, and an agent layer answers plain-language questions by generating queries against those governed tables rather than raw data. Each boundary is where privacy, cost, and trust decisions get made.

## FAQ

### Why normalize resistance genes into a shared schema before storing them?

Raw AMRFinderPlus output varies in shape, so the AWS HealthOmics example converts results into the hAMRonization schema before loading them. That common schema lets a reprocessed isolate be merged idempotently into Iceberg tables, updating rows instead of duplicating them. Without normalization, every downstream query has to know the quirks of each tool version.

**Tags:** [AI Agents](https://yomu.fyi/topic/ai-agents.md), [Antimicrobial Resistance](https://yomu.fyi/topic/antimicrobial-resistance.md), [Biology](https://yomu.fyi/topic/biology.md), [Data Governance](https://yomu.fyi/topic/data-governance.md), [Healthcare](https://yomu.fyi/topic/healthcare.md), [Public Health](https://yomu.fyi/topic/public-health.md)

## Related reading

### [GrabMart data for faster disease outbreak detection](https://yomu.fyi/post/grabmart-data-for-faster-disease-outbreak-detection.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Muqi Li
- Published: Oct 13, 2026
- Type: Explainer

GrabMart transaction data could supplement clinical surveillance because over-the-counter demand sometimes shifts before confirmed case reports. The concept aggregates eligible purchases, maps SKUs to broad symptom classes, and compares demand series against seasonal and historical baselines to produce an Early Outbreak Detection Index. Public health experts would review that aggregate signal alongside independent evidence, and the method excludes identity linkage and individual-level health inference. Statistical evaluation would need sensitivity, false-alert rate, calibration, geographic stability, and retrospective testing against approved public health data, with ongoing monitoring for catalogue changes, promotions, and shifting purchasing behaviour. Privacy relies on data minimisation, coarse geography, aggregate time windows, minimum-count thresholds, and possibly differential privacy under an approved budget; the authors state this is early-stage planning only, with no deployment, measured performance, or integration claimed.


### [The data lakehouse is a better data warehouse: 2026 benchmarks and proof](https://yomu.fyi/post/the-data-lakehouse-is-a-better-data-warehouse-2026-benchmarks-and-proo.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Joanna Zouhour
- Published: Oct 12, 2026
- Type: Problem & solution

Data warehouses have been the standard for structured SQL analytics for two decades but cannot serve modern AI, ML, streaming, and unstructured data workloads. The data lakehouse now matches warehouse SQL performance while serving all these workloads from a single governed foundation. Databricks Lakehouse improved production SQL workload performance by 77% from a 2022 baseline, with BI queries improving 14% over a recent five-month period. Customer migrations show 90% faster queries and 30–40% cost reduction at Lumen Technologies, and 80–90% faster analytics at Trek Bicycle. The lakehouse eliminates redundant data copies, supports open formats like Delta Lake and Iceberg, and extends governance to AI agents through Unity Catalog and Unity Gateway.


### [Filter, reshape, and redact logs before export with Transformers](https://yomu.fyi/post/filter-reshape-and-redact-logs-before-export-with-transformers.md)
- Company: [Cloudflare](https://yomu.fyi/company/cloudflare.md)
- Author: Sahidya Devadoss
- Published: Oct 12, 2026
- Type: Announcement

Cloudflare Logpush now supports Transformers, allowing teams to filter, redact, and reshape log streams using SQL before export to SIEMs, analytics platforms, or object storage. This eliminates the need for a separate processing service that teams would otherwise have to deploy, scale, and maintain. Transformers are SQL queries applied per batch within Logpush’s high-throughput pipeline, using Apache DataFusion and Arrow for columnar execution. The implementation embeds DataFusion in Bufstream, the Go-based delivery service, using cgo and Arrow IPC to pass typed batches across the language boundary without introducing network hops. Customers can create, preview, and version Transformers via the dashboard or API, and attach them to NDJSON Logpush jobs to reduce downstream data volume and cost.


### [Agentic antimicrobial resistance surveillance with AWS HealthOmics and Amazon Bedrock AgentCore](https://yomu.fyi/post/agentic-antimicrobial-resistance-surveillance-with-aws-healthomics-and.md)
- Company: [AWS](https://yomu.fyi/company/aws.md)
- Author: Nicolo Cosimo Albanese
- Published: Oct 12, 2026
- Type: Problem & solution

Antimicrobial resistance, estimated to cause 1.27 million direct deaths in 2019, requires surveillance that normally demands bioinformatics skills to turn raw reads into answers. It ingests public SRA data via a Step Functions workflow and Fargate task, then runs a HealthOmics Nextflow pipeline that trims reads, assembles them, and detects resistance genes with AMRFinderPlus using a baked-in database. Results are normalized into the hAMRonization schema and merged idempotently into Iceberg tables on S3 Tables, so reprocessing an isolate updates rows rather than duplicating them. A Strands agent on Amazon Bedrock AgentCore answers plain-language questions by generating Athena queries, with Guardrails keeping responses grounded. The solution is defined entirely in AWS CDK, deploys with a single command, and avoids manual bioinformatics steps, remaining inexpensive when idle.


### [How California built its Behavioral Health Public County Profile on Databricks](https://yomu.fyi/post/how-california-built-its-behavioral-health-public-county-profile-on-da.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Michael Smith, Cameron Kahn, Stephen Hsu, Scott Black, Ben McKay, Varun Ganti
- Published: Oct 10, 2026
- Type: Explainer

The California Department of Health Care Services built the Behavioral Health Public County Profile on Databricks, a public dashboard consolidating behavioral health data from all 58 counties for nearly 40 million Californians. Launched on October 1, 2026, following the 2024 passage of Proposition 1, the dashboard supports the new Behavioral Health Services Act (BHSA) tying funding to statewide goals for outcomes, equity, and transparency. The platform uses Spark Declarative Pipelines and Lakeflow Connect for unified data ingestion, Semantic Views for standardized metrics across counties, and serverless compute to process data at scale without infrastructure management. It operates on a FedRAMP-authorized environment meeting federal security standards for health data, with data quality and lineage ensuring every number can be traced to its source. The County Profile enables public comparison of county plans, funding, service delivery, and performance measures, creating a feedback loop between policy intent and measurable outcomes for democratic oversight.


### [Solving defense supply chain visibility through governed data sharing](https://yomu.fyi/post/solving-defense-supply-chain-visibility-through-governed-data-sharing.md)
- Company: [Databricks](https://yomu.fyi/company/databricks.md)
- Author: Mike Daniels, Molly Just-Behr, Kacey Hertan
- Published: Oct 9, 2026
- Type: Problem & solution

The Department of War cannot obtain cost transparency across its supply chain of over 200,000 suppliers due to reliance on manual data calls and fragmented spreadsheets. The proposed solution is a federated data exchange where each contractor publishes validated cost and pricing evidence from existing records to named government recipients using open-source sharing protocols. Databricks provides the platform, enforcing governance, access controls, CUI markings, and audit lineage without storing raw records centrally or requiring contractors to adopt Databricks. Small and mid-tier suppliers submit through validated intake forms that produce the same governed data contracts as large primes, requiring no new accounting systems. The reporting obligation already exists through CSDR, FlexFile, and DD Form 1921, so a pilot exchange can launch in months by defining two data contracts on a single program.