Loading post…
DatabricksReynold Xin, Andrew Krioukov, Molly Limaye, Taylor Kain, Keegan DubbsAnnouncement
Databricks Announces Lakewatch: New, Agentic SIEM
Summary
Databricks announces Lakewatch, an agentic SIEM intended to help organizations defend against increasingly sophisticated, AI-driven attackers. Launching in Private Preview, it unifies security, IT, and business data in a governed lakehouse built on Unity Catalog, while supporting open OCSF formats and storage in customer-controlled cloud object stores. Lakeflow Connect ingests and normalizes major security sources, and Detection-as-Code lets teams define YAML rules with SQL queries or Python notebooks, backtest them against historical data, and deploy through CI/CD. Genie Code and Genie Spaces provide AI-assisted ingestion, detection authoring, rule refinement, investigation, and natural-language threat hunting, while custom models can support anomaly detection, behavioral analytics, and entity risk scoring. The announcement also introduces an Open Security Lakehouse Ecosystem and acquisitions of Antimatter and SiftD.ai; Lakewatch is available in Private Preview, with Adobe and Dropbox among the named customers.
Context
The announcement frames traditional SIEM architecture as poorly matched to AI-driven attacks operating at machine speed. It identifies coupled storage and compute, limited retention, costly duplication, and inability to process multimodal data as constraints that leave defenders with only a fraction of their available security context.
Approach / What changed
Lakewatch runs security operations directly on the lakehouse, combining governed access through Unity Catalog, OCSF-based open formats, automated ingestion with Lakeflow Connect, and customer-controlled storage in Delta Lake or Apache Iceberg. Genie tools assist with ingestion, detections, and investigations; Detection-as-Code supports YAML, SQL, Python notebooks, historical backtesting, and CI/CD deployment. Serverless compute separates query cost from retained storage.
Takeaways
Lakewatch stores security telemetry alongside business data under Unity Catalog, enabling correlation across HR, collaboration, application, and transaction data without duplicating or moving files.
Detection-as-Code supports YAML rules with SQL queries or Python notebooks, historical backtesting, and deployment through CI/CD pipelines.
By decoupling storage from compute, Lakewatch is designed to retain full-fidelity, potentially multimodal telemetry in customer-controlled cloud storage while charging for compute when queries run.
FAQ
How does Lakewatch reduce the cost of retaining security data?
It separates storage from compute, allowing organizations to retain petabytes of full-fidelity telemetry in cloud object storage they control. The announcement says customers pay for compute rather than per-byte license fees and can run serverless analytics when needed.
What can Genie automate for security teams?
Genie Code can ingest and parse new log sources into OCSF, author detections, modify rules to reduce false positives, and translate natural-language questions into SQL. Genie Spaces provides a natural-language interface for complex, multi-step threat hunting.
How are detections tested and deployed?
Detection-as-Code defines rules in YAML using SQL queries or Python notebooks. Teams can backtest those rules against historical data and deploy them through CI/CD pipelines.
Source and freshness
- Source
- databricks.com
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.