Loading…
Topic
Amazon Redshift
1 post about Amazon Redshift. Every summary links to the original.
- 1 post
- 1 company
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Aderant modernizes analytics data pipeline with Apache Iceberg and AWS Lambda
Aderant modernized its Stridyn analytics pipeline using Apache Iceberg, AWS Lambda, and AWS Glue, replacing a 15-minute batch cycle with event-driven processing. The new architecture handles 380 million CDC events per month across 700 client environments and 266 source tables, processing changes in single-digit seconds. It uses a medallion architecture (bronze, silver, gold) on Iceberg tables in S3, with Lambda functions deployed as Docker containers for ingestion and AWS Glue for transformations. The system achieved 10–26x faster query performance on Amazon Redshift and a 77 percent cost reduction compared to streaming alternatives. Key design choices include Iceberg for ACID transactions and schema evolution, DynamoDB-based watermarking for exactly-once semantics, and a hybrid gold-layer strategy using incremental Glue jobs and materialized views with snapshot comparison.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.