# Trident - Real-time Event Processing at Scale

[Grab](https://yomu.fyi/company/grab) · Jie Zhang · Jan 13, 2021

**Type:** Problem & solution

## Summary

Trident serves as Grab's internal real-time event-processing and workflow automation engine, driving user campaigns, rewards, and notifications across multiple business lines. To handle peak loads exceeding 2,000 events per second without duplicate execution, the system consumes decoupled Kafka streams and enforces exactly-once semantics using Redis and MySQL deduplication checks. Processing efficiency relies on server autoscaling aligned with Kafka partition counts, combined with dynamic goroutine allocation per consumer. To minimize rule evaluation overhead, Trident indexes active campaigns into an in-memory hash map by event type, cutting processing time by at least 90%. Furthermore, condition evaluation is optimized through lazy loading and a weighted sorting algorithm that checks low-cost in-memory data prior to executing expensive database queries or external service calls.

## Context

Executing thousands of automated campaign workflows across billions of events required a system that could scale under surging traffic without duplicate processing or excessive overhead from evaluating complex rules.

## Approach / What changed

Grab built Trident using decoupled Kafka ingestion with Redis and MySQL deduplication, aligned partition auto-scaling, in-memory event prefiltering, and weighted lazy rule evaluation.

## Takeaways

- Trident achieves deduplication and exactly-once processing by pairing Kafka's delivery with unique event keys cached in Redis for 24 hours and persisted in MySQL when actions trigger.
- Prefiltering active campaigns into an in-memory hash map keyed by event type narrowed evaluation candidates and eliminated at least 90% of processing time.
- Weighted rule evaluation dynamically sorts condition checks by data source cost—in-memory, database, and external service—to trigger early exits before invoking expensive remote calls.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Go](https://yomu.fyi/topic/go), [Kafka](https://yomu.fyi/topic/kafka), [Performance](https://yomu.fyi/topic/performance), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://engineering.grab.com/trident-real-time-event-processing-at-scale)