From Ringing Line to Resolved Contact: How a Call Center Works
Follow one contact through a call center: arrival, routing, handling by a human or AI agent, resolution, and the cost-per-contact math that decides which path to use.
A step-by-step look at how Chronon turns streaming feature commits into fresh model inputs, cutting recommender staleness from nearly two days to seconds.
Chronon is the feature pipeline layer behind Airbnb’s sequence recommender. In the old daily batch path, embeddings could be almost two days stale because same-day activity was unavailable until the next run. The streaming path uses Push Mode to emit a notification when a feature value commits, then a Near-real-time Model Transform invokes the deployed sequence encoder inside that pipeline. Each page view or search is merged with long-term and short-term sequence data, scored, and written to the same low-latency store used by batch serving, so embeddings typically update in under a minute. Treat feature freshness as a pipeline design decision, not just a model choice.
Daily batch left activity from earlier the same day unavailable until the next run, creating up to nearly two days of staleness. Push Mode emits a notification when streaming feature values commit, so the near-real-time transform can act immediately. That moved typical embedding updates to under a minute, often 10–30 seconds.
It invokes the deployed sequence encoder inside the streaming pipeline. Each page view or search is merged with the guest’s long-term and short-term sequence data, triggering inference, then the fresh embedding is written to the same low-latency store used by batch serving. Serving can then read a fresher embedding without waiting for a separate batch job.
The posts behind it
Apache Spark 4.3 introduces stateful processing in Real-Time Mode, achieving sub-150 ms P99 end-to-end latency for streaming aggregation, deduplication, and custom stateful processing using transformWithState. This eliminates the need for a separate streaming engine for millisecond-scale latency in fraud detection, user activity monitoring, and device monitoring, enabled by native concurrent stage scheduling and pipelined shuffles. The release also includes Project Feather for faster iteration on small data by reducing execution overhead, and SCD Type 2 Auto CDC for tracking record changes over time. SQL enhancements include JSON_TABLE, VARIANT transformations, ASOF JOIN for temporal matching, and a nanosecond timestamp preview. Spark 4.3 marks a shift to quarterly feature releases, with over 200 contributors addressing more than 1,100 Jira tickets, and is now available in Databricks Runtime 19.
The MongoDB CDC connector for ClickPipes is now generally available after replicating over 1.5PB of data for hundreds of production customers. It leverages MongoDB Change Streams for low-latency synchronization and introduces parallel snapshotting that automatically chunks collections by document ID to speed up initial loads and resyncs. A structured ingestion mode samples documents to infer a schema, mapping top-level fields to typed ClickHouse columns while storing unexpected or type-mismatched values in a JSON column for later handling. Observability has been extended with metrics for CPU, memory, network bandwidth, and replication lag, available in the ClickPipes UI and via Prometheus endpoint. The connector supports programmatic management through Open API and Terraform, and its billing is based on data volume with a fixed compute cost per service, offering 5-10x cost savings over third-party ETL tools.
Meta’s Facebook Feature Framework (F3) compiles all ads ML features into a single merged graph for efficient execution, but this graph also serves as the release unit, so one failing feature blocks the entire pipeline. F3 Lightning decouples feature releases from shared execution, allowing each feature to be compiled, validated, deployed, and rolled back independently. The main challenge was preserving F3’s computational efficiency, as early per-feature release prototypes increased CPU usage by losing shared computation. F3 Lightning closes this gap through four optimizations: stable computation identifiers, a single shared execution plan, one shared memory buffer across all features, and dense index mapping for direct array access. With F3 Lightning, comparable extraction features reach production ~3.6x faster than traditional F3 with no increase in infrastructure compute cost.
Airbnb’s guest sequence recommender previously refreshed Transformer-based embeddings in a daily batch, leaving activity from earlier the same day unavailable until the following run and creating up to nearly two days of staleness. To close this gap, Airbnb added Push Mode, which emits notifications when streaming feature values commit, and Near-real-time Model Transform, which invokes the deployed sequence encoder inside that streaming pipeline. Each page view or search is merged with the guest’s long-term and short-term sequence data, triggers inference, and writes a fresh embedding to the same low-latency store used by batch serving. The resulting system typically updates embeddings in under a minute, often within 10–30 seconds, improved offline NDCG by 1.67%, and increased uncancelled bookings by approximately one-third of a percent in online A/B tests.
More like this
Follow one contact through a call center: arrival, routing, handling by a human or AI agent, resolution, and the cost-per-contact math that decides which path to use.
A circuit breaker turns dependency failures into fast failures, then probes for recovery. See the state loop and how half-open probes can quietly consume worker capacity.
Cilium uses eBPF programs in the Linux kernel to enforce identity-based network policies and capture L7 observability for Kubernetes workloads. This diagram traces the path from packet arrival to policy verdict and Hubble flow.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.