Optimizing ML Workload Network Efficiency (Part I): Feature Trimmer

Summary

Pinterest deployed a root-leaf architecture for online machine learning serving to decouple CPU-based feature preprocessing from GPU-based inference. Because the root cluster retrieved a union of features across models and fanned out the full set to every leaf, network bandwidth became a severe bottleneck. After initial tests with fbthrift LZ4 compression yielded a 20% bandwidth reduction at the cost of CPU and latency overhead, the team implemented Feature Trimmer. This mechanism uses exported model signatures as the source of truth to prune unused features on the root before transmission. By dispatching only required features per model, the system reduced network bandwidth, decreased latency, and enabled cluster rightsizing that saved over $4M annually.

Context

In Pinterest's root-leaf ML serving architecture, the root cluster fetches the union of features needed by all leaf models and distributes them across the network. Transmitting unneeded features created a severe network bottleneck that constrained GPU utilization and forced the root fleet to use expensive network-optimized AWS m6in instances.

Approach / What changed

Pinterest adopted a Send What You Use design via Feature Trimmer. The root inspects model signatures exported with TorchScript artifacts to establish an allowlist of required inputs for each leaf model. Root nodes trim unneeded features before fanning out RPC scoring requests, keeping feature allowlists synchronized across model deployments and rollouts.

Takeaways

• Enabling fbthrift LZ4 compression reduced root-to-leaf network usage by 20%, but introduced a 5% CPU usage increase and a 5ms (~10%) p90 latency increase.
• Feature Trimmer decreased egress network throughput by approximately 45% in Search and 65% in Notification, enabling instance changes that cut costs by at least 30%.
• The optimization downsized the Ads root cluster by 27%, unlocked roughly 5% in leaf GPU capacity, reduced Related Pins p99 latency by 25-30%, and saved over $4M annually.

Related reading

• Pinterest · Apr 27, 2026

  From Clicks to Conversions: Architecting Shopping Conversion Candidate Generation at Pinterest

  Pinterest built a dedicated shopping candidate generation model to optimize for lower-funnel purchase conversions rather than relying strictly on engagement retrieval models. Because offsite conversion events are sparse and delayed, engineers trained a single multi-surface two-tower model using conversion data supplemented by duration-weighted click engagement and unengaged ad impressions as hard negatives. The two-tower retrieval architecture incorporates a parallel DCN v2 and MLP layer structure that allows simultaneous learning from the same inputs without information bottlenecks. Later revisions transitioned the system from a multi-head loss structure to a unified single-head multi-task architecture paired with an advertiser-level objective. Deployed across more than 600 million monthly active users, the system increased shopping conversion volume by 2.3% and boosted return on ad spend by 3.1% for United States shopping campaigns.

  Pinterest Engineering

• Pinterest