# Optimizing ML Workload Network Efficiency (Part I): Feature Trimmer

[Pinterest](https://yomu.fyi/company/pinterest) · Pinterest Engineering · May 1, 2026

**Type:** Problem & solution

## Summary

Pinterest deployed a root-leaf architecture for online machine learning serving to decouple CPU-based feature preprocessing from GPU-based inference. Because the root cluster retrieved a union of features across models and fanned out the full set to every leaf, network bandwidth became a severe bottleneck. After initial tests with fbthrift LZ4 compression yielded a 20% bandwidth reduction at the cost of CPU and latency overhead, the team implemented Feature Trimmer. This mechanism uses exported model signatures as the source of truth to prune unused features on the root before transmission. By dispatching only required features per model, the system reduced network bandwidth, decreased latency, and enabled cluster rightsizing that saved over $4M annually.

## Context

In Pinterest's root-leaf ML serving architecture, the root cluster fetches the union of features needed by all leaf models and distributes them across the network. Transmitting unneeded features created a severe network bottleneck that constrained GPU utilization and forced the root fleet to use expensive network-optimized AWS m6in instances.

## Approach / What changed

Pinterest adopted a Send What You Use design via Feature Trimmer. The root inspects model signatures exported with TorchScript artifacts to establish an allowlist of required inputs for each leaf model. Root nodes trim unneeded features before fanning out RPC scoring requests, keeping feature allowlists synchronized across model deployments and rollouts.

## Takeaways

- Enabling fbthrift LZ4 compression reduced root-to-leaf network usage by 20%, but introduced a 5% CPU usage increase and a 5ms (~10%) p90 latency increase.
- Feature Trimmer decreased egress network throughput by approximately 45% in Search and 65% in Notification, enabling instance changes that cut costs by at least 30%.
- The optimization downsized the Ads root cluster by 27%, unlocked roughly 5% in leaf GPU capacity, reduced Related Pins p99 latency by 25-30%, and saved over $4M annually.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [AWS](https://yomu.fyi/topic/aws), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Performance](https://yomu.fyi/topic/performance), [Scalability](https://yomu.fyi/topic/scalability)

[Read original post](https://medium.com/pinterest-engineering/optimizing-ml-workload-network-efficiency-part-i-feature-trimmer-ae20beb08d69)