Topic

Protocol Buffers

2 posts about Protocol Buffers. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Posts

Swiggy
How Swiggy Migrated from protobuf-java to Square Wire
Swiggy migrated from protobuf-java to Square Wire after profiling showed that the runtime and protoc-generated code consumed 6.735 MB of Android app size. protobuf-javalite was rejected because Swiggy relies extensively on google.protobuf.Any, while Wire retained support for Any and other well-known types with a smaller runtime and immutable Kotlin models. The migration required updating snake_case property access, message construction, serialization calls, ProGuard rules, and a custom pipeline that compiled all schemas together before publishing package-specific AARs to AWS CodeArtifact. The team also fixed repeated enum fields with [packed = true] after discovering a silent wire-format mismatch. It reduced Swiggy’s app size by 3.52 MB and produced measurable reductions for Instamart and Toing.

Heroku
Heroku Metrics
Heroku’s original Dashboard Metrics system stored 10-minute rollups in a 21-shard InfluxDB cluster, but new requirements for one-minute data, richer events, extensibility, and failure tolerance exposed infrastructure limits. To replace it, the team built a Kafka-based pipeline on Private Spaces: an ingress app parses Logplex frames into Protocol Buffers messages, Go consumers aggregate measurements into one-minute summaries, and a sink writes rolled-up data to seven sharded, partitioned Heroku Postgres databases. The custom Yahtzee framework uses accumulator maps for counts and streaming min, max, and sum calculations with deterministic partition ownership; late data and aggregation-process restarts were implementation hurdles. The design reduces reliance on expensive InfluxDB continuous queries and makes adding another datastore straightforward, although Postgres performance remains merely acceptable and measurements can still be lost before Kafka because the logging pipeline is lossy by design.

The best engineering reads, once a week.

Topic

2 posts about Protocol Buffers. Every summary links to the original.

Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.

Newest first

Swiggy
How Swiggy Migrated from protobuf-java to Square Wire
Swiggy migrated from protobuf-java to Square Wire after profiling showed that the runtime and protoc-generated code consumed 6.735 MB of Android app size. protobuf-javalite was rejected because Swiggy relies extensively on google.protobuf.Any, while Wire retained support for Any and other well-known types with a smaller runtime and immutable Kotlin models. The migration required updating snake_case property access, message construction, serialization calls, ProGuard rules, and a custom pipeline that compiled all schemas together before publishing package-specific AARs to AWS CodeArtifact. The team also fixed repeated enum fields with [packed = true] after discovering a silent wire-format mismatch. It reduced Swiggy’s app size by 3.52 MB and produced measurable reductions for Instamart and Toing.

Heroku
Heroku Metrics
Heroku’s original Dashboard Metrics system stored 10-minute rollups in a 21-shard InfluxDB cluster, but new requirements for one-minute data, richer events, extensibility, and failure tolerance exposed infrastructure limits. To replace it, the team built a Kafka-based pipeline on Private Spaces: an ingress app parses Logplex frames into Protocol Buffers messages, Go consumers aggregate measurements into one-minute summaries, and a sink writes rolled-up data to seven sharded, partitioned Heroku Postgres databases. The custom Yahtzee framework uses accumulator maps for counts and streaming min, max, and sum calculations with deterministic partition ownership; late data and aggregation-process restarts were implementation hurdles. The design reduces reliance on expensive InfluxDB continuous queries and makes adding another datastore straightforward, although Postgres performance remains merely acceptable and measurements can still be lost before Kafka because the logging pipeline is lossy by design.

The best engineering reads, once a week.