Caching Without Marshal Part 1: Marshal from the Inside Out

Summary

Rails caching commonly relies on Ruby’s Marshal serialization, which can encode complex objects but also embeds their class names in cache entries. Shopify describes an incident in which a beta-flag refactor changed classes while old and new code overlapped during deployment; old code then failed on cached instances containing unfamiliar class names and methods, despite passing CI. The article examines Marshal through Ruby’s marshal.c implementation, covering atomic and composite types, instance variables, object references, circularity via TYPE_LINK, and core-type subclasses via TYPE_UCLASS. It presents MessagePack as a more compact, stricter, and controllable alternative intended to make caching safer by default, while deferring the cache migration and implementation details to the series’ next part.

Context

Shopify’s incident showed that Marshal-serialized cache entries can outlive a deployment and contain class names from newly deployed code. During the rollout, older code encountered beta-flag instances using refactored classes and failed, even though the change passed CI and was not itself incorrect. Reducing cache usage or slowing code changes were considered unacceptable mitigations.

Approach / What changed

The article investigates Marshal’s behavior using Ruby’s marshal.c implementation and examples of serialized records, self-referential arrays, and subclasses of core types. It proposes replacing Marshal with MessagePack, a more compact binary format with stricter typing and greater control, and identifies circularity and core-type subclasses as features a replacement must address.

Takeaways

• Marshal serializes an object’s class along with its data, so a deployment can leave older processes unable to read cache entries created with newly refactored classes.
• Rails cache backends share read and write operations and use Marshal.dump and Marshal.load to serialize cached data, including data stored through Memcached, file, memory, or Redis backends.
• Marshal represents circular references with TYPE_LINK and core-type subclasses with TYPE_UCLASS; a replacement must account for both behaviors to preserve supported object structures.

Related reading