Introducing on-demand CPU and memory profiling with flamegraphs for Workers and Durable Objects

Summary

Cloudflare introduced on-demand CPU and memory profiling for Workers and Durable Objects, accessible via the CLI or the Dashboard’s Observability page. Users can generate interactive flamegraphs and download pprof files, with each rectangle’s width representing CPU time or memory usage over a chosen duration. Profiling the R2 binding Worker revealed that genericR2JsonReplacer, called by JSON.stringify, processed nested values multiple times; fixing it made the function 2.7x faster. A different Worker with P999 memory at 133 MB (above the 128 MB limit) was found to have a partially enabled Prometheus code path accounting for 66.7% of allocations; disabling it reduced P999 memory to 118 MB and eliminated out-of-memory errors. The profiling system locates the correct data center and isolate for the Worker or Durable Object, and continuous profiling is being developed to automatically capture samples without manual initiation.

Takeaways

1. 1

   The on-demand profiler captures CPU or heap samples of a specific Worker version or Durable Object in production and displays them as interactive flamegraphs.

2. 2

   Profiling the R2 binding Worker identified that genericR2JsonReplacer processed nested values multiple times; fixing it made the function 2.7x faster.

3. 3

   A Worker with memory issues had a partially enabled Prometheus code path consuming 66.7% of allocations; removing it reduced P999 memory from 133 MB to 118 MB and eliminated out-of-memory errors.

FAQ