Loading…
How to Export Datadog Metrics for Exploration in Jupyter Notebooks
2023-10-18
- Source
- Shopify
- Published
- Added to Yomu
Summary
Datadog dashboards can constrain metric analysis because they offer a limited set of visualizations and lack tooling for complex work such as statistical modeling. They also aggregate data over wider time ranges: the examples contrast one-second metrics across 15 minutes with two-hour intervals across 30 days, potentially hiding interesting events. The guide uses Datadog’s REST API and Python in Jupyter notebooks, requiring an API key, an APP key, and a metric query such as CPU utilization over time. It defines a time range, splits it into buckets whose width is controlled by time_delta, requests each window in a loop, appends the results, and converts them into a dataframe. The resulting data can be examined with tools such as seaborn; a KDE plot is used to inspect the distribution of system CPU utilization, with the exported data offering greater granularity than the dashboard example.
Context
Datadog provides limited visualization options and lacks tooling for complex analysis such as statistical modeling. Its fixed-width dashboards also show less granular data over longer time ranges, where aggregation can smooth or hide interesting events.
Approach / What changed
Use Datadog’s REST API from Python in a Jupyter notebook. Provide an API key, an APP key, and a metric query; divide the requested time range into buckets controlled by time_delta, query each bucket, collect the responses, and convert them into a dataframe for local analysis and visualization.
Takeaways
- Datadog dashboard examples show one-second metrics over 15 minutes but two-hour intervals over 30 days, reducing granularity across the longer window.
- The extraction process queries bucketed time windows and appends the returned data before converting the collected lists into a dataframe.
- Seaborn KDE plots can be applied to the exported data to examine the distribution of system CPU utilization.