Shopify2023-10-18Tutorial
ShopifyQL Notebooks: Simplifying Querying with Commerce Data Models
Summary
ShopifyQL Notebooks gives merchants a self-serve way to investigate changing business metrics, understand their causes, and share findings as data stories. It combines the ShopifyQL query language with commerce-specific data models designed for users without SQL expertise, covering domains such as products, orders, and customers. The modeling process starts with business questions, translates them into columns and an appropriate grain, validates the design with mock data and queries, and evaluates upstream data, freshness, and performance. To support large merchants, the implementation uses incremental models and roll-ups, which can reduce build and query times while adding maintenance, pipeline, and monitoring complexity.
Context
Business-savvy merchants need to answer questions spanning different business processes without SQL expertise, while the underlying datasets may have mismatched granularity and large volumes that can affect query speed.
Approach / What changed
The data-modeling approach begins with stakeholder-driven business questions, derives the required columns and grain, tests the design with mock data and ShopifyQL queries, discovers upstream data, and assesses quality, freshness, and performance. Incremental models and roll-up tables are used to reduce processing and query costs.
Takeaways
Fully additive metrics can be aggregated across all available dimensions; semi-additive metrics such as customers who placed an order can double-count entities, while non-additive metrics such as discount percentages should not be summed.
Incremental data models process only newly available differences, improving build times but making schema changes, pipeline dependencies, and monitoring more challenging than full-drop models.
Roll-ups reduce query work by storing older data at the finest required grain, such as daily sales by product title, so the engine scans fewer rows.
Source and freshness
- Source
- shopify.engineering
- Published
- Added to Yomu
- Last updated
Summaries are generated from the original by a language model. Check the original before relying on it.
Keep reading
Related reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
The best engineering reads, once a week
One useful email. Unsubscribe anytime.