---
title: "Shopify's Data Science & Engineering Foundations"
description: "Shopify’s Data Science & Engineering team describes the foundations it uses to provide daily insights to internal teams, merchants, and partners across a rapidly changing commerce ecosystem. Its data warehouse follows a shared dimensional-modelling philosophy, with modelled data built on Spark in a single GitHub repository and made broadly queryable through Presto, while ETL jobs are unit tested and prevented from failing silently. Centralized dashboards, reproducible vetted data points, peer review, product-aligned sub-teams, accessible communication, and cross-team collaboration extend those foundations into analysis and decision-making. Together, these practices let team members reuse existing work, produce trusted and understandable metrics, address the specialist gap through collaboration, and derive new metrics with fewer than 50 lines of SQL."
---

# Shopify's Data Science & Engineering Foundations

[Shopify](https://yomu.fyi/company/shopify) · 2023-10-18 · Jun 18, 2020

**Type:** Explainer

## Summary

Shopify’s Data Science & Engineering team describes the foundations it uses to provide daily insights to internal teams, merchants, and partners across a rapidly changing commerce ecosystem. Its data warehouse follows a shared dimensional-modelling philosophy, with modelled data built on Spark in a single GitHub repository and made broadly queryable through Presto, while ETL jobs are unit tested and prevented from failing silently. Centralized dashboards, reproducible vetted data points, peer review, product-aligned sub-teams, accessible communication, and cross-team collaboration extend those foundations into analysis and decision-making. Together, these practices let team members reuse existing work, produce trusted and understandable metrics, address the specialist gap through collaboration, and derive new metrics with fewer than 50 lines of SQL.

## Context

Shopify’s Data Science & Engineering team supports internal teams, merchants, and partners with daily insights in a rapidly changing commerce environment. The organization emphasizes shared foundations so data can be discovered, trusted, reused, and connected to decisions across its commerce ecosystem.

## Approach / What changed

The team uses shared dimensional modelling, a Spark-based modelling platform in one GitHub repository, company-wide Presto access, tested and observable ETL, centralized dashboards, reproducible vetted data points, peer review, product-aligned sub-teams, clear communication, cross-team task forces, and leadership engagement with data-informed recommendations.

## Takeaways

- Shopify’s shared dimensional-modelling philosophy standardizes fact and dimension tables and unresolved rows, so analysts can join dimensions without sneaky nulls silently destroying rows.
- All data pipeline jobs are unit tested for errors and edge cases, and failures generate alerts rather than being allowed to fail silently.
- Vetted data points store the original question, answer, and generating code together, with the goal that a result remains unchanged over time.

**Tags:** [Apache Spark](https://yomu.fyi/topic/apache-spark), [Data Analytics](https://yomu.fyi/topic/data-analytics), [Data Pipelines](https://yomu.fyi/topic/data-pipelines)

- Source: [Shopify](https://shopify.engineering/shopifys-data-science-engineering-foundations)
- Source URL: https://shopify.engineering/shopifys-data-science-engineering-foundations
- Ingested by Yomu: 2026-08-31T01:12:27.081Z

[Read original post](https://shopify.engineering/shopifys-data-science-engineering-foundations)