# Our Journey to Continuous Delivery at Grab (Part 2)

[Grab](https://yomu.fyi/company/grab) · Sylvain Bougerel · May 10, 2021

**Type:** Problem & solution

## Summary

Conveyor, an in-house continuous delivery system at Grab, introduces hermetic deployments by tracking application code alongside static and dynamic configuration parameters. This hermeticity guarantees that production releases use combinations of versioned artifacts previously verified in staging, preventing rollback incompatibilities. Conveyor replaces single multi-environment pipelines with decoupled pipelines, while automating cluster locking, deployment slot scheduling, release note generation, and canary monitoring with automated rollbacks. These automations reduced production deployment failure rates from 1.5% to an average of 0.3% over a three-month period. Grab also doubled the volume of production changes between 2018 and 2020 while saving more than 5,000 man-days of engineering effort in 2020.

## Context

Rollback incompatibilities between application versions and dynamic configurations caused incidents, while manual deployment tasks and multi-environment pipelines created engineering toil.

## Approach / What changed

Grab introduced hermetic deployments into its custom continuous delivery platform, Conveyor, tracking code, static config, and dynamic config as versioned artifacts while automating locks, deployment window scheduling, canary monitoring, and rollbacks.

## Takeaways

- Conveyor enforces deployment hermeticity by tracking application code, static configuration, and dynamic configuration to ensure production rollouts only use parameter combinations previously tested in staging.
- Decoupling staging from production pipelines reduced engineer toil, as fewer than 20% of staging changes advance to production, while allowing flexibility in the sequence and type of production pipelines executed.
- Automating staging and production deployment steps reduced production change failure rates from 1.5% to 0.3% and saved over 5,000 man-days of engineering effort across Grab in 2020.

**Tags:** [CI/CD](https://yomu.fyi/topic/ci-cd), [Deployment](https://yomu.fyi/topic/deployment), [Developer Experience](https://yomu.fyi/topic/developer-experience), [Microservices](https://yomu.fyi/topic/microservices), [Reliability](https://yomu.fyi/topic/reliability)

[Read original post](https://engineering.grab.com/our-journey-to-continuous-delivery-at-grab-part2)