Securing GitOps pipelines

Summary

Grab's real-time data platform team transitioned from an Atlantis-driven Terraform workflow to an in-house GitOps platform called Khone to manage streaming infrastructure resources like Kafka topics and Flink pipelines. The earlier setup suffered from coarse-grained access controls, required manual merge request comments, and lacked flexible validation capabilities within native configuration files. Khone derives environment parameters directly from standardized directory paths and uses Python with the python-hcl2 library to inspect and validate resource definitions before executing Terraform stages in parallel. To prevent configuration tampering in merge requests, CI/CD pipeline definitions and execution scripts are isolated in a separate administrative repository and fetched during job runs using shallow Git clones.

Takeaways

1. 1

   Khone derives parameters like Kafka broker endpoints and validates inputs by encoding properties such as namespace, resource type, environment, and cluster name directly into directory paths.

2. 2

   Pipeline tampering is prevented by isolating GitLab CI configuration files and validation scripts in a separate administrative repository and shallow-cloning them during job execution.

3. 3

   HCL source code is inspected using the python-hcl2 library to validate module sources, enforce resource whitelists, and restrict resource types without relying on native HCL functions.