Loading…
Docker lazy loading at Grab: Accelerating container startup times
GrabHuong Vuong
Summary
Grab addressed slow cold starts and auto-scaling bottlenecks caused by large container images across data platforms like Airflow and Spark Connect. To eliminate the requirement of downloading complete images before launching, the team evaluated Docker lazy loading via remote snapshotters using eStargz and Seekable OCI (SOCI). Unlike eStargz, which modifies image layers and increased application startup delays during benchmarks, SOCI stores index metadata separately as OCI Artifacts without altering image digests. In production on Amazon EKS, tuning SOCI concurrency and chunk parameters reduced fresh-node image download times by 60%, ultimately delivering a 30% to 40% reduction in P95 startup times.
Takeaways
- Benchmarking showed SOCI matched standard OverlayFS with a 5.0-second Airflow application startup time, whereas eStargz took 25.0 seconds.
- SOCI preserves existing image digests and signatures without requiring image conversions by storing file indices as separate OCI Artifacts.
- Tuning SOCI snapshotter parameters for Amazon ECR—increasing concurrent downloads and unparks to 10 and chunk size to 16MB—lowered fresh-node download times from 60 to 24 seconds.
Related reading
Grab ·
The evolution of Grab's machine learning feature store
Grab redesigned its initial machine learning feature store, Amphawa, to address high-dimensional data, complex entity retrieval, and versioning challenges during feature updates. The new architecture adopts a feature-table model where data scientists output Parquet datasets to Amazon S3 using Spark, which are then atomically ingested into Amazon Aurora PostgreSQL via a reverse ETL workflow. To prevent noisy-neighbor contention and optimize infrastructure costs, the platform utilizes Aurora's distributed storage to separate reads from writes. Grab pairs Aurora Serverless on writer nodes to scale up during daily batch ingestion with Provisioned instances on read replicas for steady serving traffic.
Daniel TaiGrab ·
Machine-learning predictive autoscaling for Flink
Managing resource allocations for expanding Apache Flink stream-processing workloads often leads to over-provisioning or unstable reactive scaling. Reactive autoscaling with Kubernetes Horizontal Pod Autoscaling triggers severe restart spikes because pipelines must reprocess backlogged records from checkpoints, inducing feedback loops and scaling fluctuations. To prevent these spirals, a predictive vertical autoscaling system uses upstream Kafka message throughput as an independent workload metric. The architecture forecasts future workload using time-series models and maps the projected throughput to required TaskManager CPU allocations through a trained regression model. A custom controller vertically scales TaskManager CPU resources before traffic changes arrive, avoiding restart-induced latency spikes.
Minh Nhat NguyenGrab ·
SpellVault’s evolution: Beyond LLM apps, towards the agentic future
Grab developed SpellVault as an internal no-code platform to democratize the creation of AI applications backed by Retrieval-Augmented Generation (RAG) and plugin integrations. To advance beyond static retrieval and linear input-output processing, the platform transitioned from its legacy executor to a graph-based execution model supporting branching, looping, and ReAct agent patterns. Capabilities like Python code execution and internal repository searching were unbundled from the prompt builder and consolidated alongside user plugins into unified Native and Community Built Tools. The platform also introduced a drag-and-drop deterministic workflow designer, automated task scheduling, and support for the Model Context Protocol (MCP).
Felix Haryanto LieGrab ·
Cursor at Grab: Adoption and impact
Following a multi-tool AI strategy, Grab integrated the AI coding assistant Cursor into its engineering toolkit in late 2024 to accelerate software development. Technical staff adoption reached 98% monthly active usage with a 50% suggestion acceptance rate, supported by custom monorepo indexing and preconfigured rules aligned with internal coding conventions. Engineers frequently apply the tool to unit test generation, code refactoring, cross-repository navigation, and routine API scaffolding, with over a third of merge requests incorporating Cursor. The rollout also encompasses non-technical personnel and product designers who, after receiving Git training, submit direct production UI fixes. Statistical evaluations using fixed-effects regression indicate a dose-response relationship between Cursor usage intensity and measurable productivity gains.
Akshay Misra