---
title: "AWS"
description: "205 posts about AWS, summarised, each linking to the original."
---

# AWS
> 205 posts about AWS, summarised, each linking to the original.

## Articles

### [Deploy Hugging Face models easily with Amazon SageMaker](https://yomu.fyi/post/deploy-hugging-face-models-easily-with-amazon-sagemaker.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Philipp Schmid
- Published: Jul 8, 2021
- Type: Announcement

Hugging Face and Amazon introduced an inference solution integrating Hugging Face Transformers with Amazon SageMaker. The SageMaker Hugging Face Inference Toolkit and specialized Deep Learning Containers enable deployment of trained models or publicly available Hub models to managed production endpoints. Users configure endpoints with minimal code using the SageMaker Python SDK, standard pipelines, or custom inference scripts that override default behaviors. The architecture supports models stored in Amazon S3 as well as direct references via HF\_MODEL\_ID and HF\_TASK environment variables. Deployments benefit from native AWS infrastructure features, including built-in monitoring, Identity and Access Management permission controls, and Virtual Private Cloud connectivity.


### [Scaling-up BERT Inference on CPU (Part 1)](https://yomu.fyi/post/scaling-up-bert-inference-on-cpu-part-1.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Morgan Funtowicz
- Published: Apr 20, 2021
- Type: Benchmark

Scaling BERT-like model inference on CPUs presents practical deployment challenges as transformer architectures proliferate in production environments. Using a reproducible Hydra-based benchmarking framework, inference workloads were evaluated on an Amazon Web Services c5.metal instance featuring an Intel Xeon Platinum 8275 processor with AVX512 and VNNI support. Out-of-the-box evaluations showed PyTorch 1.8.1 delivering lower latency than Google TensorFlow 2.4.0, likely due to PyTorch relying on OpenMP and Intel MKL/oneDNN rather than Eigen. Implementing multiple inference streams by pinning parallel model instances to dedicated CPU cores yielded near-linear throughput scaling. Matching instance core allocations to specific problem sizes also demonstrated that smaller workloads can achieve optimal latency on fewer cores, enabling up to a sixfold reduction in cloud compute costs.


### [Introducing 🤗 Accelerate](https://yomu.fyi/post/introducing-accelerate.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Sylvain Gugger
- Published: Apr 16, 2021
- Type: Announcement

🤗 Accelerate enables PyTorch developers to execute raw training loops across CPUs, multi-GPU setups, and TPUs with minimal boilerplate modifications. Standard distributed training typically demands manual device assignments, DistributedSampler configurations, and divergent code paths that break single-device portability. By wrapping models, optimizers, and dataloaders through a unified prepare call, Accelerate automatically manages hardware placement, mixed precision operations, and sampler batch slicing without requiring custom distributed samplers. Distributed evaluation is streamlined using a gather utility that consolidates prediction tensors across active worker processes. Furthermore, a dedicated command-line interface provides an interactive questionnaire to store runtime configuration defaults and orchestrate local or AWS SageMaker runs.


### [Distributed Training: Train BART/T5 for Summarization using 🤗 Transformers and Amazon SageMaker](https://yomu.fyi/post/distributed-training-train-bart-t5-for-summarization-using-transformer.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Philipp Schmid
- Published: Apr 8, 2021
- Type: Tutorial

Hugging Face and Amazon SageMaker introduced optimized Deep Learning Containers alongside a SageMaker Python SDK estimator to streamline distributed transformer model training. This workflow demonstrates fine-tuning the 400-million-parameter facebook/bart-large-cnn model for dialogue summarization on the 16,000-conversation samsum dataset. Using SageMaker Data Parallelism enabled in the distribution configuration, training executes across two ml.p3dn.24xlarge compute instances equipped with sixteen total GPUs and a cumulative batch size of 64. The HuggingFace estimator pulls the run\_summarization.py script directly from GitHub, enables mixed precision with fp16, and tracks evaluation performance through ROUGE scores. Following completion, the unzipped model artifacts and a generated model card are pushed to the Hugging Face Hub using the huggingface\_hub library for hosted inference.


### [Storage is now available in Supabase](https://yomu.fyi/post/storage-is-now-available-in-supabase.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Inian Parameshwaran
- Published: Mar 30, 2021
- Type: Announcement

Supabase launched Storage, adding a scalable object store to its existing Postgres, authentication, and API services. Existing open-source storage servers like Ceph, Swift, Minio, and Zenko were evaluated but rejected due to auth incompatibilities and external dependencies like etcd, MongoDB, and Kafka. Consequently, the team implemented a custom Storage API server built with Fastify and TypeScript behind the Kong gateway. Object metadata and access control reside directly in Postgres, leveraging Row Level Security policies written in SQL rather than a proprietary domain-specific language. Objects stream directly to managed backends like AWS S3 using Node streams with minimal in-memory buffering, accompanied by a default one-hour Cache-Control header.


### [The Partnership: Amazon SageMaker and Hugging Face](https://yomu.fyi/post/the-partnership-amazon-sagemaker-and-hugging-face.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Philipp Schmid
- Published: Mar 23, 2021
- Type: Announcement

Hugging Face and Amazon formed a strategic partnership establishing Amazon Web Services as Hugging Face's preferred cloud provider to simplify training and deploying Transformer models. To facilitate this collaboration, the teams introduced Hugging Face Deep Learning Containers optimized for PyTorch and TensorFlow in Amazon SageMaker. The integration provides an extension to the SageMaker Python SDK, enabling data science teams to configure distributed training jobs across single-node and multi-node GPU clusters. Workflows integrate directly with SageMaker Automatic Model Tuning, SageMaker Studio, and Amazon S3 for managed experiment tracking and artifact storage. The open-source containers allow trained models to be exported for inference via SageMaker or published directly to the Hugging Face Model Hub while utilizing EC2 Spot instances to reduce compute expenses.


### [How Grab Leveraged Performance Marketing Automation to Improve Conversion Rates by 30%](https://yomu.fyi/post/how-grab-leveraged-performance-marketing-automation-to-improve-convers.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Sc Ng
- Published: Mar 22, 2021
- Type: Problem & solution

Grab faced operational bottlenecks managing direct-response Google Ads campaigns across thousands of ad groups due to its hyperlocal marketing across Southeast Asian markets. To eliminate the manual burden of tracking and updating ad creatives, the team built CARA, an in-house automation tool deployed on AWS serverless compute. CARA utilizes standardized file naming conventions to map assets to specific campaigns and connects with Google Ads and YouTube APIs to detect and replace low-performing assets. During an experimental rollout across more than 8,000 active ad groups, CARA replaced nearly 2,000 underperforming creatives. The automated asset replacement workflow produced an 18% to 30% increase in clickthrough and conversion rates.


### [Supabase Beta January 2021](https://yomu.fyi/post/supabase-beta-january-2021.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Feb 2, 2021
- Type: Announcement

Supabase released a collection of platform updates during January 2021 covering database tools, authentication integrations, regional availability, and infrastructure performance. Client libraries added support for PostgREST exact, planned, and estimated count queries to eliminate counting friction commonly experienced in platforms like Firebase. Authentication capabilities expanded to include Facebook and Azure login providers, an in-dashboard GoTrue audit trail, a helper function for user emails, and a prebuilt React Auth Widget. The platform also enabled new database hosting regions in London and Sydney while transitioning storage to GP3 for improved throughput. Furthermore, migrating API subdomains to Route53 alongside custom Let's Encrypt certificates yielded read performance benchmark gains of up to twelve percent.


### [Optimally Scaling Kafka Consumer Applications](https://yomu.fyi/post/optimally-scaling-kafka-consumer-applications.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Shubham Badkur
- Published: Oct 13, 2020
- Type: Problem & solution

Grab's Coban platform runs Golang-based stream processing pipelines on Kubernetes, servicing roughly 400 billion events weekly from Kafka. The initial Horizontal Pod Autoscaler setup caused resource waste and uneven load distribution across Kafka partitions during scale-in and scale-out events. To resolve this, Grab moved to a fixed pod count matching the topic's partition count and adopted Vertical Pod Autoscaling, reducing resource usage versus requests by approximately 45%. The team also introduced Kubernetes priority classes to segment latency-sensitive workloads onto On-Demand nodes and non-critical jobs onto Spot instances. Additionally, overprovisioning via low-priority placeholder pods managed by Cluster Proportional Autoscaler enabled rapid pod rescheduling and reduced deployment delays.


### [Securing and Managing Multi-cloud Presto Clusters with Grab’s DataGateway](https://yomu.fyi/post/securing-and-managing-multi-cloud-presto-clusters-with-grab-s-datagate.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Vinnson Lee
- Published: Aug 24, 2020
- Type: Problem & solution

Grab's data engineering team needed to manage data access across growing Presto workloads spanning hundreds of users, thousands of tables, and multiple clouds without exposing infrastructure churn to clients. Out-of-the-box Presto lacked fine-grained schema- and table-level access controls, unified endpoint routing, and seamless cluster switching. To solve this, the team built DataGateway, a smart HTTP proxy and abstraction layer deployed as microservices on Kubernetes. The platform parses incoming SQL statements, verifies user access control lists against target schemas and tables, and dynamically routes queries to appropriate AWS or Azure Presto clusters without requiring endpoint updates. This architecture enabled zero-disruption cluster migrations across 40 clusters and unified access management across a multi-cloud data lake environment.


### [Continuous PostgreSQL Backups using WAL-G](https://yomu.fyi/post/continuous-postgresql-backups-using-wal-g.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Angelico de los Reyes
- Published: Aug 2, 2020
- Type: Tutorial

Continuous PostgreSQL backups require coordinating physical base snapshots with ongoing write-ahead log archiving to support point-in-time recovery. The open-source WAL-G utility simplifies this workflow by managing backup pushes and fetches to cloud storage services such as Amazon S3. In a typical setup on Ubuntu with PostgreSQL 12, envdir supplies AWS credentials and storage prefixes to WAL-G commands embedded directly in the PostgreSQL archive configuration. Regular base backups can be automated via cron jobs, which minimizes the volume of WAL archives that must be replayed during a restore. When recovering to a new instance, administrators fetch the latest base backup, define a restore command with targeted recovery timestamps in the configuration, and initiate replay using a recovery signal file.


### [The Journey of Deploying Apache Airflow at Grab](https://yomu.fyi/post/the-journey-of-deploying-apache-airflow-at-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Chandulal Kavar
- Published: Jul 14, 2020
- Type: Problem & solution

Engineering and data teams across Grab originally operated independent Apache Airflow instances, causing duplicate maintenance overhead and frequent job failures around scaling, logging, and dependency management. To resolve this, a dedicated team developed a centralized orchestration platform that runs isolated, containerized Airflow instances per team on Amazon EKS. The platform categorizes deployments into three size tiers and provisions dedicated Redis brokers, RDS metadata stores, and Vault secret sidecars using Terraform and custom Helm charts. Teams customize container images using shared GitLab CI/CD templates, while worker scaling is handled via Kubernetes Horizontal Pod Autoscalers. Today, the platform runs roughly 20 Airflow instances executing between 1,000 and 60,000 daily jobs per instance.


### [Alpha Launch Postmortem](https://yomu.fyi/post/alpha-launch-postmortem.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Jul 10, 2020
- Type: Incident / postmortem

An unexpected viral submission on Hacker News prompted sudden alpha traffic for Supabase, attracting 30,000 visitors and over 1,400 signups in one week. The surge strained initial infrastructure, which initially relied on a single server running Docker Compose for middleware and Digital Ocean for customer databases. System bottlenecks emerged rapidly, including hit limits on Digital Ocean droplet counts, frequent physical server faults, and a three-hour provisioning outage when Cloudflare reached its 1,000-subdomain limit. In response, the team migrated middleware to multiple globally load-balanced ECS clusters via AWS Global Accelerator. Supabase also transitioned 1,800 servers from Digital Ocean to AWS t3a instances, nearly halving database startup times from 90 seconds to 50 seconds.


### [A Lean and Scalable Data Pipeline to Capture Large Scale Events and Support Experimentation Platform](https://yomu.fyi/post/a-lean-and-scalable-data-pipeline-to-capture-large-scale-events-and-su.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Oscar Cassetti
- Published: Jan 16, 2019
- Type: Problem & solution

Controlled online experimentation across diverse product verticals requires tracking interactions across systems to prevent local optimizations from causing global degradation. Grab built a batch data pipeline to capture, ingest, and process petabytes of event data to support its experimentation platform and analytics stakeholders. The architecture loads ingested event data from Amazon S3, transforms and sorts it, and writes partitioned output back to S3 with metadata registered in Apache Hive. Using Apache Spark on AWS Elastic MapReduce with Apache Airflow for orchestration, the system handles roughly 400,000 incoming events per second. The data is partitioned by event type and ingestion time and stored in Apache ORC format to streamline query workloads and reduce retrieval overhead.


### [A House of Cards: An Exploration of Security When Building Docker Containers](https://yomu.fyi/post/a-house-of-cards-an-exploration-of-security-when-building-docker-conta.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Etienne Stalmans
- Published: Mar 8, 2018
- Type: Incident / postmortem

The post examines how a Docker-based, multi-tenant build environment can expose untrusted code when layered isolation is misconfigured. It traces a setup using AWS CodeBuild, Docker-in-Docker, and per-step build containers, then tests directory traversal through configurable Dockerfile paths before focusing on daemon networking. The vulnerability comes from starting dockerd with tcp://0.0.0.0:2375 and placing containers on a shared default network; build code can reach the daemon, issue Docker commands, mount the host filesystem, and retrieve container-managed AWS credentials. The excerpt reports a reverse shell, access to the CodeBuild filesystem, and temporary credentials whose impact depends on IAM permissions, while noting that the build must remain alive. The stated fix is to remove the TCP listener and retain the mapped Unix socket.


### [Dealing with the Meltdown Patch at Grab](https://yomu.fyi/post/dealing-with-the-meltdown-patch-at-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Althaf Hameez
- Published: Jan 7, 2018
- Type: Incident / postmortem

AWS infrastructure maintenance related to Meltdown patches led to severe CPU utilization spikes across Grab's ElastiCache Redis instances. Because Redis is single-threaded, spikes past 50% CPU on two-vCPU instances threatened service capacity, and initial Multi-AZ failovers only provided temporary relief until the new master nodes received rolling patches. To handle the increased overhead before their peak traffic window, the engineering team horizontally scaled both clustered and non-clustered Redis fleets. For Redis 3.2.4 clusters lacking live re-sharding support, they provisioned larger clusters, warmed caches, and redirected traffic. Non-clustered workloads were resolved by provisioning extra nodes, migrating compatible services to Redis Cluster, or updating application code to shard data across multiple instances.


### [Migrating Existing Datastores](https://yomu.fyi/post/migrating-existing-datastores.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Nishant Gupta
- Published: Aug 8, 2017
- Type: Problem & solution

Grab's Identity team faced imminent memory exhaustion on a single Redis node used to cache mobile authentication tokens under rapid user growth. Because read traffic outweighed write traffic by roughly 200 times, the team opted for an AWS ElastiCache cluster with three shards and two read replicas per shard. They executed a zero-downtime, six-phase migration plan while handling a peak load of 20,000 queries per second. The migration transitioned through initial one-time data replication, asynchronous shadow writes, synchronous dual writes, asynchronous read validation, switching primary reads, and final write cleanup. Controlled by feature flags and monitored with metrics at every stage, the migration completed without invalidating tokens or causing service disruptions.


### [DNS Resolution in Go and Cgo](https://yomu.fyi/post/dns-resolution-in-go-and-cgo.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Ryan Law
- Published: May 24, 2017
- Type: Problem & solution

Go applications experiencing load balancing issues across AWS Elastic Load Balancer (ELB) nodes trace uneven traffic distribution to IP address sorting defined in RFC 6724. Comparing Go's native DNS resolver with Cgo and glibc's getaddrinfo shows that both initially sort destination addresses using Rule 9 longest matching prefix rules. Disabling IPv6 on the network interface causes C and Cgo resolvers to return IP addresses in randomized order, while the native Go resolver continues deterministic sorting. Examination of net/addrselect.go reveals that Go's native resolver implements only a subset of the RFC rules and omits dynamic source address selection. Achieving permanent parity requires modifying the Go source code directly.


### [Driving Southeast Asia Forward with AWS](https://yomu.fyi/post/driving-southeast-asia-forward-with-aws.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Arul Kumaravel
- Published: May 21, 2017
- Type: Explainer

Grab transitioned its transportation platform from a single Ruby on Rails monolith on Amazon EC2 and Amazon RDS MySQL to a microservices architecture hosted on Amazon Web Services. The platform processes multi-petabyte real-time data flows and hundreds of millions of GPS data points to match drivers with passengers and push proactive demand heat maps. Operational efficiency is sustained with fewer than ten full-time infrastructure engineers, supported by AWS managed services. The analytics backend also transitioned from MySQL to Amazon Redshift, eventually moving to an Amazon S3 data lake using Amazon EMR and Presto. These data-driven matching systems improved driver-passenger allocation rates by up to 30%.


### [Troubleshooting Unusual AWS ELB 5XX Error](https://yomu.fyi/post/troubleshooting-unusual-aws-elb-5xx-error.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Dharmarth Shah
- Published: May 10, 2017
- Type: Problem & solution

Grab experienced intermittent HTTP 5XX alerts when its Gothena service sent driver location updates to the Astrolabe service through an AWS Elastic Load Balancer (ELB). CloudWatch metrics revealed that requests were failing to reach healthy backend instances because of an uneven load distribution favoring a single ELB node in one Availability Zone. The team verified that Route 53 was properly using Alias records and ruled out OS-level DNS caching since Linux does not cache DNS queries by default. Connection inspection with netstat across multiple Go services confirmed a heavily skewed distribution of connections toward specific ELB IP addresses. Comparative tests with cURL, tcpdump, Go, Python, and Ruby in an isolated environment demonstrated that Go reused connections across requests while other runtimes opened new connections per request.


[Newer posts](https://yomu.fyi/topic/aws/page/9.md) · [Older posts](https://yomu.fyi/topic/aws/page/11.md)