---
title: "Latest reads"
description: "The engineering internet, summarised so you can actually read it."
---

# Latest reads
> The engineering internet, summarised so you can actually read it.

## Articles

### [How Grab Leveraged Performance Marketing Automation to Improve Conversion Rates by 30%](https://yomu.fyi/post/how-grab-leveraged-performance-marketing-automation-to-improve-convers.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Sc Ng
- Published: Mar 22, 2021

Grab faced operational bottlenecks managing direct-response Google Ads campaigns across thousands of ad groups due to its hyperlocal marketing across Southeast Asian markets. To eliminate the manual burden of tracking and updating ad creatives, the team built CARA, an in-house automation tool deployed on AWS serverless compute. CARA utilizes standardized file naming conventions to map assets to specific campaigns and connects with Google Ads and YouTube APIs to detect and replace low-performing assets. During an experimental rollout across more than 8,000 active ad groups, CARA replaced nearly 2,000 underperforming creatives. The automated asset replacement workflow produced an 18% to 30% increase in clickthrough and conversion rates.


### [Fine-Tune Wav2Vec2 for English ASR in Hugging Face with 🤗 Transformers](https://yomu.fyi/post/fine-tune-wav2vec2-for-english-asr-in-hugging-face-with-transformers.md)
- Company: huggingface.co
- Author: Patrick von Platen
- Published: Mar 12, 2021

Wav2Vec2 is a pretrained Automatic Speech Recognition model that learns speech representations from over 50,000 hours of unlabeled audio using a contrastive objective and masked feature vectors. The model can be fine-tuned end-to-end on labeled speech datasets using Connectionist Temporal Classification without requiring an external language model. To demonstrate this process, a base-sized checkpoint is fine-tuned on the Timit dataset, which contains five hours of training data. Data preparation involves normalizing transcriptions to lowercase, removing special punctuation characters, and configuring both a specialized feature extractor and tokenizer. Evaluating the fine-tuned acoustic model without padding on the Timit test dataset yields a word error rate of 22.1%.


### [Using Supabase in Replit](https://yomu.fyi/post/using-supabase-in-replit.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Ant Wilson
- Published: Mar 11, 2021

Replit provides a collaborative, browser-based coding environment, but applications requiring structured relational queries or larger datasets need external database infrastructure. Developers can integrate a Node.js repl directly with a hosted Supabase relational database without provisioning or managing server instances. The setup process involves importing the @supabase/supabase-js client library and authenticating using the project URL alongside the anonymous API key obtained from the Supabase dashboard settings. Once initialized, developers can populate sample datasets like Countries and execute read queries using standard promise handlers or async/await syntax. The client library also supports diverse query filtering methods such as .or(), .is(), .in(), and .neq() to manipulate relational datasets directly in JavaScript.


### [Hugging Face Reads, Feb. 2021 - Long-range Transformers](https://yomu.fyi/post/hugging-face-reads-feb-2021-long-range-transformers.md)
- Company: huggingface.co
- Author: Victor Sanh
- Published: Mar 9, 2021

Standard transformer models face severe memory and compute bottlenecks due to the quadratic scaling of self-attention with sequence length. To scale beyond short sequence limits, efficient architectures introduce varied mechanisms such as custom sparse attention patterns, compressed recurrence, low-rank projections, and kernel approximations. Longformer integrates local dilated windowed attention with task-specific global tokens, functioning as a drop-in replacement that enables standard pre-trained models to process long inputs without full retraining. Concurrently, Compressive Transformers extend temporal memory by compressing discarded activations, while models like Linformer and Performer reduce computational complexity through projection and kernel estimation. Progress in long-range modeling also reveals that incrementally training models from short to longer sequences accelerates convergence and improves downstream performance.


### [Toad, a link shortener with simple APIs for low-coders](https://yomu.fyi/post/toad-a-link-shortener-with-simple-apis-for-low-coders.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Published: Mar 8, 2021

Supabase compiles an overview of customer case studies documenting how organizations deploy its platform across production environments. Multiple companies report infrastructure migrations, including Firecrawl switching from Pinecone to Supabase Vector for PostgreSQL vector embeddings and Mobbin moving 200,000 users from Firebase. Engineering teams document operational gains and cost efficiencies, such as Shotgun achieving an 83% reduction in data infrastructure expenses and Quilia saving 75% in development time using Data API and RLS features. Other organizations highlight infrastructure performance improvements, including Good Tape recording a 60% cost reduction alongside database efficiency gains after migrating to Supabase managed Postgres and Authentication. Teams also report accelerated implementation speeds, with Replenysh delivering OTP functionality in less than 24 hours and Xendit releasing a full solution to production within one week.


### [Postgres as a CRON Server](https://yomu.fyi/post/postgres-as-a-cron-server.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Mar 5, 2021

Postgres databases can schedule and execute external webhook calls periodically by combining the pg\_cron and pgsql-http extensions. When configured, pg\_cron schedules recurring tasks using standard cron syntax directly inside the database engine. In conjunction, the pgsql-http extension dispatches HTTP requests such as GET, POST, PATCH, and DELETE to external endpoints within SQL queries. Because pg\_cron relies on background workers that start one process per server loop iteration, scheduled jobs do not interfere with core postmaster tasks or external client connections. Administrators can monitor job status, review execution logs, and unschedule active tasks directly through dedicated cron schema tables and helper functions.


### [Supabase Beta February 2021](https://yomu.fyi/post/supabase-beta-february-2021.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Mar 2, 2021

Supabase published its February 2021 beta update marking one year of development as an open-source Firebase alternative. The release introduces dashboard interface improvements across the Table view, Auth section, and SQL Editor, which now features autocomplete suggestions for tables and columns. Authentication updates allow user redirection to specific site routes upon invoking signIn and signUp, while a new database management interface lets users configure Postgres replication settings and events for the realtime API. Infrastructure additions include a new deployment region in South Africa and a Docker Compose configuration intended to power local development in the forthcoming CLI. Documentation expanded with resources for GoTrue and Google OAuth, alongside ongoing RFCs covering connection pooling and local workflows.


### [Simple considerations for simple people building fancy neural networks](https://yomu.fyi/post/simple-considerations-for-simple-people-building-fancy-neural-networks.md)
- Company: huggingface.co
- Author: Victor Sanh
- Published: Feb 25, 2021

Building and training neural networks often presents frustrating debugging challenges because implementations can contain subtle errors while still running without crashes and achieving decent performance. The author shares a mental framework derived from natural language processing research to systematically construct and debug deep learning workflows. Practitioners should begin with thorough qualitative and quantitative dataset analysis before implementing simple baseline models to establish performance benchmarks. When coding the network, developers should verify implementation correctness by overfitting a tiny batch without regularization and tracking training dynamics such as losses, parameters, and gradients. Hyperparameter optimization should prioritize understanding underlying component behaviors through targeted random grid searches rather than launching hundreds of blind runs.


### [One Small Step Closer to Containerising Service Binaries](https://yomu.fyi/post/one-small-step-closer-to-containerising-service-binaries.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Stan Halka
- Published: Feb 23, 2021

Engineering teams at Grab initiated a transition to containerized microservices to standardize environments, enhance security, and decouple services from internal runtime tooling. During this migration, developers noticed that statically-linked Go service binaries were reaching bloated sizes over 100 MB. By analyzing the binaries using the open-source tool go-binsize-viz alongside the Go nm toolchain, the team visualized compiled symbols as interactive treemaps. This analysis revealed that 11 MB of unused message format symbols were being pulled in because a generic interface shared a directory with auto-generated streaming code. Restructuring the packages to isolate interfaces from generated code successfully decreased the binary size down to 78 MB.


### [Retrieval Augmented Generation with Huggingface Transformers and Ray](https://yomu.fyi/post/retrieval-augmented-generation-with-huggingface-transformers-and-ray.md)
- Company: huggingface.co
- Author: system
- Published: Feb 10, 2021

Huggingface Transformers added the Retrieval Augmented Generation architecture to combine standard sequence-to-sequence models with external document retrieval for knowledge-intensive tasks. Scaling RAG fine-tuning across multiple GPUs previously created bottlenecks because the torch.distributed implementation forced the single rank 0 worker to handle all retrieval index lookups. To resolve this synchronization bottleneck, Ray was integrated into the contextual document retrieval mechanism using stateful actor abstractions. Dedicated Ray processes separate from the training workers now load the large index and process retrieval requests concurrently. This architectural separation delivers up to a 2x speedup per retrieval call and removes the strict dependency on PyTorch for training.


### [Roboflow.com choose Supabase to power Paint.wtf leaderboard](https://yomu.fyi/post/roboflow-com-choose-supabase-to-power-paint-wtf-leaderboard.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Rory Wilding
- Published: Feb 9, 2021

Brad Dwyer and Erik Dunteman created Paint.wtf to experiment with OpenAI's CLIP model, allowing users to draw images evaluated against text prompts and scored on a leaderboard. While the team initially considered Firebase, the platform lacked built-in document counting functionality essential for their leaderboard architecture. To avoid implementing custom workarounds or exporting data to external systems, they selected Supabase to leverage PostgreSQL's native counting capabilities. After launching on Hacker News, Reddit, and Product Hunt, the platform experienced viral traffic that peaked at over two drawing submissions per second and surpassed one hundred thousand participating users within twenty-four hours. The PostgreSQL-backed infrastructure maintained reliable performance throughout the surge, delivering accurate and up-to-date score rankings under heavy load.


### [Hugging Face on PyTorch / XLA TPUs](https://yomu.fyi/post/hugging-face-on-pytorch-xla-tpus.md)
- Company: huggingface.co
- Author: ds, Lysandre
- Published: Feb 9, 2021

PyTorch / XLA integrates Cloud TPU training into Hugging Face transformers while preserving the standard Trainer interface. The implementation adds an xla device type and uses lazy execution to trace intermediate representation graphs before lowering them to XLA Higher Level Operations for compilation. Parallel data loading through MpDeviceLoader overlaps CPU tracing with TPU execution, while xm.optimizer\_step coordinates gradient consolidation across data-parallel replicas. Checkpoints use xm.save to transfer tensors via CPU storage and restrict disk writes to the master process. Benchmarks on a Cloud TPU v3-8 system training bert-large-uncased on WikiText-103 demonstrate completion times of 178.4 minutes in FP32 precision and 106.4 minutes in BF16 precision.


### [Customer Support Workforce Routing](https://yomu.fyi/post/customer-support-workforce-routing.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Suman Anand
- Published: Feb 5, 2021

Grab replaced its third-party customer support routing software with an in-house workforce routing system for Livechat to gain better priority controls, bespoke configurations, and deeper analytics. The platform separates requests into distinct priority and business queues, using parallel workers that spend varied time slices dequeuing higher-priority issues like safety concerns. To prevent request starvation, workers operate out of sync across queue priority levels while dynamic queue limits cap incoming volume based on agent availability and performance. The system routes requests through an intermediate Agent Group layer, calculating eligibility scores from proficiency and concurrency metrics while managing per-agent locks to prevent over-allocation.


### [Supabase Beta January 2021](https://yomu.fyi/post/supabase-beta-january-2021.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Feb 2, 2021

Supabase released a collection of platform updates during January 2021 covering database tools, authentication integrations, regional availability, and infrastructure performance. Client libraries added support for PostgREST exact, planned, and estimated count queries to eliminate counting friction commonly experienced in platforms like Firebase. Authentication capabilities expanded to include Facebook and Azure login providers, an in-dashboard GoTrue audit trail, a helper function for user emails, and a prebuilt React Auth Widget. The platform also enabled new database hosting regions in London and Sydney while transitioning storage to GP3 for improved throughput. Furthermore, migrating API subdomains to Route53 alongside custom Let's Encrypt certificates yielded read performance benchmark gains of up to twelve percent.


### [Serving Driver-partners Data at Scale Using Mirror Cache](https://yomu.fyi/post/serving-driver-partners-data-at-scale-using-mirror-cache.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Indrajit Sarkar
- Published: Jan 26, 2021

Grab's Drivers Data service handles up to 10,000 requests per second during peak hours to supply driver information across backend microservices. The original setup used MySQL with Redis and standalone in-memory local caches, but yielded a low 25% local cache hit rate due to traffic patterns characterized by high burst frequency for individual drivers alongside redundant database calls across nodes. To solve this, the team developed Mirror Cache, an in-memory caching system that pairs Dgraph's Ristretto library with an asynchronous gRPC replication layer to mirror updates across cluster nodes. The replicator batches updates within the same AWS availability zone and forwards data to single nodes across zones to minimize transfer overhead. Production deployment increased the in-memory cache hit rate to approximately 75% and reduced direct MySQL queries by 5%.


### [Faster TensorFlow models in Hugging Face Transformers](https://yomu.fyi/post/faster-tensorflow-models-in-hugging-face-transformers.md)
- Company: huggingface.co
- Author: Julien Plu
- Published: Jan 26, 2021

Hugging Face updated Transformers to improve the computational performance and robustness of its TensorFlow models, including BERT, RoBERTa, ELECTRA, and MPNet. In version 4.2.0, the optimized BERT implementation runs up to approximately 10 percent faster than Google's official implementation and twice as fast as Transformers release 4.1.1 when tested on a V100 GPU. The release also improves TensorFlow SavedModel generation by allowing flexible sequence lengths across runs, exposing all model inputs, and grouping hidden states or attention outputs into single tensors. In addition, users can customize serving signatures by subclassing models and decorating custom methods with explicit input specifications. Models exported to the SavedModel format can be packaged into Docker containers and served via TensorFlow Serving using either HTTP REST or gRPC APIs for production inference.


### [Fit More and Train Faster With ZeRO via DeepSpeed and FairScale](https://yomu.fyi/post/fit-more-and-train-faster-with-zero-via-deepspeed-and-fairscale.md)
- Company: huggingface.co
- Author: Stas Bekman
- Published: Jan 19, 2021

Machine learning model sizes frequently outpace consumer GPU memory, preventing practitioners from loading or fine-tuning large architectures on standard hardware. To address these resource constraints, the Hugging Face Trainer introduced experimental support for Zero Redundancy Optimizer features from DeepSpeed and FairScale starting in transformers v4.2.0. In dual-GPU benchmarks with a t5-large model, FairScale and DeepSpeed integrations expanded allowable batch sizes and shortened training durations compared to baseline DistributedDataParallel runs. Furthermore, DeepSpeed permitted single-GPU fine-tuning of a t5-3b model on a 24GB RTX-3090 card at batch size 20, whereas the baseline execution crashed immediately with an out-of-memory error. These optimizations manage memory allocation internally to curb memory fragmentation while requiring changes only to training scripts rather than model code.


### [The GrabMart Journey](https://yomu.fyi/post/the-grabmart-journey.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Clarisse Peralta
- Published: Jan 18, 2021

During the COVID-19 pandemic, Grab accelerated the rollout of its GrabMart on-demand grocery delivery service to meet surging regional demand. Early pilot testing within the existing GrabFood interface revealed that grocery shopping fundamentally diverges from meal ordering, as users search at an item level rather than browsing merchants. To support grocery workflows, the team restructured the user experience with item and store categories, one-click add-to-cart functionality, and scheduled delivery options. Engineering teams simultaneously updated core infrastructure across order management, driver allocation systems, and merchant Partner APIs to handle specialized mart bookings. Rollouts were validated across two main phases using the internal Grab Early Access program to catch issues before regional release.


### [How we sped up transformer inference 100x for 🤗 API customers](https://yomu.fyi/post/how-we-sped-up-transformer-inference-100x-for-api-customers.md)
- Company: huggingface.co
- Author: Nicolas Patry
- Published: Jan 18, 2021

Deploying large transformer models in production poses severe latency and scaling hurdles for machine learning engineers. Hugging Face achieved a cumulative 100x inference acceleration on its Accelerated Inference API by pairing library-level algorithmic adjustments with low-level hardware compilation. The initial 10x improvement reduces forward-pass computations, such as restricting attention calculations to the latest token in GPT generation tasks, while utilizing Rust-based tokenizers and caching mechanisms. The subsequent 10x speedup requires custom static graph compilation targeted to selected CPU or GPU hardware profiles, applying unused flow removal, instruction-specific layer fusion, and precision quantization via ONNX Runtime. These coordinated techniques enable real-time prediction latency across varied model sizes and demand profiles without sacrificing task accuracy.


### [Trident - Real-time Event Processing at Scale](https://yomu.fyi/post/trident-real-time-event-processing-at-scale.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Jie Zhang
- Published: Jan 13, 2021

Trident serves as Grab's internal real-time event-processing and workflow automation engine, driving user campaigns, rewards, and notifications across multiple business lines. To handle peak loads exceeding 2,000 events per second without duplicate execution, the system consumes decoupled Kafka streams and enforces exactly-once semantics using Redis and MySQL deduplication checks. Processing efficiency relies on server autoscaling aligned with Kafka partition counts, combined with dynamic goroutine allocation per consumer. To minimize rule evaluation overhead, Trident indexes active campaigns into an in-memory hash map by event type, cutting processing time by at least 90%. Furthermore, condition evaluation is optimized through lazy loading and a weighted sorting algorithm that checks low-cost in-memory data prior to executing expensive database queries or external service calls.


[Newer posts](https://yomu.fyi/page/66.md) · [Older posts](https://yomu.fyi/page/68.md)