--- title: "Latest reads" description: "The engineering internet, summarised so you can actually read it." --- # Latest reads > The engineering internet, summarised so you can actually read it. ## Articles ### [BERT 101 - State Of The Art NLP Model Explained](https://yomu.fyi/post/bert-101-state-of-the-art-nlp-model-explained.md) - Company: huggingface.co - Author: Britney Muller - Published: Mar 2, 2022 Developed in 2018 by Google AI Language, Bidirectional Encoder Representations from Transformers addresses the historical challenge of machines lacking contextual understanding of human language. The model relies on an encoder-only Transformer architecture pre-trained on a 3.3-billion-word corpus consisting of Wikipedia and Google BooksCorpus. Training simultaneously combines masked language modeling, which hides 15% of tokenized words to enforce bidirectional context learning, with next sentence prediction across balanced sentence pairs. Pre-trained on Cloud TPUs over four days, BERT unifies solutions for more than eleven common NLP tasks and can be fine-tuned on task-specific annotated data within minutes. Unmasking experiments demonstrate that the model can also inherit distinct societal and gender biases from its underlying training corpora when predicting professions. ### [Abacus - Issuing points for multiple sources](https://yomu.fyi/post/abacus-issuing-points-for-multiple-sources.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Chandrakanth - Published: Mar 1, 2022 Grab needed a centralised points management architecture to issue loyalty points across a growing catalog of products, membership tiers, and external partner exchanges. To address this, the engineering team built Abacus, an issuance platform designed to process millions of daily transactions with high availability. The system ingests completed transaction streams or API calls, dynamically computes points via configured multipliers, and passes calculations through Amazon Simple Queue Service queues. Once the Point Awarding module updates a persistent ledger, Abacus notifies consumers, emits events to Kafka for downstream consumers, and recalculates rolling point expiration dates. ### [Supabase Beta January 2022](https://yomu.fyi/post/supabase-beta-january-2022.md) - Company: [Supabase](https://yomu.fyi/company/supabase.md) - Author: Paul Copplestone - Published: Feb 22, 2022 Supabase announced product updates, platform improvements, and community resources released during January 2022. The platform expanded authentication options by adding Notion and LinkedIn OAuth support alongside Vonage and Textlocal SMS providers for one-time passwords. Observability tooling gained SQL querying capabilities for logs, timestamp filtering, and expanded time spans across project usage charts to accelerate issue diagnosis. Database infrastructure advanced with the release of pg\_graphql version 0.1.0 introducing SQL Comment Directives, while PostgREST replaced pg\_listen with built-in schema reloading and automated compute-based connection pool scaling. To streamline customer assistance, a priority selector was added to dashboard support forms with urgent tiers for paid projects. ### [Exposing a Kafka Cluster via a VPC Endpoint Service](https://yomu.fyi/post/exposing-a-kafka-cluster-via-a-vpc-endpoint-service.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Fabrice Harbulot - Published: Feb 18, 2022 To replace VPC peering and reduce attack surfaces, Grab exposed a multi-Availability Zone Apache Kafka cluster in its main AWS VPC to clients in a separate GrabKios VPC using AWS VPC Endpoint Service. Because Kafka requires clients to establish deterministic connections to individual brokers, the team configured a Network Load Balancer with unique TCP ports and dedicated target groups for each broker alongside a shared bootstrap port. They added custom listeners on the Kafka brokers to advertise endpoints using private Route 53 CNAMEs rather than raw interface hostnames. To eliminate unnecessary cross-AZ network latency and data transfer costs, the architecture was refined to advertise AZ-specific private CNAMEs mapped directly to zonal endpoint interfaces. ### [How Grab built a scalable, high-performance ad server](https://yomu.fyi/post/how-grab-built-a-scalable-high-performance-ad-server.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Anthony McCallum - Published: Feb 11, 2022 Grab transitioned from an off-the-shelf MVP to an in-house ad serving system to accommodate business scale, hyperlocal requirements, and machine learning personalization. The architecture orchestrates core microservices and data pipelines across sequential steps: targeting, capping, pacing, scoring, ranking, pricing, and tracking. ElasticSearch serves as the targeting ads repository, while ScyllaDB acts as the high-throughput stats store fed by Kafka streams and data pipelines. The system operates on key engineering principles including parallelization and tuned latency limits, graceful fallbacks for slow dependency calls, and a unified server serving all ad types across the superapp. ### [Fine-Tune ViT for Image Classification with šŸ¤— Transformers](https://yomu.fyi/post/fine-tune-vit-for-image-classification-with-transformers.md) - Company: huggingface.co - Author: Nate Raw - Published: Feb 11, 2022 Vision Transformer models bring transformer architectures to computer vision by splitting images into grids of sub-image patches and projecting them into token sequences. To classify healthy and diseased leaves using the beans dataset, the Hugging Face datasets and transformers libraries enable streamlined data ingestion and model fine-tuning. Preprocessing relies on ViTImageProcessor paired with lazy on-the-fly dataset transforms to dynamically generate normalized pixel tensors. Training leverages ViTForImageClassification with the Trainer API, requiring remove\_unused\_columns set to False so raw image data is preserved for batch collation. Over four training epochs, fine-tuning the google/vit-base-patch16-224-in21k checkpoint achieves an evaluation accuracy of 98.5% alongside an evaluation loss of 0.0637. ### [Getting Started with Sentiment Analysis using Python](https://yomu.fyi/post/getting-started-with-sentiment-analysis-using-python.md) - Company: huggingface.co - Author: Federico Pascual - Published: Feb 2, 2022 Modern natural language processing tools allow developers to perform sentiment analysis without deep machine learning expertise. Sentiment analysis categorizes text polarity into positive, negative, or neutral labels to extract insights from large volumes of social posts, reviews, and support tickets. Using the Hugging Face Transformers library and the Hub, practitioners can quickly run pre-trained transformer pipelines or target specific multilingual and emotion-detection models with minimal Python code. For custom requirements, developers can fine-tune models like DistilBERT using the IMDB dataset via the Trainer API or train models automatically with AutoNLP. Applying these workflows to social media data enables automated sentiment distribution analysis and visualization using pandas, matplotlib, and word clouds. ### [Making automatic speech recognition work on large files with Wav2Vec2 in šŸ¤— Transformers](https://yomu.fyi/post/making-automatic-speech-recognition-work-on-large-files-with-wav2vec2.md) - Company: huggingface.co - Author: Nicolas Patry - Published: Feb 1, 2022 Transformer-based automatic speech recognition models like Wav2Vec2 crash on long audio inputs because the O(nĀ²) attention mechanism rapidly exhausts GPU memory. Simple audio chunking avoids out-of-memory errors but causes poor transcription accuracy at chunk boundaries due to a lack of surrounding context. To resolve this, Hugging Face Transformers leverages the Connectionist Temporal Classification architecture to process overlapping audio chunks with configurable strides. The pipeline discards low-quality logits at chunk edges and chains the remaining central predictions together to reconstruct a seamless transcript. This striding technique operates out of the box with language model-augmented pipelines and adapts directly to low-latency live audio streaming. ### [Supercharged Searching on the šŸ¤— Hub](https://yomu.fyi/post/supercharged-searching-on-the-hub.md) - Company: huggingface.co - Author: Zachary Mueller - Published: Jan 25, 2022 Programmatically searching the Hugging Face Hub previously required navigating web browser widgets or enduring trial and error to guess exact query string formats. The huggingface\_hub library solves this issue by introducing helper utilities such as ModelSearchArguments, DatasetSearchArguments, and ModelFilter alongside HfApi. These namespace helpers translate accessible Python attributes into the formatted parameters expected by the backend API, covering datasets, tasks, and libraries. For complex queries across multiple tasks, frameworks, and datasets, developers can combine criteria inside ModelFilter instances and retrieve matching model metadata via api.list\_models. Under the hood, the library uses AttributeDictionary, a data structure inspired by fastcore that enables nested tab-completion while supporting standard dictionary key indexing for special characters. ### [Welcome Stable-baselines3 to the Hugging Face Hub šŸ¤—](https://yomu.fyi/post/welcome-stable-baselines3-to-the-hugging-face-hub.md) - Company: huggingface.co - Author: Thomas Simonini - Published: Jan 21, 2022 Hugging Face announced an official integration with Stable-Baselines3, a popular PyTorch library for training and testing Deep Reinforcement Learning agents across diverse environments like Gym, Atari, MuJoco, and Procgen. The integration enables researchers and developers to host their saved reinforcement learning checkpoints on the Hugging Face Hub and download pre-trained community models. Interacting with the Hub requires installing the huggingface\_hub and huggingface\_sb3 packages, which supply helper methods for authentication, downloading, and uploading. Practitioners can retrieve checkpoint zip files using the load\_from\_hub function by providing the target repository identifier and filename before loading them into Stable-Baselines3 algorithms. Furthermore, users authenticated via CLI or notebook login can train policies such as PPO and publish their saved zip files to the Hub using push\_to\_hub. ### [Supabase Beta December 2021](https://yomu.fyi/post/supabase-beta-december-2021.md) - Company: [Supabase](https://yomu.fyi/company/supabase.md) - Author: Paul Copplestone - Published: Jan 20, 2022 Supabase announced a series of platform updates, educational content, and community milestones for December 2021. PostgreSQL instances now include the pg\_sodium extension, enabling users to perform encryption, decryption, hashing, and cryptographic signing directly within database queries and functions through the Supabase Dashboard. The release details educational video guides demonstrating remote procedure calls from JavaScript, invoking external HTTP endpoints from database functions, and setting up triggers to run SQL on table modifications. In addition to ecosystem updates featuring new Python library releases and integrations with Divjoy and n8n, the project surpassed 26,000 GitHub stars and opened fully remote job positions across engineering, marketing, and human resources. ### [Biometric authentication - Why do we need it?](https://yomu.fyi/post/biometric-authentication-why-do-we-need-it.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Chad Burgess - Published: Jan 20, 2022 Grab addressed the vulnerabilities and costs associated with SMS one-time passwords and PINs by implementing device-level biometric authentication. The architecture pairs device biometric sensors with hardware secure enclaves to protect private keys separately from the main operating system. During enrollment, Grab generates a public-private key pair using SHA512withECDSA, authenticates the user locally, and stores reference identifiers in encrypted device storage. HellfireSDK verifies that the device is not rooted, ensuring raw biometric data never leaves the handset. Early experimental runs indicate an adoption rate exceeding 90% and a login success rate near 90%. ### [Case Study: Millisecond Latency using Hugging Face Infinity and modern CPUs](https://yomu.fyi/post/case-study-millisecond-latency-using-hugging-face-infinity-and-modern.md) - Company: huggingface.co - Author: Philipp Schmid, Jeff Boudier, Morgan Funtowicz - Published: Jan 13, 2022 Deploying large Transformer models at scale often faces severe prediction latency bottlenecks, driving up infrastructure costs and limiting real-time production use cases. To address this challenge, Hugging Face evaluated Infinity, a containerized hardware-optimized inference solution paired with the Infinity Multiverse model optimization service. Testing covered 192 configurations on Amazon EC2 C6i instances powered by 3rd generation Intel Xeon Scalable processors across varying CPU cores, sequence lengths, and batch sizes. The benchmarks demonstrated that an Ice Lake-optimized DistilBERT container achieved up to 800% higher throughput than vanilla Transformers and delivered 1 to 4 millisecond end-to-end latencies for sequence lengths up to 64 tokens. Although Infinity was later discontinued in favor of Inference Endpoints and Optimum libraries, the results demonstrated substantial efficiency gains on modern CPU hardware. ### [Boosting Wav2Vec2 with n-grams in šŸ¤— Transformers](https://yomu.fyi/post/boosting-wav2vec2-with-n-grams-in-transformers.md) - Company: huggingface.co - Author: Patrick von Platen - Published: Jan 12, 2022 Wav2Vec2 models fine-tuned with Connectionist Temporal Classification transcribe speech without external language models, but decoding can still suffer from spelling inaccuracies. Hugging Face Transformers addressed this by integrating Kensho Technologies' pyctcdecode library to support decoding with n-gram language models. Instead of decoding using simple argmax operations over logits, the Wav2Vec2ProcessorWithLM class feeds full probability matrices into beam search guided by KenLM n-gram probabilities. KenLM's build\_binary utility compresses language model ARPA files into binary formats, reducing file size by more than half for faster loading and hub deployment. In Swedish xls-r-300m-sv benchmarks on Common Voice 7, this 5-gram boosted decoding setup achieved an 18.85% word error rate, delivering an approximate 30% relative performance gain. ### [Deploy GPT-J 6B for inference using Hugging Face Transformers and Amazon SageMaker](https://yomu.fyi/post/deploy-gpt-j-6b-for-inference-using-hugging-face-transformers-and-amaz.md) - Company: huggingface.co - Author: Philipp Schmid - Published: Jan 11, 2022 Deploying EleutherAI's 6 billion parameter GPT-J model for production inference presents latency hurdles due to large memory footprints and lengthy startup times. Loading the model via standard methods takes up to several minutes, conflicting with strict real-time response limits such as Amazon SageMaker's 60-second threshold. To overcome this limitation, the model is serialized using PyTorch's native save mechanisms, packaged into a compressed archive with supporting assets, and stored on Amazon S3. This alternative loading workflow reduces GPT-J load times down to 7.7 seconds. An Amazon SageMaker real-time endpoint is then deployed on an NVIDIA T4 GPU instance using the Hugging Face Inference Toolkit. ### [Active Learning with AutoNLP and Prodigy](https://yomu.fyi/post/active-learning-with-autonlp-and-prodigy.md) - Company: huggingface.co - Author: Abhishek - Published: Dec 23, 2021 Active learning requires iteratively adding labeled data, retraining models, and serving them to end users. Building such pipelines often demands substantial effort in data labeling, model selection, hyperparameter tuning, and training infrastructure. The author demonstrates a low-code active learning pipeline using Explosion's Prodigy for entity annotation and Hugging Face's AutoNLP for automatic training and evaluation. After first training a news categorization model achieving 98.67% accuracy on Kaggle's BBC News dataset, the author manually annotated named entities across iterative batches. An export script converted annotations to JSONL with IOB tags, showing progressive metric improvements from 20 samples to 250 samples, where the token classification model attained 95.9% accuracy, 0.73 precision, and 0.79 recall. ### [Gradio is joining Hugging Face!](https://yomu.fyi/post/gradio-is-joining-hugging-face.md) - Company: huggingface.co - Author: Abubakar Abid - Published: Dec 21, 2021 Hugging Face has acquired Gradio, the open-source machine learning library designed for building and sharing interactive model interfaces. Gradio originated in 2019 when its founder struggled to share a medical computer vision model with a physician collaborator who did not write Python. Co-founded alongside Ali Abdalla, Ali Abid, and Dawood Khan, the project expanded from computer vision into text, speech, and video modalities. Over 300,000 demos have been built using Gradio, allowing interdisciplinary industry teams and researchers to debug models internally and showcase them externally to non-technical users. The acquisition unites Gradio with Hugging Face to broaden browser-based machine learning accessibility and expand hiring efforts across the joint team. ### [Holiday Hackdays Winners 2021](https://yomu.fyi/post/holiday-hackdays-winners-2021.md) - Company: [Supabase](https://yomu.fyi/company/supabase.md) - Author: Thor Schaeff - Published: Dec 17, 2021 Supabase organized the Holiday Hackdays 2021 hackathon following their launch week and announced the winning and runner-up community submissions. Selected winners include Swappy.one by Zernonia, the realtime polling platform rtPoll by Emilio and Federico Schepis, Santa Banter for holiday jokes by Andy Keogh, and the wishlist app the get list by glowdexapp. Recognized runners-up include Chivel for YouTube channel landing pages, the end-to-end encrypted e2ee-chat, a Flutter-based Holiday Sweater voting app, and a greeting card platform built by high school hackers. Winners receive limited-edition gold medal shirts, while runners-up earn silver medal shirts. All project submissions were made available for viewing on madewithsupabase.com. ### [Supabase Beta November 2021: Launch Week Recap](https://yomu.fyi/post/supabase-beta-november-2021-launch-week-recap.md) - Company: [Supabase](https://yomu.fyi/company/supabase.md) - Author: Ant Wilson - Published: Dec 15, 2021 Supabase celebrated its third Launch Week and one year in beta by rolling out several major infrastructure and tooling updates. The company open-sourced its Dashboard for self-hosting and integrated Logflare to provide searchable database and API logs directly in the interface. For database querying and APIs, Supabase introduced a Postgres extension that resolves each GraphQL request with a single SQL statement to minimize network IO overhead, while upgrading its default hosted database to PostgreSQL 14 and deploying PostgREST 9.0. Security capabilities were expanded by enabling Row Level Security policies across the Realtime API to restrict streams and subscriptions on a per-user basis. Additional releases during the week included faster media file delivery, an open-sourced launch methodology, and a free course combining Next.js, Stripe, and Supabase. ### [Perceiver IO: a scalable, fully-attentional model that works on any modality](https://yomu.fyi/post/perceiver-io-a-scalable-fully-attentional-model-that-works-on-any-moda.md) - Company: huggingface.co - Author: Niels Rogge - Published: Dec 15, 2021 Standard Transformer architectures scale poorly in compute and memory because pairwise dot-product self-attention depends quadratically on input size. Perceiver IO addresses this constraint by computing self-attention across a small set of latent variables rather than directly on high-dimensional inputs. Inputs and outputs interact with the model via cross-attention operations, decoupling compute and memory costs from input and output dimensions. Integrated into Hugging Face Transformers via the PerceiverModel class, the architecture supports diverse data types using optional preprocessors, decoders, and postprocessors. Experiments demonstrate competitive performance across text, multimodal video classification, 3D point cloud classification on ModelNet40, and StarCraft II reinforcement learning in AlphaStar. [Newer posts](https://yomu.fyi/page/58.md) Ā· [Older posts](https://yomu.fyi/page/60.md)