---
title: "Machine Learning"
description: "622 posts about Machine Learning, summarised, each linking to the original."
---

# Machine Learning
> 622 posts about Machine Learning, summarised, each linking to the original.

## Articles

### [Large Language Models: A New Moore's Law?](https://yomu.fyi/post/large-language-models-a-new-moore-s-law.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Julien Simon
- Published: Oct 26, 2021
- Type: Problem & solution

Recent advancements in generative artificial intelligence have spurred the rapid growth of large language models like Megatron-Turing NLG 530B, which require vast financial investments and substantial energy consumption. Training these massive parameter architectures demands hundreds of multi-GPU servers and generates significant carbon footprints for relatively modest benchmark gains. Rather than relying on brute-force scaling, engineering teams can adopt smaller, more frugal architectures through knowledge distillation and transfer learning techniques. Practical alternatives include fine-tuning existing pretrained models, deploying on energy-efficient cloud infrastructure, and applying optimizations like pruning, layer fusion, and quantization. These methods deliver low-latency inference and high task accuracy while significantly decreasing hardware requirements, development time, and environmental impact.


### [Train a Sentence Embedding Model with 1B Training Pairs](https://yomu.fyi/post/train-a-sentence-embedding-model-with-1b-training-pairs.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Antoine SIMOULIN
- Published: Oct 25, 2021
- Type: Announcement

Sentence embeddings map sentences into dense vectors to capture semantic meaning for downstream tasks such as clustering, asymmetric question answering, and text search. To train general-purpose models at scale, the team utilized a dataset compiled from multiple sources containing up to one billion sentence pairs. Training leveraged contrastive learning with Multiple Negative Ranking Loss and in-batch negatives, running across seven TPU v3-8 accelerators with JAX and Flax. Batches combined samples from at least two datasets to balance hard negatives within topics against broader cross-topic semantic structure. The project resulted in twenty trained models—including variants of RoBERTa, DistilBERT, and MPNet—alongside eight specialized evaluation datasets and an interactive demo covering sentence similarity, search, and gender bias.


### [The Age of Machine Learning As Code Has Arrived](https://yomu.fyi/post/the-age-of-machine-learning-as-code-has-arrived.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Julien Simon
- Published: Oct 20, 2021
- Type: Explainer

Recent findings from the 2021 State of AI Report and Kaggle State of Machine Learning and Data Science Survey indicate that machine learning is expanding into critical infrastructure while Transformers become general-purpose architectures across text, vision, and audio. In response, organizations face questions about scaling infrastructure, team composition, and engineering maturity. Rather than treating machine learning as isolated sandbox experiments or hiring solely data scientists, teams benefit from adopting established software engineering and DevOps principles like versioning, testing, automation, and continuous deployment. Furthermore, the rise of pre-trained Transformer architectures enables practitioners to fine-tune existing off-the-shelf models rather than training from scratch, reducing compute costs and training duration. Tools from platforms such as Hugging Face streamline model deployment, latency optimization, and infrastructure abstraction.


### [Fine tuning CLIP with Remote Sensing (Satellite) images and captions](https://yomu.fyi/post/fine-tuning-clip-with-remote-sensing-satellite-images-and-captions.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Arto, Dev Vidhani, Goutham, Mayank Bhaskar, Ritobrata Ghosh, Sujit Pal
- Published: Oct 13, 2021
- Type: Problem & solution

Pre-trained vision-language models like CLIP generalize well across everyday imagery but struggle with remote sensing domain characteristics. To enable text-based search across satellite imagery, a distributed team fine-tuned the openai/clip-vit-base-patch32 model on TPU hardware using JAX and Flax during the Hugging Face Flax/JAX Community Week. The training pipeline incorporated satellite image and caption pairs from the RSICD, UCM, and Sydney datasets alongside extensive image transforms and backtranslation-based text augmentation. Contrastive optimization using Adam with a learning rate of 5e-6 achieved top-1 retrieval accuracy of 0.883 compared to 0.572 on the baseline model. An interactive demo uses a NMSLib index for text-to-image matching, image-to-image matching, and patch-level feature detection.


### [Winning AI4TSP: Solving the Travelling Salesperson Problem with Self-programming Machines](https://yomu.fyi/post/winning-ai4tsp-solving-the-travelling-salesperson-problem-with-self-pr.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 6, 2021
- Type: Benchmark

The 2021 AI4TSP Competition addressed the Travelling Salesperson Problem when travel times are sampled from probability distributions and locations carry rewards and time windows. With n locations, the number of possible tours is n!, making exhaustive enumeration impractical, while the competition also required decisions under uncertain travel outcomes. One track required a fixed tour optimized for expected performance across scenarios; the other allowed an adaptive policy to select the next location as time elapsed, with results evaluated across many instances and realizations. The Shopify-led team used dialectic search for the fixed-tour track and deep reinforcement learning for the on-the-fly track, and won first prize in both.


### [Hosting your Models and Datasets on Hugging Face Spaces using Streamlit](https://yomu.fyi/post/hosting-your-models-and-datasets-on-hugging-face-spaces-using-streamli.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: merve
- Published: Oct 5, 2021
- Type: Tutorial

Streamlit enables developers to create interactive web demonstrations and visualizations for machine learning models and datasets. In an illustrative text completion interface recreating Write with Transformer, UI components such as text areas, sidebars, number inputs, and continuous sliders collect user text and inference hyperparameters. For data inspection, the streaming feature in Hugging Face Datasets loads large corpora directly into memory without requiring complete local downloads, integrating with pandas, matplotlib, and seaborn for tabular and chart rendering. Deploying these interactive applications to Hugging Face Spaces involves uploading the code files while defining dependencies inside a requirements.txt file. Matching local Streamlit versions ensures consistent runtime execution when hosting models and datasets online.


### [Showcase Your Projects in Spaces using Gradio](https://yomu.fyi/post/showcase-your-projects-in-spaces-using-gradio.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: merve
- Published: Oct 5, 2021
- Type: Tutorial

Demonstrating machine learning projects to the community requires accessible hosting and minimal interface boilerplate. Gradio integrates directly with the Hugging Face Hub and its underlying Inference API, enabling developers to load models using repository identifiers and launch interactive interfaces with a few lines of code. Hugging Face Spaces provides a free hosting environment where users select Gradio as the SDK, commit an app.py script, and deploy live web demonstrations for frameworks like Transformers, spaCy, SpeechBrain, and Asteroid. For complex workflows, developers can chain multiple models sequentially via Gradio Series or host custom checkpoints directly when models lack native Inference API support.


### [Summer at Hugging Face](https://yomu.fyi/post/summer-at-hugging-face.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: system
- Published: Sep 24, 2021
- Type: Announcement

Hugging Face released several major updates across its ecosystem, expanding its public Hub repository to over 16,000 models. Platform additions include Spaces for deploying Gradio and Streamlit demo applications, automatic TensorBoard instances, and evaluation metric tracking integrated with Papers With Code leaderboards. The Transformers library gained JAX/Flax support across more than 5,000 models, improved TensorFlow implementations, and introduced the transformers.onnx export module for model conversion. In research, the BigScience project completed large-scale training of a 13-billion-parameter English decoder model on Jean Zay, while the DeDLOC method enabled training the sahajBERT Bengali model without HPC infrastructure. Accepted conference papers detailed the Datasets library, prompt data-point equivalence, prompt-based fine-tuning heuristics, and block pruning techniques that reduced BERT size by 74 percent.


### [Hugging Face and Graphcore partner for IPU-optimized Transformers](https://yomu.fyi/post/hugging-face-and-graphcore-partner-for-ipu-optimized-transformers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Sally Doherty
- Published: Sep 14, 2021
- Type: Announcement

Hugging Face launched its Hardware Partner Program at the 2021 AI Hardware Summit to offer device-optimized models and software integrations. As a founding member of the program, Graphcore collaborated with Hugging Face to optimize Transformer models for its Intelligence Processing Unit (IPU). Graphcore IPUs use a massively parallel MIMD architecture with on-die ultra-high bandwidth memory designed for AI workloads featuring fine-grained parallelism, low precision arithmetic, and sparsity. Through Hugging Face's open-source Optimum toolkit, developers will receive plug-and-play access to certified, hardware-optimized models across vision, speech, translation, and text generation. Graphcore's Poplar SDK integrates with PyTorch, TensorFlow, Docker, and Kubernetes, simplifying the process of porting workloads to IPU-POD datacenter compute systems.


### [Introducing Optimum: The Optimization Toolkit for Transformers at Scale](https://yomu.fyi/post/introducing-optimum-the-optimization-toolkit-for-transformers-at-scale.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Morgan Funtowicz, Ella Charlaix, Michael Benayoun, Jeff Boudier
- Published: Sep 14, 2021
- Type: Announcement

Running Transformer models fast and efficiently at production scale presents a significant challenge due to the complex compatibility between model architectures, acceleration techniques like quantization or sparsity, and specific silicon hardware features. Hugging Face introduced Optimum, an open-source library built to abstract hardware acceleration complexity and optimize training and inference on target devices through direct collaborations with hardware partners. Using quantization as a primary example, engineering teams often struggle with eager-mode model graph modifications, kernel compatibility checks, calibration parameter tuning, and acceptable accuracy loss trade-offs. Optimum addresses these challenges by integrating tools such as Intel Neural Compressor, which supports post-training quantization, quantization-aware training, and dynamic quantization driven by user-defined YAML configurations. Through these hardware-tailored integrations, the library aims to democratize production performance and reduce the aggregate energy consumed by machine learning workloads.


### [Using Rich Image and Text Data to Categorize Products at Scale](https://yomu.fyi/post/using-rich-image-and-text-data-to-categorize-products-at-scale.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Sep 8, 2021
- Type: Explainer

Shopify revisited product categorization as its platform expanded to millions of merchants selling billions of products across diverse industries. The model uses product titles, descriptions, types, vendors, collections, tags, and images, converting them into embeddings with Multilingual BERT and MobileNet-V2 before hierarchical classification across the seven-level Google Product Taxonomy. Each taxonomy level is treated as a separate multi-class task, with parent predictions fed into child predictions and weighted losses combined during training; confidence thresholds filter predictions at each level. The upgraded system increased leaf precision by 8% while almost doubling coverage, although thresholds require balancing accuracy, coverage, merchant experience, and performance in sensitive categories, and future work includes addressing data imbalance and adding merchant-level features.


### [5 Steps for Building Machine Learning Models for Business](https://yomu.fyi/post/5-steps-for-building-machine-learning-models-for-business.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Aug 25, 2021
- Type: Explainer

The article presents a five-step workflow for deciding when and how to build machine learning products in a business setting. It recommends delaying machine learning for new products, establishing data-collection foundations, and testing simple heuristics as baselines before investing in a model. For an initial model, teams should favor simple algorithms, limited feature sets, and off-the-shelf solutions where appropriate, then measure business-aligned impact before optimizing. The workflow also calls for planned model iteration and explicit comparison with existing versions, while accounting for prediction stability when changes could affect user trust. The authors say they used these practices at Shopify to help build and scale machine learning products and to productionize models intended to drive impact.


### [Open Source Hackathon Winners](https://yomu.fyi/post/open-source-hackathon-winners.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Ant Wilson
- Published: Aug 9, 2021
- Type: Announcement

Supabase announced the winners and runners-up for its inaugural open-source hackathon after concluding deliberations across a larger-than-expected pool of entries. Top entries included Work From, an employee schedule tracking application, and Fireplace, a video streaming platform that leverages Supabase Storage to synchronize playback events across multiple viewers. Additional recognized projects encompassed Feature.so for modular application component management, Flutter Add-ons for telemetry and crash reporting, and Peckish Peach, which uses machine learning to generate recipes from input ingredients. All submitted projects remain fully open source for community inspection, contribution, and forking. Gold and silver prize winners receive custom limited-edition Supabase apparel, while every hackathon participant receives a participation prize.


### [Protecting Personal Data in Grab's Imagery](https://yomu.fyi/post/protecting-personal-data-in-grab-s-imagery.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Adrian Popovici
- Published: Jul 26, 2021
- Type: Problem & solution

Grab's KartaView platform collects geotagged street imagery across over 100 countries, requiring automated obfuscation of faces and licence plates to protect personal privacy. Because off-the-shelf solutions struggled with diverse global environments and equirectangular 360-degree camera formats, Grab built a custom machine learning pipeline. The system projects varied image formats into standardized planar views, applies a YOLOv4 object detection model to locate target regions, and transforms bounding coordinates back to the original imagery for blurring. Training the detector required iterative dataset updates to accommodate edge cases like face masks and mirror reflections, paired with offline view splitting and oversampling of scarce large bounding boxes. Assessments confirmed that obfuscating these regions had minimal negative impact on downstream map feature extraction services.


### [Deep Learning over the Internet: Training Language Models Collaboratively](https://yomu.fyi/post/deep-learning-over-the-internet-training-language-models-collaborative.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Max Ryabinin, Lucile Saulnier
- Published: Jul 15, 2021
- Type: Problem & solution

Pretraining state-of-the-art Transformer language models typically requires vast computing resources and expensive hardware clusters that are inaccessible to many researchers. Standard distributed deep learning struggles over public Internet infrastructure because consumer connections suffer from limited bandwidth, dynamic peer disconnections, and restrictive firewalls. To resolve these networking bottlenecks, Distributed Deep Learning in Open Collaborations (DeDLOC) introduces an adaptive training framework that accumulates massive batch sizes across heterogeneous volunteer devices before applying optimizer updates. The framework splits gradient vectors dynamically based on participant connection speeds and allows firewalled nodes to contribute data without performing aggregation. In a practical deployment with 40 volunteers, the 18-million-parameter sahajBERT model achieved Bengali downstream benchmark performance comparable to much larger multilingual models trained on dedicated supercomputers.


### [Welcome spaCy to the Hugging Face Hub](https://yomu.fyi/post/welcome-spacy-to-the-hugging-face-hub.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Omar Sanseviero, Ines Montani
- Published: Jul 13, 2021
- Type: Announcement

Hugging Face has integrated spaCy pipelines into the Hugging Face Hub to simplify the distribution, testing, and deployment of natural language processing models across the community. More than 60 canonical models from the spaCy 3.1 release are hosted in the spaCy organization, alongside discoverable community pipelines. Users can package and upload models directly through the command line using the spacy-huggingface-hub extension, which auto-generates required metadata and model cards. Hosted pipelines offer direct pip installation from live wheel URLs, interactive browser exploration, and production-ready HTTP endpoints through the Inference API. The platform provides immediate interactive widget support for named entity recognition components, with upcoming support planned for text classification and part-of-speech tagging pipelines.


### [Deploy Hugging Face models easily with Amazon SageMaker](https://yomu.fyi/post/deploy-hugging-face-models-easily-with-amazon-sagemaker.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Philipp Schmid
- Published: Jul 8, 2021
- Type: Announcement

Hugging Face and Amazon introduced an inference solution integrating Hugging Face Transformers with Amazon SageMaker. The SageMaker Hugging Face Inference Toolkit and specialized Deep Learning Containers enable deployment of trained models or publicly available Hub models to managed production endpoints. Users configure endpoints with minimal code using the SageMaker Python SDK, standard pipelines, or custom inference scripts that override default behaviors. The architecture supports models stored in Amazon S3 as well as direct references via HF\_MODEL\_ID and HF\_TASK environment variables. Deployments benefit from native AWS infrastructure features, including built-in monitoring, Identity and Access Management permission controls, and Virtual Private Cloud connectivity.


### [Reshaping Chat Support for Our Users](https://yomu.fyi/post/reshaping-chat-support-for-our-users.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Elisa Monacchi
- Published: Jul 7, 2021
- Type: Problem & solution

Grab transitioned from voice hotlines and third-party tools to an in-house native chat support system integrated into their CRM. The team validated the platform through an MVP and user shadowing to address session disconnections, agent context switching, and routing bottlenecks. To optimize support operations at scale, they introduced dynamic queue limits based on Little's law, machine learning autocomplete suggestions for agents, and duration timers with visual nudges. These enhancements reduced chat waiting times by 30%, unresponsive users by 7%, and overall chat handling duration by 22%.


### [Sentence Transformers in the Hugging Face Hub](https://yomu.fyi/post/sentence-transformers-in-the-hugging-face-hub.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Omar Sanseviero, Nils Reimers
- Published: Jun 28, 2021
- Type: Announcement

Sentence Transformers has collaborated with the Hugging Face Hub as part of its v2 release to make sentence, paragraph, and image embedding models readily accessible. The Hub now hosts over 90 pretrained models supporting more than 100 languages for use cases like semantic search and multilingual zero-shot classification. Alongside direct model loading in minimal code, the platform provides interactive Hub widgets for feature extraction and sentence similarity computation. Developers can execute inference programmatically through Inference API endpoints. Additionally, the library supports direct model uploading to the Hub via save\_to\_hub, which automatically generates a model card detailing layer architecture and dual usage with Sentence Transformers and Transformers.


### [Few-shot learning in practice: GPT-Neo and the 🤗 Accelerated Inference API](https://yomu.fyi/post/few-shot-learning-in-practice-gpt-neo-and-the-accelerated-inference-ap.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Philipp Schmid
- Published: Jun 3, 2021
- Type: Tutorial

Limited labeled data often prevents machine learning models from achieving high performance during traditional fine-tuning. Few-shot learning addresses this hurdle in natural language processing by supplying a short task description and a few demonstration examples at inference time. EleutherAI developed GPT-Neo, an open transformer-based model trained on the Pile dataset, which can execute few-shot tasks via the Hugging Face Accelerated Inference API. Compared to larger models like GPT-3, the 2.7B parameter version of GPT-Neo requires three to four examples alongside tuned hyperparameters like temperature and stopping sequences to generate accurate completions. Because few-shot learning relies heavily on pre-trained associations, careful output monitoring and human feedback mechanisms remain necessary to prevent automated bias.


[Newer posts](https://yomu.fyi/topic/machine-learning/page/28.md) · [Older posts](https://yomu.fyi/topic/machine-learning/page/30.md)