---
title: "Python"
description: "252 posts about Python, summarised, each linking to the original."
---

# Python
> 252 posts about Python, summarised, each linking to the original.

## Articles

### [Making automatic speech recognition work on large files with Wav2Vec2 in 🤗 Transformers](https://yomu.fyi/post/making-automatic-speech-recognition-work-on-large-files-with-wav2vec2.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Nicolas Patry
- Published: Feb 1, 2022
- Type: Problem & solution

Transformer-based automatic speech recognition models like Wav2Vec2 crash on long audio inputs because the O(n²) attention mechanism rapidly exhausts GPU memory. Simple audio chunking avoids out-of-memory errors but causes poor transcription accuracy at chunk boundaries due to a lack of surrounding context. To resolve this, Hugging Face Transformers leverages the Connectionist Temporal Classification architecture to process overlapping audio chunks with configurable strides. The pipeline discards low-quality logits at chunk edges and chains the remaining central predictions together to reconstruct a seamless transcript. This striding technique operates out of the box with language model-augmented pipelines and adapts directly to low-latency live audio streaming.


### [Supercharged Searching on the 🤗 Hub](https://yomu.fyi/post/supercharged-searching-on-the-hub.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Zachary Mueller
- Published: Jan 25, 2022
- Type: Tutorial

Programmatically searching the Hugging Face Hub previously required navigating web browser widgets or enduring trial and error to guess exact query string formats. The huggingface\_hub library solves this issue by introducing helper utilities such as ModelSearchArguments, DatasetSearchArguments, and ModelFilter alongside HfApi. These namespace helpers translate accessible Python attributes into the formatted parameters expected by the backend API, covering datasets, tasks, and libraries. For complex queries across multiple tasks, frameworks, and datasets, developers can combine criteria inside ModelFilter instances and retrieve matching model metadata via api.list\_models. Under the hood, the library uses AttributeDictionary, a data structure inspired by fastcore that enables nested tab-completion while supporting standard dictionary key indexing for special characters.


### [Welcome Stable-baselines3 to the Hugging Face Hub 🤗](https://yomu.fyi/post/welcome-stable-baselines3-to-the-hugging-face-hub.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Thomas Simonini
- Published: Jan 21, 2022
- Type: Announcement

Hugging Face announced an official integration with Stable-Baselines3, a popular PyTorch library for training and testing Deep Reinforcement Learning agents across diverse environments like Gym, Atari, MuJoco, and Procgen. The integration enables researchers and developers to host their saved reinforcement learning checkpoints on the Hugging Face Hub and download pre-trained community models. Interacting with the Hub requires installing the huggingface\_hub and huggingface\_sb3 packages, which supply helper methods for authentication, downloading, and uploading. Practitioners can retrieve checkpoint zip files using the load\_from\_hub function by providing the target repository identifier and filename before loading them into Stable-Baselines3 algorithms. Furthermore, users authenticated via CLI or notebook login can train policies such as PPO and publish their saved zip files to the Hub using push\_to\_hub.


### [Supabase Beta December 2021](https://yomu.fyi/post/supabase-beta-december-2021.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Jan 20, 2022
- Type: Announcement

Supabase announced a series of platform updates, educational content, and community milestones for December 2021. PostgreSQL instances now include the pg\_sodium extension, enabling users to perform encryption, decryption, hashing, and cryptographic signing directly within database queries and functions through the Supabase Dashboard. The release details educational video guides demonstrating remote procedure calls from JavaScript, invoking external HTTP endpoints from database functions, and setting up triggers to run SQL on table modifications. In addition to ecosystem updates featuring new Python library releases and integrations with Divjoy and n8n, the project surpassed 26,000 GitHub stars and opened fully remote job positions across engineering, marketing, and human resources.


### [Boosting Wav2Vec2 with n-grams in 🤗 Transformers](https://yomu.fyi/post/boosting-wav2vec2-with-n-grams-in-transformers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Patrick von Platen
- Published: Jan 12, 2022
- Type: Tutorial

Wav2Vec2 models fine-tuned with Connectionist Temporal Classification transcribe speech without external language models, but decoding can still suffer from spelling inaccuracies. Hugging Face Transformers addressed this by integrating Kensho Technologies' pyctcdecode library to support decoding with n-gram language models. Instead of decoding using simple argmax operations over logits, the Wav2Vec2ProcessorWithLM class feeds full probability matrices into beam search guided by KenLM n-gram probabilities. KenLM's build\_binary utility compresses language model ARPA files into binary formats, reducing file size by more than half for faster loading and hub deployment. In Swedish xls-r-300m-sv benchmarks on Common Voice 7, this 5-gram boosted decoding setup achieved an 18.85% word error rate, delivering an approximate 30% relative performance gain.


### [Deploy GPT-J 6B for inference using  Hugging Face Transformers and Amazon SageMaker](https://yomu.fyi/post/deploy-gpt-j-6b-for-inference-using-hugging-face-transformers-and-amaz.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Philipp Schmid
- Published: Jan 11, 2022
- Type: Tutorial

Deploying EleutherAI's 6 billion parameter GPT-J model for production inference presents latency hurdles due to large memory footprints and lengthy startup times. Loading the model via standard methods takes up to several minutes, conflicting with strict real-time response limits such as Amazon SageMaker's 60-second threshold. To overcome this limitation, the model is serialized using PyTorch's native save mechanisms, packaged into a compressed archive with supporting assets, and stored on Amazon S3. This alternative loading workflow reduces GPT-J load times down to 7.7 seconds. An Amazon SageMaker real-time endpoint is then deployed on an NVIDIA T4 GPU instance using the Hugging Face Inference Toolkit.


### [Active Learning with AutoNLP and Prodigy](https://yomu.fyi/post/active-learning-with-autonlp-and-prodigy.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Abhishek
- Published: Dec 23, 2021
- Type: Tutorial

Active learning requires iteratively adding labeled data, retraining models, and serving them to end users. Building such pipelines often demands substantial effort in data labeling, model selection, hyperparameter tuning, and training infrastructure. The author demonstrates a low-code active learning pipeline using Explosion's Prodigy for entity annotation and Hugging Face's AutoNLP for automatic training and evaluation. After first training a news categorization model achieving 98.67% accuracy on Kaggle's BBC News dataset, the author manually annotated named entities across iterative batches. An export script converted annotations to JSONL with IOB tags, showing progressive metric improvements from 20 samples to 250 samples, where the token classification model attained 95.9% accuracy, 0.73 precision, and 0.79 recall.


### [Gradio is joining Hugging Face!](https://yomu.fyi/post/gradio-is-joining-hugging-face.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Abubakar Abid
- Published: Dec 21, 2021
- Type: Announcement

Hugging Face has acquired Gradio, the open-source machine learning library designed for building and sharing interactive model interfaces. Gradio originated in 2019 when its founder struggled to share a medical computer vision model with a physician collaborator who did not write Python. Co-founded alongside Ali Abdalla, Ali Abid, and Dawood Khan, the project expanded from computer vision into text, speech, and video modalities. Over 300,000 demos have been built using Gradio, allowing interdisciplinary industry teams and researchers to debug models internally and showcase them externally to non-technical users. The acquisition unites Gradio with Hugging Face to broaden browser-based machine learning accessibility and expand hiring efforts across the joint team.


### [Training CodeParrot 🦜 from Scratch](https://yomu.fyi/post/training-codeparrot-from-scratch.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Leandro von Werra
- Published: Dec 8, 2021
- Type: Tutorial

Training code generation models from scratch requires careful data filtering, tokenization, and compute management. To build CodeParrot, a 1.5-billion-parameter Python generation model, engineers extracted 20 million Python files from BigQuery and cleaned the resulting 180 GB corpus down to 50 GB after discovering extreme duplication. A custom GPT-2 tokenizer was trained over streamed samples, and the GPT-2 large architecture was initialized with layer-scaled, full-precision attention mechanisms. The training pipeline leveraged Hugging Face Accelerate alongside gradient checkpointing and a streaming iterable dataset designed to yield fixed-length concatenated token sequences. CodeParrot was trained on roughly 25 to 30 billion tokens and evaluated on coding tasks using the pass@k metric.


### [Faster Python Tests](https://yomu.fyi/post/faster-python-tests.md)
- Company: [Ramp](https://yomu.fyi/company/ramp.md)
- Author: Max Lahey
- Published: Nov 30, 2021
- Type: Explainer

The title, “Faster Python Tests,” identifies test speed as the intended subject. The accompanying material consists of HTML metadata, favicon and RSS declarations, a JavaScript bundle reference, analytics setup, and a notice that JavaScript is required to run the app. It does not state which Python test framework, execution bottleneck, configuration, code change, or measurement was involved. Consequently, no implementation approach, benchmark, performance figure, or conclusion about faster tests is provided in the available text. The only technical details exposed are the page’s JavaScript application shell and its Segment and Google Analytics initialization, neither of which explains how Python tests were accelerated.


### [Getting Started with Hugging Face Transformers for IPUs with Optimum](https://yomu.fyi/post/getting-started-with-hugging-face-transformers-for-ipus-with-optimum.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Tim Santos, Julien Simon
- Published: Nov 30, 2021
- Type: Tutorial

Transformer prediction speed can hinder latency-sensitive workloads like conversational systems and search, while optimizing hardware performance requires significant effort. Hugging Face introduced Optimum, an open-source library that reduces Transformer model latency across diverse hardware targets. Through a partnership with Graphcore, BERT is introduced as the initial IPU-optimized model for parallel AI hardware. Developers configure the Poplar SDK environment and PopTorch inside Graphcloud, install optimum-graphcore, and execute the run\_qa.py script with an IPU configuration file. Fine-tuning bert-base-uncased on SQuAD v1.1 using the IPUTrainer class yields an evaluation F1 score of 88.2757 and an exact match score of 80.6623 across 10,784 evaluation samples.


### [Community Day](https://yomu.fyi/post/community-day.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Thor Schaeff
- Published: Nov 29, 2021
- Type: Announcement

Supabase kicked off its Launch Week by showcasing integrations, ecosystem updates, and community contributions across its open-source stack. The platform announced that all hosted projects are upgrading to PostgREST 9, introducing inner joins, functions with unnamed parameters, and PostgreSQL 14 compatibility. Through community contributions, Supabase expanded authentication options to include Slack, Spotify, and MessageBird alongside existing providers. Updates to the Python ecosystem include package renaming to supabase, synchronous operation support in postgrest-py, and a rewritten gotrue-py matching JavaScript feature parity. Additionally, Supabase partnered with GitGuardian to automate secret detection and remediation across development lifecycles, protecting sensitive database service role keys.


### [Introducing the Data Measurements Tool: an Interactive Tool for Looking at Datasets](https://yomu.fyi/post/introducing-the-data-measurements-tool-an-interactive-tool-for-looking.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Sasha Luccioni, Yacine Jernite, Margaret Mitchell
- Published: Nov 29, 2021
- Type: Announcement

Hugging Face released an open-source Python library and no-code interface named the Data Measurements Tool to calculate metrics for responsible dataset curation. In AI development, scraped data often enters training pipelines without detailed analysis of representation or downstream model influence, partly because querying raw datasets typically requires complex coding. Built using Hugging Face Dataset and Spaces Hubs alongside Streamlit, the tool provides descriptive, distributional, and comparison statistics. It computes missing values, vocabulary sizes, duplicate counts, label distributions, and Zipf's law adherence to identify unnatural language patterns or markup artifacts. Additionally, it offers embedding clusters via a Sentence-Transformer model and normalized pointwise mutual information metrics across gender and sexual orientation identity terms.


### [Accelerating PyTorch distributed fine-tuning with Intel technologies](https://yomu.fyi/post/accelerating-pytorch-distributed-fine-tuning-with-intel-technologies.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Julien Simon
- Published: Nov 19, 2021
- Type: Tutorial

Transfer learning makes CPU-based clusters a viable, cost-effective option for shorter deep learning fine-tuning jobs. To accelerate PyTorch workloads, clusters can leverage Intel Ice Lake architecture featuring AVX-512 and Vector Neural Network Instructions alongside specialized software optimizations. Implementing the Intel Extension for PyTorch and the oneAPI Collective Communications Library (oneCCL) addresses communication bottlenecks across distributed nodes. In demonstrated tests fine-tuning transformer models on GLUE tasks across Amazon EC2 c6i.16xlarge instances, two-node jobs delivered roughly a 1.7x speedup, while four-node setups achieved a 3x speedup. These performance libraries and multi-node configurations maintain consistent scaling across both smaller datasets like MRPC and larger benchmarks like QQP.


### [Fine-Tune XLSR-Wav2Vec2 for low-resource ASR with 🤗 Transformers](https://yomu.fyi/post/fine-tune-xlsr-wav2vec2-for-low-resource-asr-with-transformers.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Patrick von Platen
- Published: Nov 15, 2021
- Type: Tutorial

Fine-tuning the multilingual XLS-R speech model adapts pretrained cross-lingual audio representations to automatic speech recognition tasks with limited labeled data. The demonstration configures a Wav2Vec2-XLS-R-300M checkpoint using Connectionist Temporal Classification on roughly four hours of validated Turkish audio from Common Voice. Building the pipeline requires pairing a Wav2Vec2FeatureExtractor for audio signals with a custom Wav2Vec2CTCTokenizer derived from dataset transcriptions. Training with Hugging Face Trainer over 3,200 steps decreases the validation word error rate from 0.7000 down to 0.3195. While the resulting transcription demonstrates recognizable phonetic alignment, output quality can be further improved by extending training schedules, refining preprocessing, and adding language model decoding.


### [Shopify's Path to a Faster Trino Query Execution: Custom Verification, Benchmarking, and Profiling Tooling](https://yomu.fyi/post/shopify-s-path-to-a-faster-trino-query-execution-custom-verification-b.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 27, 2021
- Type: Problem & solution

Shopify needed safer ways to evaluate Trino configuration changes and software upgrades as its forked, hundreds-node cluster handled more data and data scientists depended on fast queries. The Data Reliability team built a lightweight Python library that centralizes Trino query orchestration, then exposed it through PyTest fixtures for repeatable verification, including correctness checks and candidate-versus-control comparisons. The broader framework also supports benchmarking and profiling concepts such as replaying historical queries, reproducing traffic patterns, and testing autoscaling, although the project ultimately focused most of its performance-testing scope on verification. Extensive correctness tests helped the Data Foundations team complete a successful Trino upgrade, while p95 query execution time remained stable over the reported period; the library was designed to leave room for more advanced interfaces later.


### [Fine tuning CLIP with Remote Sensing (Satellite) images and captions](https://yomu.fyi/post/fine-tuning-clip-with-remote-sensing-satellite-images-and-captions.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Arto, Dev Vidhani, Goutham, Mayank Bhaskar, Ritobrata Ghosh, Sujit Pal
- Published: Oct 13, 2021
- Type: Problem & solution

Pre-trained vision-language models like CLIP generalize well across everyday imagery but struggle with remote sensing domain characteristics. To enable text-based search across satellite imagery, a distributed team fine-tuned the openai/clip-vit-base-patch32 model on TPU hardware using JAX and Flax during the Hugging Face Flax/JAX Community Week. The training pipeline incorporated satellite image and caption pairs from the RSICD, UCM, and Sydney datasets alongside extensive image transforms and backtranslation-based text augmentation. Contrastive optimization using Adam with a learning rate of 5e-6 achieved top-1 retrieval accuracy of 0.883 compared to 0.572 on the baseline model. An interactive demo uses a NMSLib index for text-to-image matching, image-to-image matching, and patch-level feature detection.


### [Hosting your Models and Datasets on Hugging Face Spaces using Streamlit](https://yomu.fyi/post/hosting-your-models-and-datasets-on-hugging-face-spaces-using-streamli.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: merve
- Published: Oct 5, 2021
- Type: Tutorial

Streamlit enables developers to create interactive web demonstrations and visualizations for machine learning models and datasets. In an illustrative text completion interface recreating Write with Transformer, UI components such as text areas, sidebars, number inputs, and continuous sliders collect user text and inference hyperparameters. For data inspection, the streaming feature in Hugging Face Datasets loads large corpora directly into memory without requiring complete local downloads, integrating with pandas, matplotlib, and seaborn for tabular and chart rendering. Deploying these interactive applications to Hugging Face Spaces involves uploading the code files while defining dependencies inside a requirements.txt file. Matching local Streamlit versions ensures consistent runtime execution when hosting models and datasets online.


### [Showcase Your Projects in Spaces using Gradio](https://yomu.fyi/post/showcase-your-projects-in-spaces-using-gradio.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: merve
- Published: Oct 5, 2021
- Type: Tutorial

Demonstrating machine learning projects to the community requires accessible hosting and minimal interface boilerplate. Gradio integrates directly with the Hugging Face Hub and its underlying Inference API, enabling developers to load models using repository identifiers and launch interactive interfaces with a few lines of code. Hugging Face Spaces provides a free hosting environment where users select Gradio as the SDK, commit an app.py script, and deploy live web demonstrations for frameworks like Transformers, spaCy, SpeechBrain, and Asteroid. For complex workflows, developers can chain multiple models sequentially via Gradio Series or host custom checkpoints directly when models lack native Inference API support.


### [Summer at Hugging Face](https://yomu.fyi/post/summer-at-hugging-face.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: system
- Published: Sep 24, 2021
- Type: Announcement

Hugging Face released several major updates across its ecosystem, expanding its public Hub repository to over 16,000 models. Platform additions include Spaces for deploying Gradio and Streamlit demo applications, automatic TensorBoard instances, and evaluation metric tracking integrated with Papers With Code leaderboards. The Transformers library gained JAX/Flax support across more than 5,000 models, improved TensorFlow implementations, and introduced the transformers.onnx export module for model conversion. In research, the BigScience project completed large-scale training of a 13-billion-parameter English decoder model on Jean Zay, while the DeDLOC method enabled training the sahajBERT Bengali model without HPC infrastructure. Accepted conference papers detailed the Datasets library, prompt data-point equivalence, prompt-based fine-tuning heuristics, and block pruning techniques that reduced BERT size by 74 percent.


[Newer posts](https://yomu.fyi/topic/python/page/10.md) · [Older posts](https://yomu.fyi/topic/python/page/12.md)