---
title: "Python"
description: "252 posts about Python, summarised, each linking to the original."
---

# Python
> 252 posts about Python, summarised, each linking to the original.

## Articles

### [Hyperparameter Search with Transformers and Ray Tune](https://yomu.fyi/post/hyperparameter-search-with-transformers-and-ray-tune.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: system
- Published: Nov 2, 2020
- Type: Tutorial

Hugging Face Transformers models often require hyperparameter tuning to achieve high performance, but practitioners frequently rely on basic grid search or bypass tuning entirely. The Transformers 3.1 release integrates Ray Tune into the Hugging Face Trainer to provide advanced hyperparameter optimization without losing framework customizability. By invoking hyperparameter\_search with the Ray backend, users can configure trial budgets, parallelize runs across multiple GPUs, and plug in search algorithms like HyperOptSearch alongside schedulers like ASHAScheduler. Experimental benchmarks on a BERT model using the RTE dataset show that Population-based Training reached 70.5% test accuracy in 48 GPU minutes, outperforming Grid Search at 65.4% test accuracy. The integration also supports experiment tracking tools like Weights and Biases out of the box.


### [How to Build an Experiment Pipeline from Scratch](https://yomu.fyi/post/how-to-build-an-experiment-pipeline-from-scratch.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: 2023-10-18
- Published: Oct 15, 2020
- Type: Tutorial

The post describes building an email experimentation pipeline at Shopify after an existing framework for web properties and Shopify admin experiences could not support tests in external channels. The motivating case was a personalized blog-post recommendation algorithm whose incremental benefit needed comparison with generic blog emails, while local randomization and text-file storage created risks around unsubscribed users, shop-email relationships, and overlapping audiences. After requirements gathering, system mapping, mock-table queries, stakeholder review, and technical design review, the author implemented three pipeline phases in PySpark: experiment definitions, consolidation, and filtering with deduplication and exclusion rules. Production output supported ongoing randomization, backfills, category tags, sample limits, and integration with Shopify’s existing analysis framework; the tool was subsequently used across multiple data teams for email experiments.


### [Transformer-based Encoder-Decoder Models](https://yomu.fyi/post/transformer-based-encoder-decoder-models.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Patrick von Platen
- Published: Oct 10, 2020
- Type: Explainer

Natural language generation tasks map input sequences to target sequences whose lengths cannot be known in advance and vary by content. Standard deep neural networks struggle with these variable mappings because their inputs and targets require fixed-dimensional vector representations. While recurrent neural networks addressed this challenge by generating target sequences auto-regressively from a compressed context state, transformer-based encoder-decoder architectures became the standard paradigm in modern natural language processing. The framework processes sequence-to-sequence problems by encoding source sequences and auto-regressively decoding target sequences token by token using conditional probability distributions. A step-by-step implementation demonstrates greedy decoding with Hugging Face Transformers, passing cached encoder hidden states alongside previously generated target tokens to iteratively generate German translations from English inputs.


### [Block Sparse Matrices for Smaller and Faster Language Models](https://yomu.fyi/post/block-sparse-matrices-for-smaller-and-faster-language-models.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: François Lagunas
- Published: Sep 10, 2020
- Type: Announcement

Full dense layers in neural networks are often unnecessarily large and can be pruned without sacrificing precision, but available sparse algebra tools and PyTorch native implementations lack efficiency. To resolve this issue, the pytorch\_block\_sparse extension introduces BlockSparseLinear as a drop-in replacement for standard linear layers alongside BlockSparseModelPatcher for on-the-fly model modifications. The library uses NVIDIA CUTLASS C++ CUDA templates derived from Yulhwa Kim's cutlass tilesparse proof of concept. Although baseline sparse operations remain roughly twice as slow as cuBLAS-optimized dense counterparts, performance scales with sparsity, making a 75% sparse matrix approximately two times faster than a dense layer while reducing memory consumption by four times. Future updates will target dynamic sparsity optimization and NVIDIA Ampere Tensor Core primitives.


### [Supabase Alpha August 2020](https://yomu.fyi/post/supabase-alpha-august-2020.md)
- Company: [Supabase](https://yomu.fyi/company/supabase.md)
- Author: Paul Copplestone
- Published: Sep 3, 2020
- Type: Announcement

Six months into developing its hosted database platform, Supabase announced updates focusing on authentication, table view management, and community developer tooling. Users can now create database tables and columns directly from the table view interface as well as invite team members to their organizations. The authentication system introduced configurable email confirmations for new user registrations, allowing administrators to toggle confirmations and edit email templates directly from the dashboard. Community contributors added TypeScript support to Supabase, implemented UMD support in supabase-js, and began building Python client libraries starting with postgrest-py. Supabase is preparing to transition from Alpha to Beta by stabilizing its Auth API, improving dashboard performance, and benchmarking platform systems.


### [From Project to Productionized with Python](https://yomu.fyi/post/from-project-to-productionized-with-python.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Ed Morley
- Published: Jun 22, 2020
- Type: Tutorial

The post presents a hands-on tutorial for Python developers moving a local Django application into production on Heroku. It frames the work around 12-factor practices and covers repository hygiene, modular settings, continuous delivery stages, static-asset middleware, environment configuration, database provisioning, deployment, scaling, and troubleshooting. The configuration changes include excluding virtual environments, caches, SQLite data, and generated static files; moving settings.py to settings/base.py; updating DJANGO\_SETTINGS\_MODULE; adding Whitenoise; and supplying SECRET\_KEY and a Heroku Postgres add-on. The deployment uses Git, the Heroku CLI, git push heroku main, and heroku ps:scale web=1, with heroku logs --tail and heroku run bash for diagnosis. The walkthrough concludes with the app deployed and accessible through heroku open.


### [How to generate text: using different decoding methods for language generation with Transformers](https://yomu.fyi/post/how-to-generate-text-using-different-decoding-methods-for-language-gen.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Patrick von Platen
- Published: Mar 1, 2020
- Type: Tutorial

Auto-regressive language generation relies heavily on decoding methods to convert predicted next-token probability distributions into coherent text. Greedy search selects the single most probable token at each timestep, but it frequently leads to repetitive phrasing and misses high-probability sequences hidden behind low-probability intermediate words. Beam search alleviates this issue by maintaining multiple parallel hypotheses and can be paired with n-gram penalties to reduce immediate repetitions. Sampling approaches, including top-K and top-p sampling, offer alternative paths that can yield more fluent, varied open-ended generation. Using the Hugging Face transformers library, practitioners can configure these strategies via parameters on the model generate function.


### [How to train a new language model from scratch using Transformers and Tokenizers](https://yomu.fyi/post/how-to-train-a-new-language-model-from-scratch-using-transformers-and.md)
- Company: [Hugging Face](https://yomu.fyi/company/hugging-face.md)
- Author: Julien Chaumond
- Published: Feb 14, 2020
- Type: Tutorial

Recent library enhancements simplify pretraining custom language models from raw text corpora. A small 84-million-parameter RoBERTa architecture comprising six layers, twelve attention heads, and a hidden dimension of 768 was pretrained on a 3 GB Esperanto text dataset. Training began by fitting a 52,000-vocabulary byte-level Byte-pair encoding tokenizer, which natively handled Esperanto diacritics and reduced average sequence lengths by approximately 30 percent compared to GPT-2 tokenization. The model was trained using masked language modeling, evaluated using fill-mask pipelines, and subsequently fine-tuned for part-of-speech tagging via standard token classification scripts. Final model artifacts and configurations were uploaded for community reuse via pre-trained model interfaces.


### [Griffin, an Anti-fraud Risk Rule Engine Making Billions of Predictions Daily](https://yomu.fyi/post/griffin-an-anti-fraud-risk-rule-engine-making-billions-of-predictions.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Muqi Li
- Published: Oct 28, 2019
- Type: Problem & solution

Grab's Trust/Identity/Safety team built Griffin, an in-house anti-fraud risk rule engine designed to process billions of daily predictions across multiple business verticals. Initially, Grab managed fraud rules directly within backend service code, but escalating rule complexity, tight rule interdependencies, and translation gaps between data scientists and developers caused deployment delays and misfiring errors. To overcome the limitations and steep learning curves of third-party engines like Drools, the team separated the workflow into data orchestration and rule-based prediction. Griffin enables analysts and data scientists to author Python-based rules directly via a web portal and reload updated logic into memory without manual developer intervention. By eliminating I/O during rule evaluation and leveraging Gunicorn multi-processing, the engine handles over 100,000 queries per second at peak on six EC2 instances with single-prediction latencies under six milliseconds.


### [Getting to Know Python 3.7: Data Classes, async/await and More!](https://yomu.fyi/post/getting-to-know-python-3-7-data-classes-async-await-and-more.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: Casey
- Published: Apr 29, 2019
- Type: Explainer

Python 3.7 is presented as an upgrade for developers seeking new language features, library improvements, and performance gains over earlier Python 3 releases. The article focuses on data classes, where the @dataclass decorator inspects typed attributes and generates an init method, reducing boilerplate for data-oriented classes. It also covers asyncio.run for simpler asynchronous execution, breakpoint() for entering pdb, module-level getattr for lazy loading, and context variables that keep values separate across execution contexts. Dictionary insertion order becomes part of the language specification in 3.7, while OrderedDict remains available for advanced behavior; reported improvements include 10–30% faster startup, 45–70% faster common sorting cases, 5.5-times-faster dict.copy(), and 4–6-times-faster namedtuple creation.


### [How Shopify Merchants can Measure Retention](https://yomu.fyi/post/how-shopify-merchants-can-measure-retention.md)
- Company: [Shopify](https://yomu.fyi/company/shopify.md)
- Author: Cam Davidson-Pilon
- Published: Nov 14, 2017
- Type: Explainer

Shopify addresses customer retention for merchants operating non-contractual businesses, where churn is not directly observable. Common rules that label customers inactive after N days create arbitrary binary states, so the post proposes probabilistic modeling with the BG/NBD (Beta Geometric / Negative Binomial Distribution) model. It represents each customer with a purchase rate and churn probability, estimates their population distributions, and uses age, recency, and repeat-purchase frequency as sufficient statistics. Shopify fits a modified implementation in lifetimes using Apache Spark DataFrames, subsampling up to 50k customers per shop for performance, and produces activity probabilities and expected future purchases for more than 500 million historical merchant customers. The model is interpretable and supports customer ranking and targeted campaigns, but it handles seasonality poorly and does not easily incorporate additional variables such as country or products purchased.


### [How to Secure the Celery Task Queue with Fernet](https://yomu.fyi/post/how-to-secure-the-celery-task-queue-with-fernet.md)
- Company: [Heroku](https://yomu.fyi/company/heroku.md)
- Author: David Gouldin
- Published: Sep 15, 2014
- Type: Problem & solution

Celery jobs sent through a Redis broker can expose their contents because Redis does not currently support SSL and Heroku add-ons communicate over the public web. The proposed solution encrypts jobs before they travel between dynos and Redis by wrapping Kombu’s default serializers with Fernet symmetric encryption from Cryptography. The open-source kombu-fernet-serializers library supplies these encrypted serializers through Setuptools entry points; setup requires adding the package, storing a Fernet key in KOMBU\_FERNET\_KEY, and configuring Celery to use fernet\_json with MIMETYPE in CELERY\_ACCEPT\_CONTENT. The example creates a Celery app with REDIS\_URL as its broker and uses the serializer settings, retaining Redis’s straightforward configuration while addressing sensitive information in transit.


[Newer posts](https://yomu.fyi/topic/python/page/12.md)