Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,634 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Free Postgres Databases
Fly.io announces that Postgres is now available at no cost for small projects, addressing the storage cost that has kept persistent data out of its existing free tier. The offering adds 3GB of persistent volume space through LVM block devices as small as 1GB, alongside the free VM and RAM allowance. Production deployments use intentional two-node Postgres clusters with data on two isolated NVMe drives and volume snapshots, while a free setup begins as a single Development node and becomes a leader-replica cluster when scaled. The volumes are not limited to Postgres: applications can mount up to 3GB for SQLite or other uses. A credit card is still required for free services, which the post says helps deter cryptocurrency-mining abuse of free VMs.
Cloud, Load, and Modular Code: What 2022 Looks Like for Shopify
Shopify’s 2021 BFCM four-day event exposed the scale its commerce platform must absorb: peak app-server traffic exceeded 32 million requests per minute, load balancers surpassed 34 million RPM, and a flash sale used more than 20% of total computing capacity. The post frames larger, spikier sales, mobile and social commerce, and expanding buying environments as enduring engineering challenges for supporting millions of merchants. Planned work includes evolving the large Rails codebase into modular components, moving storefront rendering out of the monolith, expanding Spin cloud development environments, and making in-house load testing part of product development. The storefront split and refactoring produced four-times-faster cache fill rates and five-times-faster page rendering, while YJIT ran 31% faster than interpreted CRuby and 26% faster than MJIT; Shopify also remains committed to Rails and React Native.
The Adventures of Rendezvous in Heroku’s New Architecture
Heroku’s migration of Rendezvous, the bidirectional proxy that connects Heroku Run sessions to one-off dynos, from EC2 Classic to AWS VPCs caused one customer’s app to hit H12 timeouts. The move replaced a single Elastic IP ingress and process-level TLS termination with a Network Load Balancer, whose larger TLS frames exposed a blocking flaw in Rendezvous’s single-threaded Ruby channel loop. That loop used IO.select on the TCP socket and dyno terminal, but OpenSSL::SSL::SSLSocket could retain unread buffered data after a partial read; IO.select saw no TCP bytes and blocked beyond the Heroku Router’s 30-second limit. Engineers reproduced the behavior across two languages, instrumented the instance agent, then fixed the channel with separate threads and corrected socket-reading and IO.select handling. After staging tests, the customer moved back to VPC Rendezvous and confirmed the issue was resolved.
Search at Shopify—Range in Data and Engineering is the Future
Search at Shopify is presented as a case for range across data science and engineering, rather than narrow specialization in either discipline. The article describes a team that treats work as “search” instead of separating data and engineering responsibilities: engineers learn experimentation and model evaluation, while data scientists build performant, testable, maintainable production systems. It frames search as a sequence of trade-offs among relevance, correctness, scalability, performance, stability, and maintainability, arguing that people who understand only half the constraints can produce siloed handoffs, deployment delays, or suboptimal systems. Shopify’s stated model is a single collaborative team with shared planning and execution, intended to reduce territorial boundaries and support smart, fast, scalable search. The article extends this principle to personalized and conversational products, concluding that future work will require practitioners to move between data and engineering.
Case Study: Millisecond Latency using Hugging Face Infinity and modern CPUs
Deploying large Transformer models at scale often faces severe prediction latency bottlenecks, driving up infrastructure costs and limiting real-time production use cases. To address this challenge, Hugging Face evaluated Infinity, a containerized hardware-optimized inference solution paired with the Infinity Multiverse model optimization service. Testing covered 192 configurations on Amazon EC2 C6i instances powered by 3rd generation Intel Xeon Scalable processors across varying CPU cores, sequence lengths, and batch sizes. The benchmarks demonstrated that an Ice Lake-optimized DistilBERT container achieved up to 800% higher throughput than vanilla Transformers and delivered 1 to 4 millisecond end-to-end latencies for sequence lengths up to 64 tokens. Although Infinity was later discontinued in favor of Inference Endpoints and Optimum libraries, the results demonstrated substantial efficiency gains on modern CPU hardware.
That Old Certificate Expired and Started an Outage. This is What Happened Next
On September 30th, 2021, Shopify nearly suffered a platform-wide outage after Let’s Encrypt’s old root certificate expired, exposing stale certificate data in internal clients rather than at the public edge. At 10:15 a.m., monitoring detected certificate errors after deployment-related connection changes, and investigation traced them to Ruby services that depended directly or indirectly on httpclient, whose embedded cacert.pem was six years old and did not trust the applicable certificate chain. Shopify forked the library, replaced cacert.pem with a current root certificate bundle, deployed it to canaries, and automated pull requests for affected repositories. The temporary fix restored the monolith, while systems were added to track root-certificate changes and update the fork automatically. Deployment locking kept most monolith pods from cycling, limiting the incident to canaries and causing secondary effects including delayed order confirmations, throttled marketing campaigns, and missing product recommendations.
Boosting Wav2Vec2 with n-grams in 🤗 Transformers
Wav2Vec2 models fine-tuned with Connectionist Temporal Classification transcribe speech without external language models, but decoding can still suffer from spelling inaccuracies. Hugging Face Transformers addressed this by integrating Kensho Technologies' pyctcdecode library to support decoding with n-gram language models. Instead of decoding using simple argmax operations over logits, the Wav2Vec2ProcessorWithLM class feeds full probability matrices into beam search guided by KenLM n-gram probabilities. KenLM's build_binary utility compresses language model ARPA files into binary formats, reducing file size by more than half for faster loading and hub deployment. In Swedish xls-r-300m-sv benchmarks on Common Voice 7, this 5-gram boosted decoding setup achieved an 18.85% word error rate, delivering an approximate 30% relative performance gain.
Deploy GPT-J 6B for inference using Hugging Face Transformers and Amazon SageMaker
Deploying EleutherAI's 6 billion parameter GPT-J model for production inference presents latency hurdles due to large memory footprints and lengthy startup times. Loading the model via standard methods takes up to several minutes, conflicting with strict real-time response limits such as Amazon SageMaker's 60-second threshold. To overcome this limitation, the model is serialized using PyTorch's native save mechanisms, packaged into a compressed archive with supporting assets, and stored on Amazon S3. This alternative loading workflow reduces GPT-J load times down to 7.7 seconds. An Amazon SageMaker real-time endpoint is then deployed on an NVIDIA T4 GPU instance using the Hugging Face Inference Toolkit.
Self-service account deactivation: why it’s harder than you would think
Fly.io introduces a self-service UI for deactivating accounts, replacing the previous requirement to email support. The difficulty is preventing a “delete everything” action from accidentally destroying infrastructure, because users may own apps, share organizations, and administer shared applications. The workflow first addresses app deletion and resource cleanup, then points migrating users toward certificate and DNS changes, while reminding organization administrators to delete or transfer organizations. Rather than embedding every task in one action, the UI guides users to the relevant pages and usually requires deleting only the apps they own. Automatic deactivation still cannot handle accounts involved in shared organizations or organizations administered for others; those cases require help, and the UI identifies the blocking organization step.
Nerd Out on 10 of Our Favorite Posts From 2021
Shopify’s year-end roundup curates ten favorite engineering posts from 2021, spanning Ruby, storefronts, data analysis, databases, search, GraphQL, CI, APIs, and mobile development, while also noting several older reader favorites. The selection includes tutorials on building a web app with standard Ruby libraries, exploratory data analysis, GraphQL, Apache Beam for search, and React Native App Clips, alongside technical accounts of Hydrogen, YJIT, MySQL upgrades, and GraphQL rate limiting. YJIT was merged upstream and included in Ruby 3.1.0, while the CI team reduced the p95 of Shopify’s core monolith from 45 minutes to 18 minutes. The roundup also describes query-cost-based GraphQL rate limiting, rollback-testing roadblocks during a MySQL upgrade, and an App Clip effort that used a fail-fast, iterative approach.
Active Learning with AutoNLP and Prodigy
Active learning requires iteratively adding labeled data, retraining models, and serving them to end users. Building such pipelines often demands substantial effort in data labeling, model selection, hyperparameter tuning, and training infrastructure. The author demonstrates a low-code active learning pipeline using Explosion's Prodigy for entity annotation and Hugging Face's AutoNLP for automatic training and evaluation. After first training a news categorization model achieving 98.67% accuracy on Kaggle's BBC News dataset, the author manually annotated named entities across iterative batches. An export script converted annotations to JSONL with IOB tags, showing progressive metric improvements from 20 samples to 250 samples, where the token classification model attained 95.9% accuracy, 0.73 precision, and 0.79 recall.
Shopify’s Unique Data Science Hierarchy Of Needs
Shopify presents a five-level Data Science Hierarchy of Needs that organizes data work by the value it delivers, rather than by technical sophistication. Its tool-agnostic philosophy favors simple solutions first, then more complex analysis when necessary, with impact and better decisions as the goal. The model begins with collecting and modeling clean, conformed data using dimensional modeling, then moves through description, prediction and inference, prescription, and influence. Shopify applied this structure to a COVID-19 impact analysis, using statistical analysis to examine regional effects and inform measures including an extended free trial, expanded funding, shipping options, and local commerce features. The account says the analysis used neither artificial intelligence nor machine learning, yet supported decisions for merchants and Shopify during the pandemic.
Gradio is joining Hugging Face!
Hugging Face has acquired Gradio, the open-source machine learning library designed for building and sharing interactive model interfaces. Gradio originated in 2019 when its founder struggled to share a medical computer vision model with a physician collaborator who did not write Python. Co-founded alongside Ali Abdalla, Ali Abid, and Dawood Khan, the project expanded from computer vision into text, speech, and video modalities. Over 300,000 demos have been built using Gradio, allowing interdisciplinary industry teams and researchers to debug models internally and showcase them externally to non-technical users. The acquisition unites Gradio with Hugging Face to broaden browser-based machine learning accessibility and expand hiring efforts across the joint team.
Holiday Hackdays Winners 2021
Supabase organized the Holiday Hackdays 2021 hackathon following their launch week and announced the winning and runner-up community submissions. Selected winners include Swappy.one by Zernonia, the realtime polling platform rtPoll by Emilio and Federico Schepis, Santa Banter for holiday jokes by Andy Keogh, and the wishlist app the get list by glowdexapp. Recognized runners-up include Chivel for YouTube channel landing pages, the end-to-end encrypted e2ee-chat, a Flutter-based Holiday Sweater voting app, and a greeting card platform built by high school hackers. Winners receive limited-edition gold medal shirts, while runners-up earn silver medal shirts. All project submissions were made available for viewing on madewithsupabase.com.
Supabase Beta November 2021: Launch Week Recap
Supabase celebrated its third Launch Week and one year in beta by rolling out several major infrastructure and tooling updates. The company open-sourced its Dashboard for self-hosting and integrated Logflare to provide searchable database and API logs directly in the interface. For database querying and APIs, Supabase introduced a Postgres extension that resolves each GraphQL request with a single SQL statement to minimize network IO overhead, while upgrading its default hosted database to PostgreSQL 14 and deploying PostgREST 9.0. Security capabilities were expanded by enabling Row Level Security policies across the Realtime API to restrict streams and subscriptions on a per-user basis. Additional releases during the week included faster media file delivery, an open-sourced launch methodology, and a free course combining Next.js, Stripe, and Supabase.
Perceiver IO: a scalable, fully-attentional model that works on any modality
Standard Transformer architectures scale poorly in compute and memory because pairwise dot-product self-attention depends quadratically on input size. Perceiver IO addresses this constraint by computing self-attention across a small set of latent variables rather than directly on high-dimensional inputs. Inputs and outputs interact with the model via cross-attention operations, decoupling compute and memory costs from input and output dimensions. Integrated into Hugging Face Transformers via the PerceiverModel class, the architecture supports diverse data types using optional preprocessors, decoders, and postprocessors. Experiments demonstrate competitive performance across text, multimodal video classification, 3D point cloud classification on ModelNet40, and StarCraft II reinforcement learning in AlphaStar.
Building a Real-time Buyer Signal Data Pipeline for Shopify Inbox
Shopify built a real-time buyer-signal pipeline for Shopify Inbox after noting that 70 percent of its conversations involved customers making a purchasing decision. The system surfaces cart actions and order-completion information as chronological conversation events, covering events from up to 14 days before a conversation and post-conversation events while its seven-day lifecycle remains active. It uses Kafka through Monorail and Change Data Capture, Apache Beam on Google Cloud Dataflow, filtering jobs for mission-critical transactional events, and stateful aggregation with timers and a Global Window to retain buyer activity. In an equal-group A/B test, buyer signals increased merchant response rate by two percentage points and attributed conversion rate by 0.7 percentage points, while response time showed no significant change.
Scaling Shopify's BFCM Live Map: An Apache Flink Redesign
Shopify’s 2021 BFCM live map had to scale from more than 1.7 million merchants while adding product trends and unique shoppers, after the prior Cricket-and-Redis backend showed volume and delivery weaknesses. The redesign placed Apache Flink on the critical path to filter irrelevant checkout events, reducing Cricket’s workload to about one percent of event volume, while cross-region sharding and active-active deployment supported availability. Flink computed new metrics, including 500 product categories ranked by sales-volume change every five minutes, while Cricket retained existing metric computation and deduplication. In load tests, the pipeline handled 1 million checkout events per second, and its jobs ran at 100 percent uptime during BFCM without backpressure or manual intervention. Redis still caused high CPU and memory pressure, producing brief arc-visual errors for a small sample of users, so Shopify plans a simpler Flink-centered design.
Training CodeParrot 🦜 from Scratch
Training code generation models from scratch requires careful data filtering, tokenization, and compute management. To build CodeParrot, a 1.5-billion-parameter Python generation model, engineers extracted 20 million Python files from BigQuery and cleaned the resulting 180 GB corpus down to 50 GB after discovering extreme duplication. A custom GPT-2 tokenizer was trained over streamed samples, and the GPT-2 large architecture was initialized with layer-scaled, full-precision attention mechanisms. The training pipeline leveraged Hugging Face Accelerate alongside gradient checkpointing and a streaming iterable dataset designed to yield fixed-length concatenated token sequences. CodeParrot was trained on roughly 25 to 30 billion tokens and evaluated on coding tasks using the pass@k metric.
Five more things
Supabase announced a series of platform updates, technical releases, and community initiatives to conclude its Launch Week. The team open-sourced pg_graphql, an in-progress PostgreSQL extension that keeps schema generation, query parsing, and resolvers natively on the database server. Supabase also enabled a global CDN spanning over 250 cities, delivering edge caching with a default one-hour cache time for storage assets alongside TLS 1.3, HTTP/3 over QUIC, and Brotli compression. While Supabase Functions is not yet released, two public RFCs were opened to explore architecture options, leaning heavily toward container-based execution. Finally, the update introduced a free SaaS development course with egghead.io and launched the ten-day Holiday Hackdays hackathon.
Top topics
- Architecture832
- Performance814
- LLMs658
- Machine Learning620
- Open Source563
- AI Agents549
- AI465
- Developer Experience448
Weekly digest
The best engineering reads, once a week.