Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,910 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Hydrogen & Tailwind: The Perfect Match for Building Beautiful Storefronts
Hydrogen uses Tailwind as the default styling option in its React-based starter, allowing developers to build storefronts with utility classes from the moment they run create-hydrogen-app. The article presents componentization as a central way to organize those classes: React components can encapsulate responsive, hover, flexbox, spacing, typography, and visibility utilities and be reused across a storefront. It contrasts this with semantic class systems whose names can become coupled to a product component, making reuse, renaming, or duplication more cumbersome. Tailwind’s learning curve still requires CSS knowledge, but its documentation and VS Code extension support autocomplete, style representations, and color previews. The conclusion is that Tailwind’s constraints, customizable design system, and composable styles complement Hydrogen’s starter components and developer experience, although the author acknowledges that readers may still prefer other approaches.
Shopify's Playbook for Scaling Machine Learning
Shopify describes how its machine-learning practice grew from a first order-fraud product into a portfolio used across products such as Shopify Capital, product categorization, and Help Center search. The playbook starts with choosing a user-important problem, validating trustworthy and accessible data, tracing downstream dependencies, understanding existing solutions, and optimizing for product outcomes rather than model scores alone. To productionize models, Shopify recommends well-defined training, verification, and testing pipelines, deployment decisions based on volume and user commitment, and platform practices that encode validated standards without abstracting too early. Its fraud pipeline builds models in Python, serializes them with PMML for deployment in Ruby-based production systems, and uses Apache Airflow to schedule operations. The account concludes that scaling is a full-stack problem requiring data scientists and data engineers to align on one roadmap and goal.
Fine-Tune ViT for Image Classification with 🤗 Transformers
Vision Transformer models bring transformer architectures to computer vision by splitting images into grids of sub-image patches and projecting them into token sequences. To classify healthy and diseased leaves using the beans dataset, the Hugging Face datasets and transformers libraries enable streamlined data ingestion and model fine-tuning. Preprocessing relies on ViTImageProcessor paired with lazy on-the-fly dataset transforms to dynamically generate normalized pixel tensors. Training leverages ViTForImageClassification with the Trainer API, requiring remove_unused_columns set to False so raw image data is preserved for batch collation. Over four training epochs, fine-tuning the google/vit-base-patch16-224-in21k checkpoint achieves an evaluation accuracy of 98.5% alongside an evaluation loss of 0.0637.
Our User-Mode WireGuard Year
Fly.io’s flyctl needed to let users SSH into Firecracker VMs and reach private 6PN networks without requiring root access or system-level WireGuard configuration. The solution runs wireguard-go, a userland TCP/IP stack derived from gVisor’s Go implementation, and golang.org/x/crypto/ssh inside flyctl, later coordinating sessions through a cross-platform background agent. The design initially exposed serious operational and usability problems: concurrent sessions displaced one another, agent behavior was fragile, and remote peer creation was slow, while deploys and remote builds expanded the mechanism’s role. The team improved concurrency, Windows support, DNS resolution, configuration handling, and peer-creation latency, and built an unmerged WebSocket path for networks blocking WireGuard’s UDP transport. The author concludes that the complexity was costly early in 2021 but enabled capabilities such as private DNS lookup and ping.
Turbocharge your Heroku Apps with New Turboku
New Turboku clones a Heroku app on Fly.io without requiring data migration, while the resulting app should be able to use services such as Heroku Postgres or Redis through transferred environment variables. The rebuilt workflow integrates flyctl with Fly.io’s remote builders instead of AWS CodeBuild: it queries Heroku, selects a region, converts config variables to secrets, and generates fly.toml, an ephemeral Dockerfile, and a reconstituted Procfile from the slug and process map. A Phoenix LiveView web launcher wraps the deployment by starting a Fly.io Machine running flyctl, while the CLI is available through flyctl turboku. The launcher also configures webhooks and watches environment-variable changes for automatic updates, whereas the CLI lacks webhook integration; the source reports about one millisecond of added database latency for a US-based app.
React Server Components Best Practices You Can Use with Hydrogen
React Server Components (RSC) introduced a paradigm shift for the team building Hydrogen, a React-based framework for custom storefronts. The post presents patterns developed through trial and error, beginning with shared components as a middle ground and reserving client components for interactivity, useState or useReducer, lifecycle logic, unsupported libraries, or browser APIs. Components should instead become server components when they contain secrets or proprietary logic, need filesystem or database access, fetch storefront API data, or do not execute on the client. The examples extract newsletter form controls and FAQ accordion behavior into client components, then rename the remaining wrappers as server components when their usage allows it; the FAQ server component is passed through a client component as children.
Rapid Development with Hydrogen: Building a Product Page
Updated for Hydrogen 0.26.0, the tutorial builds a simple product page for a custom Shopify storefront using Hydrogen’s React-based framework. Starting from the Hello World template on StackBlitz, it adds Tailwind CSS, creates a file-based dynamic product route, and queries product data through the Storefront API with the useShopQuery hook. The implementation then uses React Server Components alongside a client component and ProductOptionsProvider to track selected variants and options, update prices, and support a BuyNowButton checkout flow. A media gallery and additional styling complete the page, which uses products and media from the Hydrogen Preview store. The post concludes that Hydrogen’s components, hooks, and utilities reduce repetitive storefront work and speed development.
Getting Started with Sentiment Analysis using Python
Modern natural language processing tools allow developers to perform sentiment analysis without deep machine learning expertise. Sentiment analysis categorizes text polarity into positive, negative, or neutral labels to extract insights from large volumes of social posts, reviews, and support tickets. Using the Hugging Face Transformers library and the Hub, practitioners can quickly run pre-trained transformer pipelines or target specific multilingual and emotion-detection models with minimal Python code. For custom requirements, developers can fine-tune models like DistilBERT using the IMDB dataset via the Trainer API or train models automatically with AutoNLP. Applying these workflows to social media data enables automated sentiment distribution analysis and visualization using pandas, matplotlib, and word clouds.
LiveBeats: Building a social music app with Phoenix LiveView
LiveBeats is an open-source social music application built as a full-stack Phoenix reference app and learning test bed for LiveView features. It demonstrates a shared real-time UI for playlist playback, presence, GitHub OAuth, and concurrent MP3 uploads, using Phoenix PubSub and the existing LiveView WebSocket connection rather than separate REST, JSON, or polling layers. Uploads are processed concurrently in temporary files to verify MP3 validity and enforce a duration below 20 minutes, while HEEx function components and slots provide reusable UI and JS commands keep immediate interactions such as modal hiding on the client. The application runs across five continents with automatic clustering, file proxying, and regional Postgres read replicas; the post presents this as a way to deliver responsive global behavior with relatively minor changes.
Making automatic speech recognition work on large files with Wav2Vec2 in 🤗 Transformers
Transformer-based automatic speech recognition models like Wav2Vec2 crash on long audio inputs because the O(n²) attention mechanism rapidly exhausts GPU memory. Simple audio chunking avoids out-of-memory errors but causes poor transcription accuracy at chunk boundaries due to a lack of surrounding context. To resolve this, Hugging Face Transformers leverages the Connectionist Temporal Classification architecture to process overlapping audio chunks with configurable strides. The pipeline discards low-quality logits at chunk edges and chains the remaining central predictions together to reconstruct a seamless transcript. This striding technique operates out of the box with language model-augmented pipelines and adapts directly to low-latency live audio streaming.
Announcing Intercom’s new program for venture-backed startups
Intercom announces a startup program aimed at fast-growing, venture-backed Seed and Series A companies, working with venture capital, accelerator, incubator, and technology partners. The program provides its main customer communication solutions at no cost in the first year, 50% off in the second, and 20% off thereafter, covering tools for bots and content, in-product messaging, onboarding, and conversational engagement. It extends Intercom’s early-stage program, which offers a 95% discount and has helped thousands of pre-seed startups, while focusing more closely on companies scaling with venture backing. Intercom says more than 5,000 startups from leading incubators use its products and cites Spendesk, Notion, Monzo, Frame.io, and Aircall among startup customers; it also plans webinars and events and invites partners to apply.
Fixing the Dependency Confusion Vulnerability in 600+ Ruby Apps - Shopify
Shopify identified a dependency confusion vulnerability through its bug bounty program; an attacker could exploit ambiguous package sources to install a malicious dependency and gain access to local, CI/CD, and production environments. To assess exposure, the Ruby Conventions team collected Bundler versions from repositories in a data lake and found around 600 susceptible Ruby applications. An experimental Bundler plugin converted Gemfile.lock files to the explicit-source format while restoring existing gem versions, and a CI-backed CLI later provided per-repository environments, parallel execution, pull requests, and easier failure tracking. After incremental migrations exposed configuration and tooling issues, Shopify updated supporting systems, set Bundler requirements, changed local and CI defaults, and worked with Bundler maintainers; the company reports migrating all its applications in less than a day and contributing to Bundler 2.2.21 and 2.2.22.
Supercharged Searching on the 🤗 Hub
Programmatically searching the Hugging Face Hub previously required navigating web browser widgets or enduring trial and error to guess exact query string formats. The huggingface_hub library solves this issue by introducing helper utilities such as ModelSearchArguments, DatasetSearchArguments, and ModelFilter alongside HfApi. These namespace helpers translate accessible Python attributes into the formatted parameters expected by the backend API, covering datasets, tasks, and libraries. For complex queries across multiple tasks, frameworks, and datasets, developers can combine criteria inside ModelFilter instances and retrieve matching model metadata via api.list_models. Under the hood, the library uses AttributeDictionary, a data structure inspired by fastcore that enables nested tab-completion while supporting standard dictionary key indexing for special characters.
Welcome Stable-baselines3 to the Hugging Face Hub 🤗
Hugging Face announced an official integration with Stable-Baselines3, a popular PyTorch library for training and testing Deep Reinforcement Learning agents across diverse environments like Gym, Atari, MuJoco, and Procgen. The integration enables researchers and developers to host their saved reinforcement learning checkpoints on the Hugging Face Hub and download pre-trained community models. Interacting with the Hub requires installing the huggingface_hub and huggingface_sb3 packages, which supply helper methods for authentication, downloading, and uploading. Practitioners can retrieve checkpoint zip files using the load_from_hub function by providing the target repository identifier and filename before loading them into Stable-Baselines3 algorithms. Furthermore, users authenticated via CLI or notebook login can train policies such as PPO and publish their saved zip files to the Hub using push_to_hub.
Supabase Beta December 2021
Supabase announced a series of platform updates, educational content, and community milestones for December 2021. PostgreSQL instances now include the pg_sodium extension, enabling users to perform encryption, decryption, hashing, and cryptographic signing directly within database queries and functions through the Supabase Dashboard. The release details educational video guides demonstrating remote procedure calls from JavaScript, invoking external HTTP endpoints from database functions, and setting up triggers to run SQL on table modifications. In addition to ecosystem updates featuring new Python library releases and integrations with Divjoy and n8n, the project surpassed 26,000 GitHub stars and opened fully remote job positions across engineering, marketing, and human resources.
Biometric authentication - Why do we need it?
Grab addressed the vulnerabilities and costs associated with SMS one-time passwords and PINs by implementing device-level biometric authentication. The architecture pairs device biometric sensors with hardware secure enclaves to protect private keys separately from the main operating system. During enrollment, Grab generates a public-private key pair using SHA512withECDSA, authenticates the user locally, and stores reference identifiers in encrypted device storage. HellfireSDK verifies that the device is not rooted, ensuring raw biometric data never leaves the handset. Early experimental runs indicate an adoption rate exceeding 90% and a login success rate near 90%.
Free Postgres Databases
Fly.io announces that Postgres is now available at no cost for small projects, addressing the storage cost that has kept persistent data out of its existing free tier. The offering adds 3GB of persistent volume space through LVM block devices as small as 1GB, alongside the free VM and RAM allowance. Production deployments use intentional two-node Postgres clusters with data on two isolated NVMe drives and volume snapshots, while a free setup begins as a single Development node and becomes a leader-replica cluster when scaled. The volumes are not limited to Postgres: applications can mount up to 3GB for SQLite or other uses. A credit card is still required for free services, which the post says helps deter cryptocurrency-mining abuse of free VMs.
Cloud, Load, and Modular Code: What 2022 Looks Like for Shopify
Shopify’s 2021 BFCM four-day event exposed the scale its commerce platform must absorb: peak app-server traffic exceeded 32 million requests per minute, load balancers surpassed 34 million RPM, and a flash sale used more than 20% of total computing capacity. The post frames larger, spikier sales, mobile and social commerce, and expanding buying environments as enduring engineering challenges for supporting millions of merchants. Planned work includes evolving the large Rails codebase into modular components, moving storefront rendering out of the monolith, expanding Spin cloud development environments, and making in-house load testing part of product development. The storefront split and refactoring produced four-times-faster cache fill rates and five-times-faster page rendering, while YJIT ran 31% faster than interpreted CRuby and 26% faster than MJIT; Shopify also remains committed to Rails and React Native.
The Adventures of Rendezvous in Heroku’s New Architecture
Heroku’s migration of Rendezvous, the bidirectional proxy that connects Heroku Run sessions to one-off dynos, from EC2 Classic to AWS VPCs caused one customer’s app to hit H12 timeouts. The move replaced a single Elastic IP ingress and process-level TLS termination with a Network Load Balancer, whose larger TLS frames exposed a blocking flaw in Rendezvous’s single-threaded Ruby channel loop. That loop used IO.select on the TCP socket and dyno terminal, but OpenSSL::SSL::SSLSocket could retain unread buffered data after a partial read; IO.select saw no TCP bytes and blocked beyond the Heroku Router’s 30-second limit. Engineers reproduced the behavior across two languages, instrumented the instance agent, then fixed the channel with separate threads and corrected socket-reading and IO.select handling. After staging tests, the customer moved back to VPC Rendezvous and confirmed the issue was resolved.
Search at Shopify—Range in Data and Engineering is the Future
Search at Shopify is presented as a case for range across data science and engineering, rather than narrow specialization in either discipline. The article describes a team that treats work as “search” instead of separating data and engineering responsibilities: engineers learn experimentation and model evaluation, while data scientists build performant, testable, maintainable production systems. It frames search as a sequence of trade-offs among relevance, correctness, scalability, performance, stability, and maintainability, arguing that people who understand only half the constraints can produce siloed handoffs, deployment delays, or suboptimal systems. Shopify’s stated model is a single collaborative team with shared planning and execution, intended to reduce territorial boundaries and support smart, fast, scalable search. The article extends this principle to personalized and conversational products, concluding that future work will require practitioners to move between data and engineering.
Top topics
- Architecture837
- Performance821
- LLMs696
- Machine Learning628
- AI Agents594
- Open Source576
- AI520
- Developer Experience448
Weekly digest
The best engineering reads, once a week.