Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,535 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Supabase Beta January 2022
Supabase announced product updates, platform improvements, and community resources released during January 2022. The platform expanded authentication options by adding Notion and LinkedIn OAuth support alongside Vonage and Textlocal SMS providers for one-time passwords. Observability tooling gained SQL querying capabilities for logs, timestamp filtering, and expanded time spans across project usage charts to accelerate issue diagnosis. Database infrastructure advanced with the release of pg_graphql version 0.1.0 introducing SQL Comment Directives, while PostgREST replaced pg_listen with built-in schema reloading and automated compute-based connection pool scaling. To streamline customer assistance, a priority selector was added to dashboard support forms with urgent tiers for paid projects.
Exposing a Kafka Cluster via a VPC Endpoint Service
To replace VPC peering and reduce attack surfaces, Grab exposed a multi-Availability Zone Apache Kafka cluster in its main AWS VPC to clients in a separate GrabKios VPC using AWS VPC Endpoint Service. Because Kafka requires clients to establish deterministic connections to individual brokers, the team configured a Network Load Balancer with unique TCP ports and dedicated target groups for each broker alongside a shared bootstrap port. They added custom listeners on the Kafka brokers to advertise endpoints using private Route 53 CNAMEs rather than raw interface hostnames. To eliminate unnecessary cross-AZ network latency and data transfer costs, the architecture was refined to advertise AZ-specific private CNAMEs mapped directly to zonal endpoint interfaces.
Changing a polymorphic_type in Rails
Rails normally persists a polymorphic association’s type as the parent class name, so moving Car under Garage changes stored values from “Car” to “Garage::Car” and can break lookups for older Key records. The post presents an alternative for the Vehicle/Key example: override polymorphic_name so Garage::Car and Garage::Boat resolve to stable arbitrary strings such as “car” and “boat,” reducing coupling between namespaces and database data. During transition, the association scope must explicitly accept both legacy class-name values and new arbitrary values, because Rails uses only the single value returned by polymorphic_name when querying; existing rows can then be cleaned up with a migration, script, or MaintenanceTasks. After cleanup, the temporary unscope can be removed, while class and module renames require updating the mappings rather than database records. The approach adds complexity and is presented as worthwhile only when near-term naming changes are likely.
How to Get an Engineering Internship at Shopify: A Complete Guide
Shopify’s guide describes its engineering internship program as a way for students, recent graduates, career switchers, and bootcamp graduates to gain hands-on experience through real projects and practical feedback. Internships last three to eight months and span front-end, back-end, infrastructure, data, and mobile development, with three application cycles each year. Applicants submit an online application, complete an Intern Challenge or specialization-specific skills assessment, and progress through a two-phase interview involving a Life Story conversation and a technical solution. The guide identifies collaboration, lifelong learning, curiosity, GitHub experience, remote-work experience, communication, and accountability as important skills, while recommending practice challenges, project portfolios, and company research; 98% of respondents in Shopify’s latest intern satisfaction survey said they would recommend the program.
How Grab built a scalable, high-performance ad server
Grab transitioned from an off-the-shelf MVP to an in-house ad serving system to accommodate business scale, hyperlocal requirements, and machine learning personalization. The architecture orchestrates core microservices and data pipelines across sequential steps: targeting, capping, pacing, scoring, ranking, pricing, and tracking. ElasticSearch serves as the targeting ads repository, while ScyllaDB acts as the high-throughput stats store fed by Kafka streams and data pipelines. The system operates on key engineering principles including parallelization and tuned latency limits, graceful fallbacks for slow dependency calls, and a unified server serving all ad types across the superapp.
Hydrogen & Tailwind: The Perfect Match for Building Beautiful Storefronts
Hydrogen uses Tailwind as the default styling option in its React-based starter, allowing developers to build storefronts with utility classes from the moment they run create-hydrogen-app. The article presents componentization as a central way to organize those classes: React components can encapsulate responsive, hover, flexbox, spacing, typography, and visibility utilities and be reused across a storefront. It contrasts this with semantic class systems whose names can become coupled to a product component, making reuse, renaming, or duplication more cumbersome. Tailwind’s learning curve still requires CSS knowledge, but its documentation and VS Code extension support autocomplete, style representations, and color previews. The conclusion is that Tailwind’s constraints, customizable design system, and composable styles complement Hydrogen’s starter components and developer experience, although the author acknowledges that readers may still prefer other approaches.
Shopify's Playbook for Scaling Machine Learning
Shopify describes how its machine-learning practice grew from a first order-fraud product into a portfolio used across products such as Shopify Capital, product categorization, and Help Center search. The playbook starts with choosing a user-important problem, validating trustworthy and accessible data, tracing downstream dependencies, understanding existing solutions, and optimizing for product outcomes rather than model scores alone. To productionize models, Shopify recommends well-defined training, verification, and testing pipelines, deployment decisions based on volume and user commitment, and platform practices that encode validated standards without abstracting too early. Its fraud pipeline builds models in Python, serializes them with PMML for deployment in Ruby-based production systems, and uses Apache Airflow to schedule operations. The account concludes that scaling is a full-stack problem requiring data scientists and data engineers to align on one roadmap and goal.
Fine-Tune ViT for Image Classification with 🤗 Transformers
Vision Transformer models bring transformer architectures to computer vision by splitting images into grids of sub-image patches and projecting them into token sequences. To classify healthy and diseased leaves using the beans dataset, the Hugging Face datasets and transformers libraries enable streamlined data ingestion and model fine-tuning. Preprocessing relies on ViTImageProcessor paired with lazy on-the-fly dataset transforms to dynamically generate normalized pixel tensors. Training leverages ViTForImageClassification with the Trainer API, requiring remove_unused_columns set to False so raw image data is preserved for batch collation. Over four training epochs, fine-tuning the google/vit-base-patch16-224-in21k checkpoint achieves an evaluation accuracy of 98.5% alongside an evaluation loss of 0.0637.
Our User-Mode WireGuard Year
Fly.io’s flyctl needed to let users SSH into Firecracker VMs and reach private 6PN networks without requiring root access or system-level WireGuard configuration. The solution runs wireguard-go, a userland TCP/IP stack derived from gVisor’s Go implementation, and golang.org/x/crypto/ssh inside flyctl, later coordinating sessions through a cross-platform background agent. The design initially exposed serious operational and usability problems: concurrent sessions displaced one another, agent behavior was fragile, and remote peer creation was slow, while deploys and remote builds expanded the mechanism’s role. The team improved concurrency, Windows support, DNS resolution, configuration handling, and peer-creation latency, and built an unmerged WebSocket path for networks blocking WireGuard’s UDP transport. The author concludes that the complexity was costly early in 2021 but enabled capabilities such as private DNS lookup and ping.
Turbocharge your Heroku Apps with New Turboku
New Turboku clones a Heroku app on Fly.io without requiring data migration, while the resulting app should be able to use services such as Heroku Postgres or Redis through transferred environment variables. The rebuilt workflow integrates flyctl with Fly.io’s remote builders instead of AWS CodeBuild: it queries Heroku, selects a region, converts config variables to secrets, and generates fly.toml, an ephemeral Dockerfile, and a reconstituted Procfile from the slug and process map. A Phoenix LiveView web launcher wraps the deployment by starting a Fly.io Machine running flyctl, while the CLI is available through flyctl turboku. The launcher also configures webhooks and watches environment-variable changes for automatic updates, whereas the CLI lacks webhook integration; the source reports about one millisecond of added database latency for a US-based app.
React Server Components Best Practices You Can Use with Hydrogen
React Server Components (RSC) introduced a paradigm shift for the team building Hydrogen, a React-based framework for custom storefronts. The post presents patterns developed through trial and error, beginning with shared components as a middle ground and reserving client components for interactivity, useState or useReducer, lifecycle logic, unsupported libraries, or browser APIs. Components should instead become server components when they contain secrets or proprietary logic, need filesystem or database access, fetch storefront API data, or do not execute on the client. The examples extract newsletter form controls and FAQ accordion behavior into client components, then rename the remaining wrappers as server components when their usage allows it; the FAQ server component is passed through a client component as children.
Rapid Development with Hydrogen: Building a Product Page
Updated for Hydrogen 0.26.0, the tutorial builds a simple product page for a custom Shopify storefront using Hydrogen’s React-based framework. Starting from the Hello World template on StackBlitz, it adds Tailwind CSS, creates a file-based dynamic product route, and queries product data through the Storefront API with the useShopQuery hook. The implementation then uses React Server Components alongside a client component and ProductOptionsProvider to track selected variants and options, update prices, and support a BuyNowButton checkout flow. A media gallery and additional styling complete the page, which uses products and media from the Hydrogen Preview store. The post concludes that Hydrogen’s components, hooks, and utilities reduce repetitive storefront work and speed development.
Getting Started with Sentiment Analysis using Python
Modern natural language processing tools allow developers to perform sentiment analysis without deep machine learning expertise. Sentiment analysis categorizes text polarity into positive, negative, or neutral labels to extract insights from large volumes of social posts, reviews, and support tickets. Using the Hugging Face Transformers library and the Hub, practitioners can quickly run pre-trained transformer pipelines or target specific multilingual and emotion-detection models with minimal Python code. For custom requirements, developers can fine-tune models like DistilBERT using the IMDB dataset via the Trainer API or train models automatically with AutoNLP. Applying these workflows to social media data enables automated sentiment distribution analysis and visualization using pandas, matplotlib, and word clouds.
LiveBeats: Building a social music app with Phoenix LiveView
LiveBeats is an open-source social music application built as a full-stack Phoenix reference app and learning test bed for LiveView features. It demonstrates a shared real-time UI for playlist playback, presence, GitHub OAuth, and concurrent MP3 uploads, using Phoenix PubSub and the existing LiveView WebSocket connection rather than separate REST, JSON, or polling layers. Uploads are processed concurrently in temporary files to verify MP3 validity and enforce a duration below 20 minutes, while HEEx function components and slots provide reusable UI and JS commands keep immediate interactions such as modal hiding on the client. The application runs across five continents with automatic clustering, file proxying, and regional Postgres read replicas; the post presents this as a way to deliver responsive global behavior with relatively minor changes.
Making automatic speech recognition work on large files with Wav2Vec2 in 🤗 Transformers
Transformer-based automatic speech recognition models like Wav2Vec2 crash on long audio inputs because the O(n²) attention mechanism rapidly exhausts GPU memory. Simple audio chunking avoids out-of-memory errors but causes poor transcription accuracy at chunk boundaries due to a lack of surrounding context. To resolve this, Hugging Face Transformers leverages the Connectionist Temporal Classification architecture to process overlapping audio chunks with configurable strides. The pipeline discards low-quality logits at chunk edges and chains the remaining central predictions together to reconstruct a seamless transcript. This striding technique operates out of the box with language model-augmented pipelines and adapts directly to low-latency live audio streaming.
Announcing Intercom’s new program for venture-backed startups
Intercom announces a startup program aimed at fast-growing, venture-backed Seed and Series A companies, working with venture capital, accelerator, incubator, and technology partners. The program provides its main customer communication solutions at no cost in the first year, 50% off in the second, and 20% off thereafter, covering tools for bots and content, in-product messaging, onboarding, and conversational engagement. It extends Intercom’s early-stage program, which offers a 95% discount and has helped thousands of pre-seed startups, while focusing more closely on companies scaling with venture backing. Intercom says more than 5,000 startups from leading incubators use its products and cites Spendesk, Notion, Monzo, Frame.io, and Aircall among startup customers; it also plans webinars and events and invites partners to apply.
Fixing the Dependency Confusion Vulnerability in 600+ Ruby Apps - Shopify
Shopify identified a dependency confusion vulnerability through its bug bounty program; an attacker could exploit ambiguous package sources to install a malicious dependency and gain access to local, CI/CD, and production environments. To assess exposure, the Ruby Conventions team collected Bundler versions from repositories in a data lake and found around 600 susceptible Ruby applications. An experimental Bundler plugin converted Gemfile.lock files to the explicit-source format while restoring existing gem versions, and a CI-backed CLI later provided per-repository environments, parallel execution, pull requests, and easier failure tracking. After incremental migrations exposed configuration and tooling issues, Shopify updated supporting systems, set Bundler requirements, changed local and CI defaults, and worked with Bundler maintainers; the company reports migrating all its applications in less than a day and contributing to Bundler 2.2.21 and 2.2.22.
Supercharged Searching on the 🤗 Hub
Programmatically searching the Hugging Face Hub previously required navigating web browser widgets or enduring trial and error to guess exact query string formats. The huggingface_hub library solves this issue by introducing helper utilities such as ModelSearchArguments, DatasetSearchArguments, and ModelFilter alongside HfApi. These namespace helpers translate accessible Python attributes into the formatted parameters expected by the backend API, covering datasets, tasks, and libraries. For complex queries across multiple tasks, frameworks, and datasets, developers can combine criteria inside ModelFilter instances and retrieve matching model metadata via api.list_models. Under the hood, the library uses AttributeDictionary, a data structure inspired by fastcore that enables nested tab-completion while supporting standard dictionary key indexing for special characters.
Welcome Stable-baselines3 to the Hugging Face Hub 🤗
Hugging Face announced an official integration with Stable-Baselines3, a popular PyTorch library for training and testing Deep Reinforcement Learning agents across diverse environments like Gym, Atari, MuJoco, and Procgen. The integration enables researchers and developers to host their saved reinforcement learning checkpoints on the Hugging Face Hub and download pre-trained community models. Interacting with the Hub requires installing the huggingface_hub and huggingface_sb3 packages, which supply helper methods for authentication, downloading, and uploading. Practitioners can retrieve checkpoint zip files using the load_from_hub function by providing the target repository identifier and filename before loading them into Stable-Baselines3 algorithms. Furthermore, users authenticated via CLI or notebook login can train policies such as PPO and publish their saved zip files to the Hub using push_to_hub.
Supabase Beta December 2021
Supabase announced a series of platform updates, educational content, and community milestones for December 2021. PostgreSQL instances now include the pg_sodium extension, enabling users to perform encryption, decryption, hashing, and cryptographic signing directly within database queries and functions through the Supabase Dashboard. The release details educational video guides demonstrating remote procedure calls from JavaScript, invoking external HTTP endpoints from database functions, and setting up triggers to run SQL on table modifications. In addition to ecosystem updates featuring new Python library releases and integrations with Divjoy and n8n, the project surpassed 26,000 GitHub stars and opened fully remote job positions across engineering, marketing, and human resources.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.