Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,535 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Graph Networks - Striking fraud syndicates in the dark
Grab's Integrity team faced escalating threats from organized fraud syndicates operating device farms and GPS spoofing tools that mimic normal user behavior. To overcome the limitations of traditional rule-based systems, Grab built a Graph-based Prediction Platform that links entities across shared attributes to expose hidden network structures. The architecture includes a scalable graph database platform holding over one billion nodes, powering real-time graph visualization for analysts alongside a configurable network feature extraction system. In addition, the team deployed Graph Neural Networks to capture structural correlations across transaction networks. These systems collectively help detect coordinated promo abuse, payment fraud, and complex money laundering schemes.
Opinion Classification with Kili and HuggingFace AutoTrain
Understanding mobile application user feedback often requires expensive and labor-intensive manual analysis. To streamline this process, an active learning pipeline was constructed using Kili for data annotation alongside HuggingFace AutoTrain and standard transformers for text classification. The workflow utilized approximately 40,000 Medium reviews collected from the Google Play Store, classifying content into categories such as Subscription, Content, Interface, and User Experience. Hyperparameter searches with Ray backend were compared against AutoTrain, which automated training within 30 minutes. Subsequent sentiment analysis of the categorized reviews revealed that while overall sentiment remained positive, subscription and interface categories contained predominantly negative feedback, especially in application version 4.5.
Data Is An Art, Not Just A Science—And Storytelling Is The Key
Data science combines science, mathematics, and art, but communication is presented as an often-neglected skill that helps stakeholders understand and act on analysis. Effective data storytelling goes beyond charts and dashboards by framing a business problem as the main character, supplying context, communicating in the audience’s language, and directing action through next steps. The article recommends investing in clean, conformed data foundations and formal support, including Shopify’s data storytelling working group, while also borrowing communication techniques such as Duarte’s Slidedocs visual reports. It further advocates interactive experiences that let audiences filter, drill into, and explore data, illustrated by Shopify’s BFCM 2021 Notebook, which helped merchants examine sales, products, days, and locations. The conclusion is that storytelling can build trust and increase data fluency.
Director of Machine Learning Insights
Directors of machine learning must balance complex mathematical modeling, system design, and sector-specific business operations across diverse industries. In digital media, practitioners leverage deep learning and Bayesian techniques to power assistive content creation tools and short-term interest recommendations while maintaining user privacy. Pharmaceutical teams employ natural language understanding, computer vision, and reinforcement learning across research and manufacturing, requiring strict GxP validation to navigate regulatory compliance and dataset diversity challenges. In the utility and energy sector, machine learning bridges disparate relational billing records, spatial GIS feeds, and physical grid machinery data to generate operational insights. Across these domains, leaders emphasize that successful integration depends on data quality validation, model monitoring safeguards, and assistive human-in-the-loop collaboration.
Getting Started with Transformers on Habana Gaudi
Habana Labs and Hugging Face partnered to accelerate Transformer model training using Habana Gaudi accelerators, which offer up to 40% better price performance than recent GPU-based Amazon EC2 instances. Setting up the environment requires launching an Amazon EC2 dl1.24xlarge instance using the Habana Deep Learning Base AMI on Ubuntu 20.04. Inside an interactive PyTorch Docker container configured for Habana runtimes, users install the Optimum Habana package from source along with necessary dependencies. Fine-tuning the bert-large-uncased-whole-word-masking model on the GLUE MRPC task completes in 2 minutes and 12 seconds with an F1 score of 0.9181. Utilizing EC2 Spot Instances reduces the hourly compute cost by 70%, from $13.11 down to $3.93.
Accessibility and Real-time Apps: Clearing Fog and Picking Fruit
This post describes an initial accessibility pass on LiveBeats, a Phoenix LiveView reference app with real-time social features, focused on screen reader users before real-time updates are introduced. It recommends labeling meaningful controls and images with visible text, alt, SVG desc, or aria-label, while hiding decorative imagery with aria-hidden="true" or blank alt attributes. It then explains that semantic HTML such as button, main, nav, and article is preferable to ARIA roles where available, while named regions can improve navigation and orientation. Roles also create behavioral obligations: a role="button" control must support focus and Enter or Space activation, and role="application" can force focus mode, blocking caret-browsing conveniences unless the app supplies comprehensive interaction behavior. The initial work clears foundational accessibility issues, with later work reserved for presenting live changes accessibly.
Introducing Hugging Face for Education 🤗
Hugging Face established an initiative to teach machine learning to 5 million people by the end of 2023. As machine learning expands across software development and impacts non-technical users, adapting workforce skills and assisting educators with ethical considerations become critical challenges. The educational framework provides browser-based model testing widgets, documentation on model biases, and free courses covering Natural Language Processing, Deep Reinforcement Learning, and interactive demo creation. Instructors receive free infrastructure to create Hub classrooms, alongside a multilingual toolkit spanning topics like Transformers and Gradio. Through ongoing global tours and free compute events, the organization supports collaborative machine learning education across diverse experience levels.
Supercharged Customer Service with Machine Learning
Customer support teams often receive high volumes of messages that cannot all be answered manually. To prioritize urgent inquiries, support workflows can be modeled as a text classification task to identify the most unsatisfied customers. Using the Hugging Face ecosystem, an NLP pipeline is established by selecting the Amazon reviews multi dataset and fine-tuning a DeBERTa model for sentiment classification across five granular categories. Evaluation on test data shows that the model identifies roughly 95% of unsatisfied messages with an 11.7% false-positive rate on satisfied messages, potentially reducing human triage workload by 83%. For production deployment, performance can be optimized through hardware acceleration, lower precision arithmetic, open-source libraries like Optimum and ONNX Runtime, and inference servers.
Building a Business System Integration and Automation Platform at Shopify
Shopify needed a maintainable way to connect increasingly modular business systems whose interfaces ranged from SOAP and FTP to REST and GraphQL. The company evaluated Mulesoft but found that its proprietary XML and Java virtual machine stack isolated the team from Shopify’s development ecosystem and complicated access to internal services and infrastructure. It instead built a Ruby on Rails integration platform around minimal validation, transparent data transformations, flexible client support, security, reusable components, documentation, and request-level testing with WebMock. The implementation favors direct manipulation of JSON hashes because that keeps mappings visible, while acknowledging risks such as typos and complex mock management. After a year, the platform had 18 integrations, had migrated all Mulesoft applications, and enabled other teams to build integrations, with ongoing work on testing and onboarding.
CO2 Emissions and the 🤗 Hub: Leading the Charge
Training and deploying machine learning models produces greenhouse gas emissions determined by infrastructure runtime, hardware type, and energy grid carbon intensity. Hugging Face introduced mechanisms to measure, report, and filter models based on their carbon footprint to improve ecological transparency. Using the huggingface_hub library, practitioners can query HfApi.list_models with an emissions_thresholds parameter to filter models by maximum or minimum carbon emissions measured in grams. In addition, installing the codecarbon package enables the Transformers Trainer to automatically attach CodeCarbonCallback and log training emissions to an emissions.csv file. Users can then publish these measured values directly into model card metadata under co2_eq_emissions for public tracking.
Works With Supabase - announcing our Partner Gallery
Supabase launched its Partner Gallery to showcase integration partners and certified expert agencies assisting developers on the platform. The open-source gallery is built using the Supabase stack, featuring Postgres Full Text Search, next-image with Supabase Storage, and email triggers via Supabase Edge Functions. Initial agency partners include Morrow, a UK app development agency specializing in React Native and Supabase, alongside Quinence, a Singapore-based product development studio. Integrated tooling covers authentication systems like Clerk and Stytch, database and developer utilities like Prisma, Snaplet, Vercel, and pgMustard, and low-code builders including Appsmith, Draftbit, Plasmic, and Teta. These ecosystem integrations allow developers to manage Postgres workflows, secure user authorization, and build applications across web and mobile platforms.
Logbook - 2022-04-20
Fly.io’s April 20, 2022 logbook combines a recent changelog with an account of how the company produces updates for a distributed platform running apps in VMs on hardware around the world. It records added IAD capacity, backup regions disabled by default, flyctl releases with NuxtJS and RedwoodJS deployment support, and reliability improvements to postgres operations, failover handling, error reporting, builders, logs, and releases. The post says commits and status updates often fail to capture completed user-relevant work or its impact. It therefore asks engineers to write updates that explain both what changed and why it matters. Collection is automated through a Slack bot and an app that collates entries into feeds, while the team plans to cultivate stronger changelog habits.
Möbius: Shopify’s Unified Edge
Shopify had more than six traffic paths for its core, services, PII-restricted systems, and mTLS-authenticated APIs, making platform-wide features harder to deploy and traffic behavior harder to observe. Möbius and Shopify Front End (SFE) were designed together: Möbius terminates TCP and TLS near clients and forwards requests to SFE, while SFE routes them from regional clusters to application clusters. SFE uses nginx deployments with custom Lua and a centralized API fed by Kubernetes controllers, enabling routing and active/active or active/passive load balancing without DNS TTL delays. Möbius automates edge domain configuration, DNS ownership validation, and certificate acquisition through vendor APIs, so application owners generally only need to change DNS. The unified design standardizes edge protection and observability, supports future edge capabilities, and leaves exceptions for services whose dependencies could create crash loops.
How we reduced our CI YAML files from 1800 lines to 50 lines
Grab's Cauldron Machine Learning Platform team managed continuous delivery across multiple pipelines by using nested GitLab CI configurations, but soon encountered platform limitations such as the 100-include ceiling and bloated 1,800-line YAML files. To address these constraints, the team implemented GitLab Dynamic Child Pipelines to programmatically create execution stages on the fly. They built a command-line utility in Rust that runs git diff against the base branch, extracts pipeline and stage metadata using configurable stop patterns, and applies a template to produce a dynamic CI definition. On the master branch, the tool fetches the diff artifact from the source branch through the GitLab API to execute identical generation logic. This architectural shift reduced the root CI file from 1,800 lines down to a constant 50 lines, allowing configuration size to remain stable regardless of repository growth.
Bring the Func Hackathon Winners 2022
Supabase announced the winners and runners-up of the 2022 Bring the Func Hackathon following their recent launch week. The event featured multiple project categories, showcasing applications built by community developers with Supabase tools. In the main showcase, mintbaseXsupabase won for an Instagram-like NFT browsing platform, while Pgm Dictionary served as runner-up with an UrbanDictionary clone built on Next.js, Mantine, and Supabase. The Edge Functions category highlighted Wen NFT for managing NFT community access and runner-up DocuPool for token-unlocked document sharing. Finally, AbileneX won the Most Fun or Visually Pleasing category for gathering feedback on ideas, alongside project management tool Kardow as runner-up.
Supabase Beta March 2022
Supabase announced several major platform updates for March 2022, introducing global edge deployments for JavaScript and TypeScript functions developed and tested locally. Database querying expanded to support GraphQL through an open-source, portable Postgres extension. Supabase Realtime received support for multiplayer and presence to enable collaborative applications, alongside enterprise and scale capabilities including Point-In-Time-Recovery, SLAs, premium support, a Prometheus endpoint, and an observability dashboard. To enhance security, an integration with GitHub secret scanning automatically notifies developers and rolls exposed service_role keys found in public repositories. Additional updates include the PostgREST 10 pre-release, Next-JS auth helpers, new OAuth providers, and full remote hiring across engineering, marketing, and human resources.
Code Ranges: A Deeper Look at Ruby Strings
Ruby's string model supports mutable and immutable operations, per-object or per-file freezing, and more than 100 encodings, while permitting byte sequences invalid for their attached encoding. This flexibility helps legacy and unusual platforms but makes encoding checks, validity scans, and character-boundary handling part of many operations. To reduce repeated O(n) validation scans, Ruby implementations cache each string's state in a four-valued code range: unknown, 7bit, valid, or broken. The post explains how these values map to public methods, how MRI stores them in two header bits, and how JRuby and TruffleRuby represent them differently. It also describes code ranges as inputs to fast and slow paths and as useful but fallible cache information, especially for native extensions and string-heavy metaprogramming.
Machine Learning Experts - Lewis Tunstall
In this interview with Britney Muller, Hugging Face machine learning engineer Lewis Tunstall discusses his background, his book Natural Language Processing with Transformers, and tooling optimizations for deploying transformer models. Tunstall notes that large model parameter sizes often lead to high latency, making off-the-shelf transformers impractical for responsive applications like chatbots. To resolve this issue, he developed single-line export tooling within the transformers library to convert models into the ONNX format. This conversion enables engineers to run PyTorch models in TensorFlow or on specialized hardware, achieving faster latency and higher throughput. Tunstall also recounts co-authoring the O'Reilly book with Leandro von Werra and Thom Wolf, collaborating on the Hugging Face Course, and training a GPT-2 model named code parrot on Python code.
Habana Labs and Hugging Face Partner to Accelerate Transformer Model Training
Training transformer models across computer vision, speech, and natural language processing tasks at scale often demands heavy compute resources, incurring significant time and financial expense. To address these bottlenecks, Habana Labs and Hugging Face partnered to integrate the SynapseAI software suite into the Hugging Face Optimum open-source library. This integration allows machine learning practitioners to accelerate transformer training workflows on Habana Gaudi processors using minimal code adjustments. Habana Gaudi hardware, featured in Amazon EC2 DL1 instances and Supermicro X12 servers, incorporates ten 100 Gigabit Ethernet ports per processor to scale from single units to thousands of chips. The combined hardware and software architecture supports TensorFlow and PyTorch while delivering price and performance metrics up to 40% lower than comparable training alternatives.
Leveraging Shopify’s API to Build the Latest Marketplace Kit
Marketplace Kit version 2 provides boilerplate app code and documentation for third-party developers integrating with Shopify, implementing selected commerce features, and launching marketplaces across channels. The team moved away from using the Shopify CLI-generated Node app because CLI changes could affect their code and documentation, while its dependencies limited control over development practices. The new kit separates a full-stack merchant-facing admin app from a buyer-facing client app, using Express, React, NextJS, GraphQL, Apollo Client, and Apollo Server; open-source versions are publicly available. Its examples demonstrate JWT authentication through App Bridge and Shopify’s Node API, serving REST and GraphQL together with Express, and choosing REST for a product-listing count where a query offered no advantage.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.