Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,671 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Gradio 3.0 is Out!
Gradio 3.0 introduces a ground-up redesign of the Gradio library to improve machine learning demo development and browser performance. The frontend has been rebuilt using modern technologies such as Svelte, resulting in smaller payloads, faster page load times, and cleaner visual designs suitable for embedding in blog posts. Component updates include revamping the Dataframe component to support CSV drag-and-drop, introducing new UI elements like Gallery, and adding the TabbedInterface class to group demos under distinct tabs. Developers also gain access to Gradio Blocks, a low-level Python API enabling custom layouts, multi-step interfaces, dynamic component property modifications, and complex data flows. A competition called the Gradio Blocks Party has been organized through the end of May to encourage community adoption.
Logbook - 2022-05-13
The May 13, 2022 log records Fly.io changes and fixes from the preceding week, spanning infrastructure capacity, account self-service, API maintenance, documentation, and customer-facing behavior. New servers in syd, iad, and dfw, plus three workers in ord, addressed capacity shortages that had caused provisioning or deployment problems. The API moved to Ruby 3.1, with anticipated performance and security-patch benefits, while an internal Elixir UI guide and clearer app and machine name validation errors aimed to reduce development and debugging friction. Operational work permanently disabled consul-templaterb on fly-proxy nodes and fixed a fly-proxy memory leak, giving those nodes more CPU/RAM headroom; an MTU workaround was intended to improve IPv6 tunnel connectivity, while a larger fix remains. Other fixes covered account-deactivation guidance, personalized support email addresses for paid plans, billing, volume metrics, forum registration, and a minor “X daysago” display error.
Maestro: The Orchestration Language Powering Shopify Flow
Shopify developed Maestro, a domain-specific orchestration language, to decouple Shopify Flow’s merchant-facing visual editor from its execution engine, whose requirements differ in declarative usability versus scalable, fault-tolerant execution. The team introduced a horizontally scalable Maestro engine, translated existing Flow workflows into Maestro, migrated them incrementally, and later connected a new visual language directly to Maestro; all workflows had moved to the new editor and engine by early 2022. Maestro coordinates calls to host-language functions rather than implementing service or database operations, while durable checkpoints capture completed work and let the interpreter recover after crashes with at-least-once semantics for primitive calls. During activation, static usage analysis identifies needed returned attributes for possible query optimization, and compilation removes runtime-irrelevant definitions before deployment; checkpoints also support execution monitoring, while planned work includes richer language features, orchestration merging, and workflow preview and testing.
Director of Machine Learning Insights [Part 2: SaaS Edition]
Engineering directors across SaaS organizations share operational experiences regarding the integration, maintenance, and impact of machine learning systems. Deploying production ML introduces significant operational overhead beyond initial modeling, requiring teams to manage data drift, resolve siloed data sources, and adapt to vertical-specific domain shifts. Leaders emphasize that integrating ML effectively often requires prioritizing core business requirements and traditional ML techniques over newly released, complex algorithms. Allowing end users to incorporate domain knowledge and business controls directly into model interactions remains critical for trust and usability. Looking forward, these leaders anticipate substantial ML-driven operational improvements across enterprise workflows, cybersecurity threat detection, and connected healthcare infrastructure.
Student Ambassador Program’s call for applications is open!
Hugging Face launched its inaugural Student Ambassador Program to advance its mission of democratizing machine learning globally. The initiative aligns with the organization's overarching goal to teach machine learning to 5 million people by 2023. Selected student ambassadors will receive dedicated workshops, direct team support, access to a peer network, merchandise, and updates on upcoming projects and features. Applicants must be enrolled in an accredited higher education institution, have completed at least one machine learning or data science course, and already use the Hugging Face Hub or libraries. The application window closed on June 13, 2022, with the program operating from June 30, 2022, through December 31, 2022.
Our Experience Porting the YJIT Ruby Compiler to Rust
Shopify’s YJIT team moved the CRuby JIT compiler from C99 to Rust after finding that C’s limited complexity-management tools made the growing project difficult to maintain. The port had to preserve a non-trivial interface with CRuby, including parsing bytecode and manipulating Ruby’s primitive types, while compiling the Rust project into a static library linked through CRuby’s makefile. The team translated functions and structs largely one by one, retaining the original architecture while using Rust features such as pattern matching, macros, Cargo features, and borrow checking. Completed in three months, the port was judged more maintainable and pleasant to work with, although the team identified challenges around C interoperability, unsafe code, tooling, ecosystem maturity, and limited use of idiomatic Rust.
Accelerated Inference with Optimum and Transformers Pipelines
Transformer-based models are moving into production for large-scale workloads, but default implementations remain slow and resource-intensive compared to traditional machine learning algorithms. Hugging Face Optimum addresses this issue by integrating performance optimization tools and accelerated runtimes like ONNX Runtime directly into standard Transformers pipelines. Developers replace standard model classes with Optimum equivalents to convert models to ONNX, run graph optimizations such as operator fusion, and apply quantization. In an evaluation on an AWS EC2 m5.xlarge CPU instance, optimizing and quantizing a RoBERTa question-answering model cut average latency from 117.61 ms to 64.94 ms while retaining 99.61 percent accuracy. Known limitations include a 2GB remote model size limit, lack of seq2seq support, omitted past key values in causal language models, and no local caching for optimized ONNX models.
I'm All-In on Server-Side SQLite
Ben Johnson argues that SQLite should be considered as the core data and persistence layer for many production full-stack applications, rather than being relegated to unit tests in an n-tier architecture. He attributes this case to cheaper, faster hardware and the fact that most successful applications will never approach terabyte-scale data, making distributed database infrastructure excessive for some workloads. Litestream works alongside standard SQLite by taking control of WAL-mode journaling, capturing WAL updates, replicating them to storage such as S3, and handling checkpoints itself. This preserves ordinary SQLite libraries and transactions while providing a path to restore copies and improve resilience to storage failure; upcoming direct database-to-database replication is also mentioned. The resulting model places data in the application process, where the post reports 10–20 microsecond per-query latency, but it remains aimed at single-node applications and has restore, deployment, and replication-guarantee limitations.
We Raised $100 Million for Open & Collaborative Machine Learning 🚀
Hugging Face announced a 100 million dollar Series C funding round led by Lux Capital, alongside participants including Sequoia, Coatue, and several angel investors. The platform hosts over 100,000 pre-trained models and 10,000 datasets spanning natural language processing, computer vision, speech, and other scientific domains for more than 10,000 customer organizations. The company plans to use the capital to expand research, open-source initiatives, tooling, and efforts focused on mitigating machine learning limitations like biases, privacy risks, and energy consumption. In addition, the organization is directing the BigScience research workshop with over 1,000 participants to train a massive open-source multilingual language model. Following organizational growth from 30 to more than 120 employees in twelve months, the remote-friendly company is actively hiring across all positions and seniority levels.
Talk Nerdy to Me: 10 Tech Podcast Recommendations from Shopify Engineering
Shopify Engineering curated ten podcast recommendations from conversations with its technology employees, extending an earlier roundup of book recommendations. The list spans software engineering, front-end development, JavaScript, career growth, cybersecurity, data science, and broader technology topics, with selections ranging from quick, technical episodes to witty, conversational shows. Several entries emphasize specific formats or practices: Software Engineering Daily conducts researched interviews and focuses on open source, Soft Skills Engineering answers two listener-submitted questions in 30 minutes, and Darknet Diaries covers hacking, cybercrime, and data breaches. The recommendations also include podcasts valued for approachable explanations, collaborative software work, diverse guests, and perspectives on leadership and professional development, while some contributors identify particular episodes or hosts as reasons they keep listening.
Welcome fastai to the Hugging Face Hub
Hugging Face Hub has integrated with the fastai deep learning library, allowing practitioners to upload and download models directly through the huggingface_hub Python client. Fastai Learner objects, which bundle model architectures, data loaders, and loss functions, can be published to a designated repository namespace using push_to_hub_fastai. Uploaded models automatically receive Git-based version control, Git LFS support for large files, and generated model cards. Practitioners can retrieve and instantiate published models locally for inference or transfer learning via from_pretrained_fastai. The integration also supports Blurr, an ecosystem library that combines fastai with Hugging Face Transformers for sequence classification and related text modeling tasks.
Logbook - 2022-05-05
Fly.io’s 2022-05-05 Logbook reports a broad set of infrastructure, product, security, documentation, and bug-fix updates intended to run customer apps faster and more reliably in VMs close to users worldwide. Remote builders now use regional Docker Hub mirrors, clean unused RUN --mount=type=cache build caches, and perform cleanup at startup and shutdown; the new behavior is default for new builders, while existing builders must be destroyed to receive it. WireGuard gateway improvements address load issues, add regional capacity, and support future redundancy, while another IAD gateway is intended to improve connectivity for customer 6pn networks and CI, especially GitHub Actions. The update also adds API security logging for SOC2, re-enables OpenTelemetry, speeds the proxy build pipeline, changes default persistent volume size from 10GB to 3GB, and ships a dashboard, payment retry feature, VSCode fly.toml validation and documentation changes, alongside GraphQL performance work and fixes for monitoring, invoices, encryption, and Homebrew updates.
Embracing a Docs-as-Code approach
Engineers at Grab faced difficulties finding and writing technical documentation because materials were fragmented across various platforms and disconnected from daily workflows. To establish a single source of truth, the organisation adopted a Docs-as-Code approach starting in 2021. Under this system, documentation is written in plaintext Markdown within dedicated folders inside each service's Git repository. An internal developer portal automatically builds and publishes documentation whenever updates are committed, integrating documentation maintenance into the standard code development and review workflow. By April 2022, approximately 80% of Grab services were projected to host their documentation on the internal portal, with the model also expanding to technical training content.
An Introduction to Deep Reinforcement Learning
This introductory chapter presents deep reinforcement learning as a machine-learning approach in which an agent learns by interacting with an environment, taking actions, and receiving positive or negative rewards. It establishes the reinforcement-learning framework through the loop of state, action, reward, and next state, and explains the reward hypothesis, expected return, Markov property, observations, states, and action spaces. The chapter then introduces policies as the agent’s decision-making mechanism and contrasts policy-based methods with value-based methods, including the use of neural networks to estimate policies or state values in deep RL. It concludes by preparing learners to train a lander agent, share it on the Hugging Face Hub, and continue with Q-Learning and value-based methods.
Accelerate Large Model Training using PyTorch Fully Sharded Data Parallel
Increasing parameter counts in modern machine learning models make loading and training them prohibitive on standard hardware. While Distributed Data Parallel replicates the entire model across GPUs, PyTorch Fully Sharded Data Parallel shards optimizer states, gradients, and parameters across data-parallel workers. Benchmarks on causal language modeling using GPT-2 Large and XL across Titan RTX GPUs demonstrate that FSDP increases allowable batch sizes and prevents out-of-memory errors encountered in standard DDP. Integrating FSDP through Hugging Face Accelerate enables these capabilities alongside CPU offloading to run models exceeding GPU memory limits. However, Accelerate requires preparing models before creating optimizers when using multiple models and notes that mixed precision support with FSDP currently has transformer compatibility issues.
React Native Skia—For Us, For You, and For Fun
React Native Skia brings Skia’s cross-platform 2D drawing primitives to React Native, with the stated goals of improving its limited graphical capabilities and aligning designers and developers around capabilities comparable to Figma. The integration uses React Native’s JavaScript Interface (JSI), enabling direct JavaScript-to-native communication through C++ rather than asynchronous messages, and exposes an API described as virtually 100% compatible with Flutter’s. On top of the imperative API, a declarative layer uses the React Reconciler to diff internal representation states and send changes to the Skia engine, supporting composable drawings and declarative optimizations. The library includes image filters, shaders, SVG, path operations, vertices, text layouts, mesh gradients, shadows, backdrop blur, and animations, with a direct connector to Reanimated 2; the authors expect it to address a major React Native pain point and describe the work as just beginning.
Graph Networks - Striking fraud syndicates in the dark
Grab's Integrity team faced escalating threats from organized fraud syndicates operating device farms and GPS spoofing tools that mimic normal user behavior. To overcome the limitations of traditional rule-based systems, Grab built a Graph-based Prediction Platform that links entities across shared attributes to expose hidden network structures. The architecture includes a scalable graph database platform holding over one billion nodes, powering real-time graph visualization for analysts alongside a configurable network feature extraction system. In addition, the team deployed Graph Neural Networks to capture structural correlations across transaction networks. These systems collectively help detect coordinated promo abuse, payment fraud, and complex money laundering schemes.
Opinion Classification with Kili and HuggingFace AutoTrain
Understanding mobile application user feedback often requires expensive and labor-intensive manual analysis. To streamline this process, an active learning pipeline was constructed using Kili for data annotation alongside HuggingFace AutoTrain and standard transformers for text classification. The workflow utilized approximately 40,000 Medium reviews collected from the Google Play Store, classifying content into categories such as Subscription, Content, Interface, and User Experience. Hyperparameter searches with Ray backend were compared against AutoTrain, which automated training within 30 minutes. Subsequent sentiment analysis of the categorized reviews revealed that while overall sentiment remained positive, subscription and interface categories contained predominantly negative feedback, especially in application version 4.5.
Data Is An Art, Not Just A Science—And Storytelling Is The Key
Data science combines science, mathematics, and art, but communication is presented as an often-neglected skill that helps stakeholders understand and act on analysis. Effective data storytelling goes beyond charts and dashboards by framing a business problem as the main character, supplying context, communicating in the audience’s language, and directing action through next steps. The article recommends investing in clean, conformed data foundations and formal support, including Shopify’s data storytelling working group, while also borrowing communication techniques such as Duarte’s Slidedocs visual reports. It further advocates interactive experiences that let audiences filter, drill into, and explore data, illustrated by Shopify’s BFCM 2021 Notebook, which helped merchants examine sales, products, days, and locations. The conclusion is that storytelling can build trust and increase data fluency.
Director of Machine Learning Insights
Directors of machine learning must balance complex mathematical modeling, system design, and sector-specific business operations across diverse industries. In digital media, practitioners leverage deep learning and Bayesian techniques to power assistive content creation tools and short-term interest recommendations while maintaining user privacy. Pharmaceutical teams employ natural language understanding, computer vision, and reinforcement learning across research and manufacturing, requiring strict GxP validation to navigate regulatory compliance and dataset diversity challenges. In the utility and energy sector, machine learning bridges disparate relational billing records, spatial GIS feeds, and physical grid machinery data to generate operational insights. Across these domains, leaders emphasize that successful integration depends on data quality validation, model monitoring safeguards, and assistive human-in-the-loop collaboration.
Top topics
- Architecture834
- Performance816
- LLMs660
- Machine Learning622
- Open Source563
- AI Agents553
- AI470
- Developer Experience448
Weekly digest
The best engineering reads, once a week.