Loading…
Latest reads
Every post, summarised
Grounded summaries of engineering blogs, newest first. Each links to the original.
- 4,535 posts
- 20 companies
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Latest posts
Search architecture revamp
Before 2021, Grab’s Deliveries search architecture used an inverted index for textual matching, leaving it unable to account for user intention, context, or non-textual recall methods. Queries such as “Roti Prata” could prioritize merchant names over dishes, while the same query could imply different objectives by time or location; merging Ads and organic results at the Deliveries API layer also discarded recall-stage data. The revamped flow adds boosting, multi-recall, and ranking, enabling intent recognition, query expansion, embedding-based and graph-based recall, and joint ranking across business verticals. A platform-defined framework assembles service-team plugins through configuration, separating core flow from application logic and allowing components such as Ads and Mart recall to be reused across Food, Mart, and universal search flows. The resulting platform provides intent recognition, semantic search, mixed Ads-and-organic ranking, and simpler onboarding, while making new search modules and use cases easier to add.
Machine Learning Experts - Sasha Luccioni
Research scientist Sasha Luccioni discusses her work on evaluating the environmental, societal, and ethical impacts of machine learning models. Her research includes measuring and reducing artificial intelligence emissions through initiatives such as CodeCarbon, regional grid selection, and the BigScience carbon footprint working group. BigScience aims to make large language models transparent by curating multilingual datasets, assessing manufacturing and compute costs, and analyzing participant workloads. Luccioni also emphasizes expanding model evaluation criteria beyond standard accuracy metrics to encompass efficiency, emissions, and subset-level fairness across demographics and data features. Overall, these efforts aim to democratize artificial intelligence and encourage deliberate trade-offs between model performance and computational resource demands.
Announcing the Hugging Face Fellowship Program
Hugging Face announced the launch of the Hugging Face Fellowship Program to support and empower prominent contributors across the open-source machine learning ecosystem. The initiative provides selected fellows with personalized resources according to their project goals, including computing power, merchandise, and official recognition from Hugging Face. Candidate selection operates on an ongoing basis through nominations made by Hugging Face team members or existing fellows, evaluated primarily on past contributions toward democratizing machine learning. Eligible open-source activities span model releases on the Hub, documentation translations, library tooling improvements for frameworks like TensorFlow and Keras, and community hackathons. Participation does not constitute employment with Hugging Face, though the program serves as a platform to scale grassroots community impact across diverse machine learning domains.
Shopify Invests in Research for Ruby at Scale
Shopify says it has invested nearly half a million dollars over the past year in academic research directed toward Ruby and the needs of its community. The work is intended to make programming-language developments account for Ruby’s metaprogramming, loose C extension API, and large-scale workloads with frequent redeployments, rather than assuming long virtual-machine warmup periods. Collaborators include Laurence Tratt, who is investigating hardware meta-tracing and basic-block stitching to generate a just-in-time compiler from the Ruby interpreter; Steve Blackburn, whose group is putting the MMTk garbage-collection toolkit into Ruby; and Stefan Marr, who is studying interpreter speed and startup and warmup time. The program also connects these researchers with Shopify’s Ruby engineers and complements internal work on YJIT and TruffleRuby, with more research and academic papers planned.
Gradio 3.0 is Out!
Gradio 3.0 introduces a ground-up redesign of the Gradio library to improve machine learning demo development and browser performance. The frontend has been rebuilt using modern technologies such as Svelte, resulting in smaller payloads, faster page load times, and cleaner visual designs suitable for embedding in blog posts. Component updates include revamping the Dataframe component to support CSV drag-and-drop, introducing new UI elements like Gallery, and adding the TabbedInterface class to group demos under distinct tabs. Developers also gain access to Gradio Blocks, a low-level Python API enabling custom layouts, multi-step interfaces, dynamic component property modifications, and complex data flows. A competition called the Gradio Blocks Party has been organized through the end of May to encourage community adoption.
Logbook - 2022-05-13
The May 13, 2022 log records Fly.io changes and fixes from the preceding week, spanning infrastructure capacity, account self-service, API maintenance, documentation, and customer-facing behavior. New servers in syd, iad, and dfw, plus three workers in ord, addressed capacity shortages that had caused provisioning or deployment problems. The API moved to Ruby 3.1, with anticipated performance and security-patch benefits, while an internal Elixir UI guide and clearer app and machine name validation errors aimed to reduce development and debugging friction. Operational work permanently disabled consul-templaterb on fly-proxy nodes and fixed a fly-proxy memory leak, giving those nodes more CPU/RAM headroom; an MTU workaround was intended to improve IPv6 tunnel connectivity, while a larger fix remains. Other fixes covered account-deactivation guidance, personalized support email addresses for paid plans, billing, volume metrics, forum registration, and a minor “X daysago” display error.
Maestro: The Orchestration Language Powering Shopify Flow
Shopify developed Maestro, a domain-specific orchestration language, to decouple Shopify Flow’s merchant-facing visual editor from its execution engine, whose requirements differ in declarative usability versus scalable, fault-tolerant execution. The team introduced a horizontally scalable Maestro engine, translated existing Flow workflows into Maestro, migrated them incrementally, and later connected a new visual language directly to Maestro; all workflows had moved to the new editor and engine by early 2022. Maestro coordinates calls to host-language functions rather than implementing service or database operations, while durable checkpoints capture completed work and let the interpreter recover after crashes with at-least-once semantics for primitive calls. During activation, static usage analysis identifies needed returned attributes for possible query optimization, and compilation removes runtime-irrelevant definitions before deployment; checkpoints also support execution monitoring, while planned work includes richer language features, orchestration merging, and workflow preview and testing.
Director of Machine Learning Insights [Part 2: SaaS Edition]
Engineering directors across SaaS organizations share operational experiences regarding the integration, maintenance, and impact of machine learning systems. Deploying production ML introduces significant operational overhead beyond initial modeling, requiring teams to manage data drift, resolve siloed data sources, and adapt to vertical-specific domain shifts. Leaders emphasize that integrating ML effectively often requires prioritizing core business requirements and traditional ML techniques over newly released, complex algorithms. Allowing end users to incorporate domain knowledge and business controls directly into model interactions remains critical for trust and usability. Looking forward, these leaders anticipate substantial ML-driven operational improvements across enterprise workflows, cybersecurity threat detection, and connected healthcare infrastructure.
Student Ambassador Program’s call for applications is open!
Hugging Face launched its inaugural Student Ambassador Program to advance its mission of democratizing machine learning globally. The initiative aligns with the organization's overarching goal to teach machine learning to 5 million people by 2023. Selected student ambassadors will receive dedicated workshops, direct team support, access to a peer network, merchandise, and updates on upcoming projects and features. Applicants must be enrolled in an accredited higher education institution, have completed at least one machine learning or data science course, and already use the Hugging Face Hub or libraries. The application window closed on June 13, 2022, with the program operating from June 30, 2022, through December 31, 2022.
Our Experience Porting the YJIT Ruby Compiler to Rust
Shopify’s YJIT team moved the CRuby JIT compiler from C99 to Rust after finding that C’s limited complexity-management tools made the growing project difficult to maintain. The port had to preserve a non-trivial interface with CRuby, including parsing bytecode and manipulating Ruby’s primitive types, while compiling the Rust project into a static library linked through CRuby’s makefile. The team translated functions and structs largely one by one, retaining the original architecture while using Rust features such as pattern matching, macros, Cargo features, and borrow checking. Completed in three months, the port was judged more maintainable and pleasant to work with, although the team identified challenges around C interoperability, unsafe code, tooling, ecosystem maturity, and limited use of idiomatic Rust.
Accelerated Inference with Optimum and Transformers Pipelines
Transformer-based models are moving into production for large-scale workloads, but default implementations remain slow and resource-intensive compared to traditional machine learning algorithms. Hugging Face Optimum addresses this issue by integrating performance optimization tools and accelerated runtimes like ONNX Runtime directly into standard Transformers pipelines. Developers replace standard model classes with Optimum equivalents to convert models to ONNX, run graph optimizations such as operator fusion, and apply quantization. In an evaluation on an AWS EC2 m5.xlarge CPU instance, optimizing and quantizing a RoBERTa question-answering model cut average latency from 117.61 ms to 64.94 ms while retaining 99.61 percent accuracy. Known limitations include a 2GB remote model size limit, lack of seq2seq support, omitted past key values in causal language models, and no local caching for optimized ONNX models.
I'm All-In on Server-Side SQLite
Ben Johnson argues that SQLite should be considered as the core data and persistence layer for many production full-stack applications, rather than being relegated to unit tests in an n-tier architecture. He attributes this case to cheaper, faster hardware and the fact that most successful applications will never approach terabyte-scale data, making distributed database infrastructure excessive for some workloads. Litestream works alongside standard SQLite by taking control of WAL-mode journaling, capturing WAL updates, replicating them to storage such as S3, and handling checkpoints itself. This preserves ordinary SQLite libraries and transactions while providing a path to restore copies and improve resilience to storage failure; upcoming direct database-to-database replication is also mentioned. The resulting model places data in the application process, where the post reports 10–20 microsecond per-query latency, but it remains aimed at single-node applications and has restore, deployment, and replication-guarantee limitations.
We Raised $100 Million for Open & Collaborative Machine Learning 🚀
Hugging Face announced a 100 million dollar Series C funding round led by Lux Capital, alongside participants including Sequoia, Coatue, and several angel investors. The platform hosts over 100,000 pre-trained models and 10,000 datasets spanning natural language processing, computer vision, speech, and other scientific domains for more than 10,000 customer organizations. The company plans to use the capital to expand research, open-source initiatives, tooling, and efforts focused on mitigating machine learning limitations like biases, privacy risks, and energy consumption. In addition, the organization is directing the BigScience research workshop with over 1,000 participants to train a massive open-source multilingual language model. Following organizational growth from 30 to more than 120 employees in twelve months, the remote-friendly company is actively hiring across all positions and seniority levels.
Talk Nerdy to Me: 10 Tech Podcast Recommendations from Shopify Engineering
Shopify Engineering curated ten podcast recommendations from conversations with its technology employees, extending an earlier roundup of book recommendations. The list spans software engineering, front-end development, JavaScript, career growth, cybersecurity, data science, and broader technology topics, with selections ranging from quick, technical episodes to witty, conversational shows. Several entries emphasize specific formats or practices: Software Engineering Daily conducts researched interviews and focuses on open source, Soft Skills Engineering answers two listener-submitted questions in 30 minutes, and Darknet Diaries covers hacking, cybercrime, and data breaches. The recommendations also include podcasts valued for approachable explanations, collaborative software work, diverse guests, and perspectives on leadership and professional development, while some contributors identify particular episodes or hosts as reasons they keep listening.
Welcome fastai to the Hugging Face Hub
Hugging Face Hub has integrated with the fastai deep learning library, allowing practitioners to upload and download models directly through the huggingface_hub Python client. Fastai Learner objects, which bundle model architectures, data loaders, and loss functions, can be published to a designated repository namespace using push_to_hub_fastai. Uploaded models automatically receive Git-based version control, Git LFS support for large files, and generated model cards. Practitioners can retrieve and instantiate published models locally for inference or transfer learning via from_pretrained_fastai. The integration also supports Blurr, an ecosystem library that combines fastai with Hugging Face Transformers for sequence classification and related text modeling tasks.
Logbook - 2022-05-05
Fly.io’s 2022-05-05 Logbook reports a broad set of infrastructure, product, security, documentation, and bug-fix updates intended to run customer apps faster and more reliably in VMs close to users worldwide. Remote builders now use regional Docker Hub mirrors, clean unused RUN --mount=type=cache build caches, and perform cleanup at startup and shutdown; the new behavior is default for new builders, while existing builders must be destroyed to receive it. WireGuard gateway improvements address load issues, add regional capacity, and support future redundancy, while another IAD gateway is intended to improve connectivity for customer 6pn networks and CI, especially GitHub Actions. The update also adds API security logging for SOC2, re-enables OpenTelemetry, speeds the proxy build pipeline, changes default persistent volume size from 10GB to 3GB, and ships a dashboard, payment retry feature, VSCode fly.toml validation and documentation changes, alongside GraphQL performance work and fixes for monitoring, invoices, encryption, and Homebrew updates.
Embracing a Docs-as-Code approach
Engineers at Grab faced difficulties finding and writing technical documentation because materials were fragmented across various platforms and disconnected from daily workflows. To establish a single source of truth, the organisation adopted a Docs-as-Code approach starting in 2021. Under this system, documentation is written in plaintext Markdown within dedicated folders inside each service's Git repository. An internal developer portal automatically builds and publishes documentation whenever updates are committed, integrating documentation maintenance into the standard code development and review workflow. By April 2022, approximately 80% of Grab services were projected to host their documentation on the internal portal, with the model also expanding to technical training content.
An Introduction to Deep Reinforcement Learning
This introductory chapter presents deep reinforcement learning as a machine-learning approach in which an agent learns by interacting with an environment, taking actions, and receiving positive or negative rewards. It establishes the reinforcement-learning framework through the loop of state, action, reward, and next state, and explains the reward hypothesis, expected return, Markov property, observations, states, and action spaces. The chapter then introduces policies as the agent’s decision-making mechanism and contrasts policy-based methods with value-based methods, including the use of neural networks to estimate policies or state values in deep RL. It concludes by preparing learners to train a lander agent, share it on the Hugging Face Hub, and continue with Q-Learning and value-based methods.
Accelerate Large Model Training using PyTorch Fully Sharded Data Parallel
Increasing parameter counts in modern machine learning models make loading and training them prohibitive on standard hardware. While Distributed Data Parallel replicates the entire model across GPUs, PyTorch Fully Sharded Data Parallel shards optimizer states, gradients, and parameters across data-parallel workers. Benchmarks on causal language modeling using GPT-2 Large and XL across Titan RTX GPUs demonstrate that FSDP increases allowable batch sizes and prevents out-of-memory errors encountered in standard DDP. Integrating FSDP through Hugging Face Accelerate enables these capabilities alongside CPU offloading to run models exceeding GPU memory limits. However, Accelerate requires preparing models before creating optimizers when using multiple models and notes that mixed precision support with FSDP currently has transformer compatibility issues.
React Native Skia—For Us, For You, and For Fun
React Native Skia brings Skia’s cross-platform 2D drawing primitives to React Native, with the stated goals of improving its limited graphical capabilities and aligning designers and developers around capabilities comparable to Figma. The integration uses React Native’s JavaScript Interface (JSI), enabling direct JavaScript-to-native communication through C++ rather than asynchronous messages, and exposes an API described as virtually 100% compatible with Flutter’s. On top of the imperative API, a declarative layer uses the React Reconciler to diff internal representation states and send changes to the Skia engine, supporting composable drawings and declarative optimizations. The library includes image filters, shaders, SVG, path operations, vertices, text layouts, mesh gradients, shadows, backdrop blur, and animations, with a direct connector to Reanimated 2; the authors expect it to address a major React Native pain point and describe the work as just beginning.
Top topics
- Architecture830
- Performance812
- LLMs648
- Machine Learning617
- Open Source561
- AI Agents542
- AI454
- Developer Experience446
Weekly digest
The best engineering reads, once a week.