--- title: "Latest reads" description: "The engineering internet, summarised so you can actually read it." --- # Latest reads > The engineering internet, summarised so you can actually read it. ## Articles ### [GPUs on Fly.io are available to everyone!](https://yomu.fyi/post/gpus-on-fly-io-are-available-to-everyone.md) - Company: [Fly.io](https://yomu.fyi/company/fly-io.md) - Published: Feb 12, 2024 - Type: Announcement Fly.io has made GPU instances available to everyone, adding datacenter-grade accelerators to its globally distributed compute platform. Users can select Ampere A100 variants with 40GB or 80GB of memory, or a Lovelace L40s with 48GB, by setting an app or process’s vm.size to the corresponding GPU kind. The post demonstrates an Ollama deployment using an L40s, a 100GB persistent volume mounted at /root/.ollama, and HTTP service settings that allow machines to stop and start automatically with zero minimum running machines. These instances are positioned for large language model inference, text transcription, and image generation; lower-cost A10 GPUs are planned for the next few weeks, while some organizations may need additional verification based on account age and payment history. ### [A Packwerk Retrospective](https://yomu.fyi/post/a-packwerk-retrospective.md) - Company: [Shopify](https://yomu.fyi/company/shopify.md) - Author: 2023-10-18 - Published: Feb 7, 2024 - Type: Explainer Shopify’s retrospective examines Packwerk, a Ruby gem released in September 2020 to enforce package boundaries and help modularize Rails applications. It describes the tool as static analysis that checks constant references against a declared, non-circular dependency graph, with todo files allowing existing violations to be recorded while new ones are blocked. The authors recount how privacy checks shifted attention toward API design, introduced awkward file organization, and were removed in version 3.0. They also explain that Packwerk cannot determine whether package boundaries or violation fixes reflect runtime coupling, so semantic grouping can mislead developers and add harmful indirection. Shopify ultimately treats running code as the strongest progress signal and retains Packwerk mainly to prevent new dependencies at the application’s base layer. ### [Event Driven Machines](https://yomu.fyi/post/event-driven-machines.md) - Company: [Fly.io](https://yomu.fyi/company/fly-io.md) - Published: Feb 5, 2024 - Type: Problem & solution Event Driven Machines presents Lambdo, a pattern for using Fly Machines as short-lived, event-triggered compute rather than relying on HTTP activation. It polls an SQS queue for JSON events, batches the messages and metadata, then creates fast-booting Fly Machines with the events written to /tmp/events.json; the Machine is destroyed after processing finishes. Application code can come from an existing codebase or Lambdo’s Node and PHP runtime images, with handlers or commands processing each event. SQS message attributes select the required image and can specify VM size and command, while the approach supports arbitrary code and even GPU-enabled Machines, giving each event control over its execution environment. ### [Managing dynamic marketplace content at scale: Grab's approach to content moderation](https://yomu.fyi/post/managing-dynamic-marketplace-content-at-scale-grab-s-approach-to-conte.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Poonam Gambhire - Published: Feb 1, 2024 - Type: Explainer Grab manages content moderation across GrabFood and GrabMart, where merchants update over 100,000 item listings daily across eight countries. The platform must adhere to internal rules, local government regulations, and external platform guidelines across these markets. To handle scale and regional nuance, Grab uses an in-house automated system powered by algorithms and machine learning to scan daily listings for violations. Items flagged as requiring subjective judgment or cultural awareness are routed to human moderators for manual review. Additionally, Grab collaborates with Google to maintain compliance with Play Store policies. ### [Delegating tasks to Fly Machines](https://yomu.fyi/post/delegating-tasks-to-fly-machines.md) - Company: [Fly.io](https://yomu.fyi/company/fly-io.md) - Published: Feb 1, 2024 - Type: Tutorial Resource-intensive web tasks such as resizing high-resolution images can require substantially more memory or CPU than ordinary request handling, making an always-powerful web server inefficient. The article presents a framework- and language-agnostic delegation pattern using Fly Machines, the private network, and two process groups that run the same application with different Machine specifications. A Next.js route calls a wrapper that executes the task when FLY\_PROCESS\_GROUP is worker; otherwise, it sends a POST request to the identical endpoint through the worker’s internal hostname, returning the worker’s response. The example uses a lighter web group and a performance-4x worker group, with local development using separate ports and production routing over .internal addresses; the author calls the implementation naïve and suggests queues and worker timeouts to improve resilience and control worker availability. ### [Rethinking Stream Processing: Data Exploration](https://yomu.fyi/post/rethinking-stream-processing-data-exploration.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Shi Kai Ng - Published: Jan 31, 2024 - Type: Problem & solution Online data materialised in Kafka topics at Grab traditionally remained uninspected until ingested into the Data Lake as offline data, introducing latency and making stream processing applications difficult to develop. To enable earlier data exploration, Grab adopted Apache Zeppelin notebooks integrated with an internal data streaming platform. Zeppelin uses a Flink interpreter to convert interactive SQL queries into Flink jobs submitted to a Flink session cluster. The platform dynamically translates Protobuf schemas into SQL Data Definition Language statements to simplify table discovery. Security and compliance risks are addressed by combining mutual TLS authentication with Strimzi's Open Policy Agent to enforce read-only access policies and audit logging on Kafka topics. ### [Macaroons Escalated Quickly](https://yomu.fyi/post/macaroons-escalated-quickly.md) - Company: [Fly.io](https://yomu.fyi/company/fly-io.md) - Published: Jan 31, 2024 - Type: Explainer Fly.io built a Macaroons-based security token system after finding its API tokens too powerful for expressing scoped permissions and roles. The design starts with a user-linked, HMAC-signed bearer token and chains caveats through HMAC, while checking every caveat independently so each added restriction can only weaken authorization. It models organizations, apps, machines, and volumes with first-party caveats, and also includes third-party caveats, service-token transformations, machine locking, nonce-based revocation, and expiration handling across customer workloads. The resulting system lets users create roles without changing the platform and supports SSO, plugins, and traceable inter-service authorization, although the author acknowledges extra complexity and awkward ways to express some roles. ### [Working with ChatGPT Functions on Heroku](https://yomu.fyi/post/working-with-chatgpt-functions-on-heroku.md) - Company: [Heroku](https://yomu.fyi/company/heroku.md) - Author: Julián Duque - Published: Jan 30, 2024 - Type: Tutorial Menu Maker is a Node.js web application that turns user-supplied ingredients into a dish name, fine-dining description, ingredient list, and recipe. Its React front end communicates with an Express back end, which adds prompt instructions and sends the request to OpenAI’s Chat Completions API. The application uses ChatGPT’s function calling feature to define an updateDish function with structured JSON fields for the title, description, ingredients, and recipe, so parsed arguments can update React state directly. The GitHub-hosted Node.js monorepo is deployed to Heroku, where an OPENAI\_API\_KEY config var authenticates requests, an optional CHATGPT\_MODEL selects the model, and Heroku runs yarn build and yarn start. ### [Introducing Notion Calendar – an integrated calendar for work and life](https://yomu.fyi/post/introducing-notion-calendar-an-integrated-calendar-for-work-and-life.md) - Company: [Notion](https://yomu.fyi/company/notion.md) - Author: Raphael Schaad - Published: Jan 17, 2024 - Type: Announcement Notion has renamed Cron as Notion Calendar, introducing a calendar designed to connect work and personal time with the rest of a Notion workspace. It links meetings to Notion documents, project timelines, task due dates, and database items, while integrating with Google Calendar, Zoom, and Google Meet. Focus-oriented features include a computer menu-bar view for schedules and meeting access, plus command-menu controls and keyboard shortcuts; event blocking and auto-blocking combine commitments across calendars and prevent scheduling conflicts, while a built-in scheduling tool shares availability. The product is available now for free, and the announcement says its longer-term vision is to bring a time layer to notes, projects, and tasks across Notion. ### [Horizontally scaling the Rails backend of Shop app with Vitess](https://yomu.fyi/post/horizontally-scaling-the-rails-backend-of-shop-app-with-vitess.md) - Company: [Shopify](https://yomu.fyi/company/shopify.md) - Author: 2023-10-18 - Published: Jan 17, 2024 - Type: Problem & solution Shop app’s hockey-stick growth pushed its Ruby on Rails backend and MySQL primary datastore toward capacity, with multi-terabyte storage, week-long schema migrations, and throttled background jobs. After splitting databases and scaling surrounding systems, the team adopted Vitess to shard and re-shard MySQL while retaining SQL compatibility and self-managed primary datastores. They organized user-owned data into a sharded “users” keyspace using user\_id, kept other data in an unsharded “global” keyspace, and built application-layer query verifiers to validate routing, distribution, and transaction correctness before migration. The move made schema migrations take hours instead of weeks, removed the reported capacity-driven throttling, and enabled further scaling by adding shards, while introducing concepts such as Sequences, Vindexes, and VSchemas that developers must learn. ### [How Yoko Li makes towns, tamagoes, and tools for local AI](https://yomu.fyi/post/how-yoko-li-makes-towns-tamagoes-and-tools-for-local-ai.md) - Company: [Fly.io](https://yomu.fyi/company/fly-io.md) - Published: Jan 8, 2024 - Type: Explainer Yoko Li’s local AI work spans playful agent experiments and practical tools for running language-model workflows on personal hardware. AI Town uses conversations among AI agents to produce changing relationships and emergent behavior, while AI Tamago feeds game state and user inputs to an unmodified LLaMA 2 7B model and uses Ollama’s JSON mode to generate pet actions. The Local AI Starter Kit packages document ingestion, vector storage, and language-model responses into a private, local “chat with a knowledge base” pipeline, avoiding setup across at least four API providers. The post also describes unreliable JSON generation, hallucinations, and grammar-guided workarounds, concluding that randomness is a liability for factual outputs but useful for game behavior and creative experimentation. ### [Deploy Your Own (Not) Midjourney Bot on Fly GPUs](https://yomu.fyi/post/deploy-your-own-not-midjourney-bot-on-fly-gpus.md) - Company: [Fly.io](https://yomu.fyi/company/fly-io.md) - Published: Jan 4, 2024 - Type: Tutorial This tutorial builds a Discord image-generation bot that uses Fooocus, an open-source interface for Stable Diffusion, to provide a Midjourney-like /imagine command. It separates the system into a normal Fly Machine running a Python Pycord bot and a GPU Machine running Fooocus-API, with Flycast connecting them over Fly.io’s private network. The bot defers Discord responses, polls for the generated image each second, and uploads the resulting file, while the post notes that this blocking flow will not perform well for hundreds of simultaneous users. Fly GPU rootfs supports the nearly 12GB image, and an L40S setup costs about $3.20/hr when running, with scale-to-zero intended to avoid charges for unused compute. GPU activity is verified with nvidia-smi, and the source code can be extended with additional commands and image-to-image features. ### [Kafka on Kubernetes: Reloaded for fault tolerance](https://yomu.fyi/post/kafka-on-kubernetes-reloaded-for-fault-tolerance.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Fabrice Harbulot - Published: Dec 26, 2023 - Type: Problem & solution Grab's real-time data streaming platform, Coban, operates Kafka on AWS Elastic Kubernetes Service using Strimzi, allocating an entire EC2 worker node with NVMe instance store volumes to each broker. An initial architecture suffered from client connection errors, broken Network Load Balancer target groups, and zombie Persistent Volume Claims when worker nodes terminated. To achieve automated fault tolerance, the team integrated the AWS Node Termination Handler in Queue Processor mode with Auto Scaling lifecycle hooks, ensuring Kafka receives a SIGTERM to migrate partition leadership gracefully before shutdown. They also introduced the Kubernetes Cluster Autoscaler to dynamically provision replacement nodes during maintenance events and used the AWS Load Balancer Controller with TargetGroupBinding custom resources to dynamically update load balancer targets using IP mode. ### [Fly With Alpine](https://yomu.fyi/post/fly-with-alpine.md) - Company: [Fly.io](https://yomu.fyi/company/fly-io.md) - Published: Dec 21, 2023 - Type: Explainer Alpine Linux can reduce container image sizes compared with Debian slim, but switching distributions involves compatibility and ecosystem trade-offs. The post compares Fly.io’s Debian-based baseline across Bun, Node, Python, and Ruby, then reports smaller Alpine base and application images, including 304MB versus 428MB for a minimal Rails/Postgresql/esbuild application. Migration consists of changing Dockerfile base images from -slim to -alpine, replacing apt-get with apk, and reviewing package-name differences; recent Bun, Node, and Rails generators can make these changes with an --alpine option. Alpine uses musl rather than Debian’s glibc, and issues can involve DNS, missing commands or packages, precompiled binaries, insufficient testing of release candidates, and software such as Chrome. Fly.io will retain Debian as its default while keeping Alpine available. ### [How I Built an Okta Documentation Chatbot in Python](https://yomu.fyi/post/how-i-built-an-okta-documentation-chatbot-in-python.md) - Company: [Okta](https://yomu.fyi/company/okta.md) - Author: Tanish Kumar - Published: Dec 20, 2023 - Type: Tutorial Okta developer documentation contains information developers need for integrations, but the author observed that it was underused. They built Oktanaut, a Python chatbot in Jupyter Notebook, comparing a LlamaIndex-based prototype trained on Okta documentation with a GPT-3.5-powered version using the OpenAI API. The first favored precision for focused questions, while the second handled a broader range; the final demo used OpenAI API for its larger models and user-friendly experience. For both versions, the author curated questions, answers, and support engineer information, stored training data in Google Drive for Colab, and retained historical interactions so replies could use session context. Panel provided the frontend, and the author chose Jupyter Notebook for simple setup despite noting bugs and glitches. ### [Championing CyberSecurity: Grab's bug bounty programme in 2023](https://yomu.fyi/post/championing-cybersecurity-grab-s-bug-bounty-programme-in-2023.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Nathaniel Callens - Published: Dec 19, 2023 - Type: Explainer Grab reviewed the performance and milestones of its security bug bounty programme for 2023, marking eight years of running the initiative. Operating quarterly campaigns through HackerOne alongside dedicated internal teams, the company processed approximately 1,000 vulnerability submissions over the year, including 400 reports during Q1 alone. Cumulative payouts to external security researchers surpassed $700,000 since the programme's 2015 launch. To adapt to an expanding service portfolio, Grab broadened its scope to encompass joint ventures and acquisitions while introducing diversified financial and recognition-based rewards. ### [New Social Caption Generator Uses AI to Help Sellers Post More Easily](https://yomu.fyi/post/new-social-caption-generator-uses-ai-to-help-sellers-post-more-easily.md) - Company: [Ebay](https://yomu.fyi/company/ebay.md) - Author: eBay News Team - Published: Dec 18, 2023 - Type: Announcement For eBay sellers, social media promotion can make listings more visible, but composing effective captions may be difficult. Its new generative AI feature adds a “Use AI Caption” button in the Social page of Seller Hub, where it drafts item-focused posts with hashtags and emojis. Sellers can ask for rewrites, manually keep or remove those extras, and publish posts containing listing links and item images. The capability is fully available in the US, UK, Germany, and Australia for sharing individual listings, categories, promotions, and stores, while an announced auto-posting option will later automatically publish new listings when they go live. The company reports a positive response from early users. ### [Introducing Fly Kubernetes](https://yomu.fyi/post/introducing-fly-kubernetes.md) - Company: [Fly.io](https://yomu.fyi/company/fly-io.md) - Published: Dec 18, 2023 - Type: Announcement Fly.io introduces Fly Kubernetes (FKS), an implementation of Kubernetes that runs on its globally distributed container platform for users who want Kubernetes tooling and declarative orchestration. Fly.io says its earlier Nomad-based approach struggled with global scheduling at more than 200,000 applications and with placing workloads in users’ requested regions, so it moved state to individual workers and exposed the Fly Machines API, whose placement requests fail quickly rather than retry elsewhere. FKS combines K3s with Virtual Kubelet, mapping pod lifecycle operations to Fly Machines API calls while running K3s and Virtual Kubelet on one Fly Machine. A generated kubeconfig enables kubectl access, and Pods run on separate Fly Machines as the platform scales the cluster. The design provides declarative replica, autoscaling, and volume definitions plus automatic recovery for failed Machines, while retaining Fly Machines’ trade-off of failing when requested placement is unavailable. ### [Sliding window rate limits in distributed systems](https://yomu.fyi/post/sliding-window-rate-limits-in-distributed-systems.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Naveen Kumar Jakuva Premkumar - Published: Dec 14, 2023 - Type: Problem & solution Marketing communications across Grab's user base risked causing notification overload and consent revocations. To enforce personalized daily and weekly frequency caps across more than 270 million users, the team addressed segment membership storage and communication rate limiting. They adopted roaring bitmaps instead of Bloom filters to compactly store user segment data without hash collisions or costly rebuilds upon deletion. For frequency capping, they chose Amazon ElastiCache for Redis over DynamoDB, executing a sliding log rate limiting algorithm directly on the cluster using Lua scripts and sorted sets. Timestamps are stored as sorted set scores, and historical data is cleaned up via eviction ranges to prevent unbounded memory growth. ### [Fly.io has GPUs now](https://yomu.fyi/post/fly-io-has-gpus-now.md) - Company: [Fly.io](https://yomu.fyi/company/fly-io.md) - Published: Dec 13, 2023 - Type: Announcement Fly.io announces GPU support for its cloud, extending its near-user deployment model to AI and machine-learning workloads at the edge. The offering attaches Nvidia A100 GPUs with 40GB or 80GB of RAM, CUDA support, eight AMD EPYC CPU cores, and volumes up to 500GB; L40s are also listed as coming soon and then noted as available. A sample fly.toml deploys Ollama with a 100GB model volume, while auto-start and auto-stop settings can reduce min\_machines\_running to zero when idle. GPU apps can scale across supported regions, retaining the same public IP address and TLS certificates, with an Amsterdam example using fly scale count 2 --region ams. Listed prices are $2.50 per hour for A100 40GB and $3.50 for A100 80GB, with reserved-instance and dedicated-host discounts available. [Newer posts](https://yomu.fyi/posts/page/189.md) · [Older posts](https://yomu.fyi/posts/page/191.md)