AI GPU Clusters, From Your Laptop, With Livebook

Summary

Livebook, FLAME, and the Nx stack combine Elixir notebooks, elastic execution, and GPU-backed machine learning into a workflow that can move from a laptop to remote infrastructure. Livebook can start runtimes on Fly Machines, connect to existing Erlang/Elixir applications, and synchronize notebook dependencies and modules across nodes while preserving access to private resources within a Fly organization. FLAME wraps arbitrary code in Flame.call, manages executor pools, controls node limits and concurrency, streams results, and shuts down idle nodes or the cluster after disconnection. In the demonstrated AI workflow, video stills are processed on GPU Fly Machines with Llama, then Mistral summarizes the returned descriptions. The same setup also drives a 64-GPU hyperparameter run for BERT, with each L40s node preparing dependencies, loading data, fine-tuning, and streaming chart results to Livebook.

Takeaways

1. 1

   Livebook can connect a laptop notebook to Fly Machines and existing Erlang/Elixir applications, including services on private networks within a Fly organization.

2. 2

   FLAME synchronizes notebook dependencies and code across nodes, lets users set minimum and maximum node counts and concurrency, and automatically replaces completed work until processing finishes.

3. 3

   A demonstrated BERT hyperparameter run used 64 Fly Machines with L40s GPUs, streaming each node’s fine-tuning results to charts in Livebook.