--- title: "Modal" description: "15 posts about Modal, summarised, each linking to the original." --- # Modal > 15 posts about Modal, summarised, each linking to the original. ## Articles ### [TypeSafe AI scales Jev to trillions of tokens in three days on Modal](https://yomu.fyi/post/typesafe-ai-scales-jev-to-trillions-of-tokens-in-three-days-on-modal.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: Oct 5, 2026 - Type: Announcement TypeSafe AI’s Jev is a System One Model that outputs structured decisions with probabilities rather than generating text, taking state and questions as input and returning typed values. It uses a custom architecture, a parallel sampler for maximum efficiency, and a training method called Reinforcement Learning for Calibrated Decisions (RLCD). Unlike autoregressive LLMs, Jev produces outputs in parallel, enabling low-latency responses suitable for workloads like large-scale automation, fast browser use, and generative user interfaces. Launched on a Thursday, Jev topped OpenRouter’s leaderboard for contexts between 1K and 10K tokens one week later, handling 15.1% of requests on that platform. To meet demand exceeding one trillion tokens in three days, TypeSafe used Modal’s serverless platform for bursty, high-concurrency inference without managing their own infrastructure, gaining control over networking, routing, and autoscaling needed for ultra-low-latency serving. ### [Modal Clusters are generally available](https://yomu.fyi/post/modal-clusters-are-generally-available.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Author: Peyton Walters - Published: Oct 1, 2026 - Type: Announcement Modal announced general availability of Modal Clusters, a serverless multi-node compute primitive for AI workloads that provides up to 6.4 Tbps InfiniBand RDMA per node with PyTorch and NCCL support. A new gang scheduler places N nodes simultaneously by observing the fleet, assigning all pending clusters, and provisioning capacity when needed. RDMA support is fully automated behind a single rdma=True flag, with Modal contributing RDMA support to the gVisor container runtime and upstreaming the changes. Customers like Decagon, 1x, and Runway use Clusters for trillion-parameter fine-tuning, pre-training home robot world models, and multi-node video inference respectively. Clusters are available to all workspaces today through the @modal.clustered decorator, with per-second billing and no reservation required. ### [Runtime Roundup: VM Sandboxes, Multi-node clusters, and more](https://yomu.fyi/post/runtime-roundup-vm-sandboxes-multi-node-clusters-and-more.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: Oct 1, 2026 - Type: Announcement Modal announced several new features at its inaugural Runtime conference. VM Sandboxes provide agents with full Linux computer access, supporting Docker stacks, databases, graphical environments, and kernel experimentation, already used at Linear, Legora, and Snorkel for coding agents and evals. Sandbox Sidecars run alongside main sandboxes with isolation but share a host, allowing agent code in the main sandbox while trusted operations run in a sidecar the agent cannot access, avoiding network round trips in execution-heavy loops. Modal Clusters, now generally available via @modal.clustered, enable multi-node jobs for training or serving trillion-parameter models, powering post-training at Decagon and real-time inference at Runway. Sticky Sessions maintain client state by routing to the same container using a session token, critical for voice model serving, while Endpoint Candidates allow mirroring production traffic to compare quality, latency, throughput, and cost before promoting. ### [Sidecars: A low-latency trust boundary for Sandboxes](https://yomu.fyi/post/sidecars-a-low-latency-trust-boundary-for-sandboxes.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: Oct 1, 2026 - Type: Announcement Modal introduces Sidecars, isolated containers that run alongside a main Sandbox on the same host, establishing a trust boundary between trusted and untrusted code. Sidecars enable communication across trust boundaries at least 3x faster than using separate Sandboxes, reducing latency for operation-heavy agent workloads. The feature addresses the security risk of running an agent’s harness and tool calls together, which creates the “lethal trifecta” of private data, untrusted content, and external communication. Existing solutions either move the harness to a remote service (adding network latency) or rely on egress controls that lack flexibility. Sidecars allow trusted services such as credential injection, proxy traffic, and harness logic to run locally alongside the Sandbox, with independent network policies and dynamic creation via the Modal SDK. ### [Modal is expanding in Europe with our new London office](https://yomu.fyi/post/modal-is-expanding-in-europe-with-our-new-london-office.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: Sep 2, 2026 - Type: Announcement Modal is expanding in Europe with a new London office, its second in the region, representing a larger investment in the area. The company already has nearly 20 engineers in Stockholm who built core parts of Modal, including its custom file system and Sandboxes product. Hugh Killingbeck-Jones joins as VP EMEA to lead go-to-market teams in London, which the company considers the AI capital of Europe. European AI-native companies like Black Forest Labs and Legora, as well as global platforms like DoorDash, run on Modal for tasks such as model evaluation and agent testing. The company is hiring across sales, engineering, and other roles in London and Stockholm, and plans meetups across EMEA. ### [How Botika runs full-stack generative AI on Modal](https://yomu.fyi/post/how-botika-runs-full-stack-generative-ai-on-modal.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: Sep 2, 2026 - Type: Problem & solution Botika builds agentic e-commerce teams that automate visual production for global fashion brands, using proprietary foundation models and a 100-terabyte image dataset. The company runs its entire full-stack AI operation on Modal, with the data pipeline processing 100 terabytes at less than 1% error rate across thousands of containers. Before Modal, the team struggled with Kubernetes fleet management and slow GCP Batch jobs; now a single researcher can launch 50 experiments daily using Claude, and they set up reinforcement learning infrastructure in less than a day. Modal handles autoscaling, cold starts, and multi-node training with RDMA, allowing Botika to double traffic in seconds, and they serve about 15 models on L4s, L40S, A100s, and H100s. The infrastructure is not Modal-specific, enabling the team to run open-source and custom frameworks without modification. ### [Kimi K3 by Moonshot now available on Modal](https://yomu.fyi/post/kimi-k3-by-moonshot-now-available-on-modal.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: Jul 27, 2026 - Type: Announcement Moonshot released Kimi K3, a 2.8 trillion parameter multimodal mixture-of-experts transformer with a 1 million token context window and native vision, now on Modal. Modal runs the model at 460 tokens per second on release day through a partnership with Moonshot and vLLM, with token-based pricing on a Shared API or dedicated Auto Endpoint. K3 activates 16 of 896 experts per token and uses Kimi Delta Attention along with Attention Residuals to achieve roughly 2.5 times the scaling efficiency of its predecessor K2. Moonshot employed quantization-aware training from the SFT stage with MXFP4 weights and MXFP8 activations, and contributed a new prefix caching implementation to vLLM to address challenges from KDA. Modal deployed a custom DFlash speculative decoding drafter tuned to K3’s architecture, yielding 360 percent faster interactivity and 88 percent higher throughput on agentic workloads. ### [Devin Outposts on Modal](https://yomu.fyi/post/devin-outposts-on-modal.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: Jul 21, 2026 - Type: Announcement Devin, an AI software engineer by Cognition, plans, writes, tests, and ships code semi-autonomously. Cognition now extends Devin with Outposts, moving execution into user-controlled environments while reasoning stays in Cognition’s cloud. Modal is a launch partner with modal-devin, an open-source integration running Outpost sessions in Modal Sandboxes with pre-built custom images, fast cold-starts, snapshotting, and cost-efficient bursting. For example, a task that requires a GPU fails immediately in the default sandbox but succeeds on a Modal-backed Outpost that spins up a GPU on demand and scales to zero when done. One command, uvx modal-devin init, sets up secure infrastructure with a scheduler and worker functions, and each team can have its own outpost with separate credentials. ### [Inkling by Thinking Machines now available on Modal](https://yomu.fyi/post/inkling-by-thinking-machines-now-available-on-modal.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: Jul 15, 2026 - Type: Announcement Thinking Machines released Inkling, a general-purpose multimodal model that accepts text, image and audio inputs and generates text outputs. The 975B-parameter mixture-of-experts transformer activates 41B parameters per token, uses a 1M-token context window, and employs local attention: five sliding window layers for every global layer. Modal provided day-zero support with a custom DFlash block-diffusion speculator, achieving 250 tokens per second per user on 8x B200s with 67% faster interactivity than the model’s built-in multi-token prediction path. The DFlash drafter uses all local attention and causal layers, keeping cost flat as block size grows. Inkling is available today as an OpenAI-compatible shared endpoint through Modal Auto Endpoints with token-based pricing, allowing users to try the model interactively. ### [Modal is a computer](https://yomu.fyi/post/modal-is-a-computer.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: Jul 9, 2026 - Type: Explainer Modal is a platform that runs programs of arithmetic and logic operations on information, making it a computer. It operates on hardware from dozens of cloud providers, building a virtualization layer that aggregates, isolates, and multiplexes demand for increased resource efficiency. Programs run in containers that receive inputs from the container runtime, similar to how a typical computer’s ALU fetches inputs from memory. Information in Modal lives in a tiered storage hierarchy, from container memory to object storage, with caches like page cache and ephemeral disk. Input and output pass through the Modal Input/Output Plane for functions and sandboxes, or the Routing Plane for HTTP servers, with direct Tunnels for extreme performance needs. ### [Anthropic integration with Modal brings scalable compute to Claude Science](https://yomu.fyi/post/anthropic-integration-with-modal-brings-scalable-compute-to-claude-sci.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: Jun 30, 2026 - Type: Announcement Anthropic launched Claude Science, an AI workbench where life sciences researchers run computational workloads from conversation with Claude. The workbench addresses the fragmented toolchain of local machines, HPC clusters, and cloud VMs by centralizing execution in one place. Modal integrates with Claude Science as the compute backend, routing GPU and CPU-heavy workloads to Modal sandboxes automatically and fanning out parallel tasks like virtual screening across hundreds of containers to reduce hours to minutes. Researchers can prompt for tasks such as predicting GFP structure, and Claude writes the folding code, runs it on a Modal GPU, and returns the structure with confidence scores inside the conversation, with Modal handling scheduling, shared storage, and reproducible environments. Modal is committing up to $100,000 in compute for the Claude Science Cohort, with 500– 2,000 allocations per project. ### [Scaling reinforcement learning at Applied Compute](https://yomu.fyi/post/scaling-reinforcement-learning-at-applied-compute.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: May 20, 2026 - Type: Problem & solution Applied Compute, founded by former OpenAI Codex and o1 researchers, builds custom AI agents for enterprises through a post-training approach they call Specific Intelligence. Their core training mechanism is Reinforcement Learning, where agents repeatedly attempt tasks in replayable environments and are scored against reward functions to refine model weights. For clients like DoorDash and Cognition, this means training specialized agents for merchant onboarding and bug-catching that improve from continuous use. The company evaluated multiple infrastructure platforms before selecting Modal, which provides ephemeral sandboxes for complex environment simulation, sub-second cold starts to keep GPUs busy, and serverless fan-out for parallel grading. Modal’s primitives allow each phase of the RL loop—rollouts, evals, and inference—to operate with appropriate resources while sharing state efficiently. ### [Introducing Claude Managed Agents with Modal Sandboxes](https://yomu.fyi/post/introducing-claude-managed-agents-with-modal-sandboxes.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: May 19, 2026 - Type: Announcement Anthropic launched Claude Managed Agents, a suite of composable APIs that separate the agent loop from code execution for building cloud-hosted agents at scale. Modal has collaborated with Anthropic to provide a first-class integration allowing all tool calls to run in self-hosted Modal Sandboxes, which offer fast cold-starts, custom images, snapshotting techniques, and cost-efficient burst pricing. These sandboxes also provide network security through short-lived connect tokens and support upwards of 100,000 concurrent sandboxes per customer. DoorDash is exploring Managed Agents as an accelerator for its merchant intelligence system, while Blend uses the integration to enable secure, scalable agent-assisted triage across complex banking environments. The integration gives teams isolation, cost efficiency, and control over observability, failure handling, and scale while maintaining the convenience of Anthropic’s managed agent loop. ### [Building an RL theorem-proving workflow on Modal](https://yomu.fyi/post/building-an-rl-theorem-proving-workflow-on-modal.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: Apr 29, 2026 - Type: Explainer AE Studio built a distributed reinforcement learning pipeline on Modal to train LLMs to prove theorems in Lean. They compared Group Relative Policy Optimization, which groups attempts and applies gradient updates, with Evolution Strategies, which evaluates a population of perturbed models and updates based on success rates. The workflow used Modal’s per-function images to separate a vLLM instance on GPU for proof generation, CPU-based Lean verification, and orchestration, with .map() fanning out runs and Sandboxes isolating verification batches. ES enabled stateless checkpointing via seed histories: each worker loaded base weights from a Volume, replayed past seeds and rewards (about 200 bytes per iteration), and reconstructed the model on GPU without transferring large weight files. Early results on the lean\_workbook dataset using the Kimina-Prover-Preview-Distill-1.5B model showed ES matching or outperforming GRPO in verified proofs per iteration, especially with limited data, though gains varied. ### [Building with Modal and the OpenAI Agents SDK](https://yomu.fyi/post/building-with-modal-and-the-openai-agents-sdk.md) - Company: [Modal](https://yomu.fyi/company/modal.md) - Published: Apr 15, 2026 - Type: Tutorial The post walks through constructing a custom agent harness with the OpenAI Agents SDK and Modal Sandboxes, starting from a minimal coding agent that executes shell commands directly on the host. To improve safety, the agent is moved into isolated Modal Sandboxes, which are secure Linux environments that can be equipped with GPUs for compute-intensive tasks like training models. Memory is added via Sessions to accumulate context across turns, and subagents are introduced to handle focused subtasks with fresh context windows, preventing context bloat in the orchestrator while allowing long-horizon work. An async subagent pool enables parallel execution, with quotas to limit GPU spend, and filesystem snapshots allow checkpointing and deduplication of setup work across subagents, saving time and offloading memory. The harness is applied to the Parameter Golf challenge, where an orchestrator delegates parallel training experiments across different ML frameworks on GPUs, demonstrating autonomous research capabilities with composable skills.