---
title: "BitNet’s CPU Trade: Small Footprint, Sharper Constraints"
description: "BitNet trades floating-point precision for a tiny, CPU-friendly footprint. This diagram maps the payoff—small memory and local serving—against the constraints that decide when the trade is worth it."
---

# BitNet’s CPU Trade: Small Footprint, Sharper Constraints

**Type:** [Trade-offs](https://yomu.fyi/visual-explainers/category/trade-offs.md)

![A three-part trade-off diagram for BitNet showing a CPU and 1.58-bit ternary weights, a deployed service with sleep/wake and checkpoint notes, and benefit, cost, and decision panels.](https://yomufyi.s3.us-east-1.amazonaws.com/visual-explainers/0a14db59-792e-4403-96c1-c38bfdd0d6ad/image-154063db-0987-4290-8a3c-01302e94e383.webp)

> BitNet trades floating-point precision for a tiny, CPU-friendly footprint. This diagram maps the payoff—small memory and local serving—against the constraints that decide when the trade is worth it.

## Explanation

BitNet is a quantized model approach that uses 1.58-bit ternary weights to run inference on CPUs with a small memory footprint. In the supplied example, a 2.4-billion-parameter model fits in 1.1 GB of RAM and reaches about 50 tokens per second on a Sprite, where it can sleep when idle and reload in roughly 1.2 seconds. The trade-off is concrete: you give up a conventional precision format and accept measured CPU throughput in exchange for simpler hosting, no GPU dependency, and checkpoint rollback for experiments. Engineers should use this page to decide whether a CPU-first BitNet deployment fits a local or low-volume workload before committing to the operational model.

## FAQ

### What does BitNet trade away to run on a CPU?

It uses 1.58-bit ternary weights instead of a conventional floating-point format. In the supplied example, that lets a 2.4-billion-parameter model fit in 1.1 GB of RAM and run at about 50 tokens per second on a CPU. The material does not establish a general quality penalty, so evaluate the quantized model on your task.

### When is a CPU-only BitNet deployment the right call?

It fits when a small memory footprint and simpler local serving matter more than peak throughput. The example runs a 2.4-billion-parameter model at roughly 50 tokens per second, so the workload must tolerate that rate. If you need more throughput or broader model support, that is the signal to evaluate a GPU-backed path.

### What operational details matter when serving BitNet on a Sprite?

The model loads in about 1.2 seconds when the Sprite wakes, and the Python server exposes an OpenAI-compatible endpoint on port 8080. Checkpoints capture the whole environment, which makes rollback practical when testing models, kernel parameters, or prompt formats. Plan for idle hibernation and service registration so the endpoint comes back cleanly.

**Tags:** [BitNet](https://yomu.fyi/topic/bitnet.md), [Cost Optimization](https://yomu.fyi/topic/cost-optimization.md), [Inference](https://yomu.fyi/topic/inference.md), [Local AI](https://yomu.fyi/topic/local-ai.md), [Model Serving](https://yomu.fyi/topic/model-serving.md), [Quantization](https://yomu.fyi/topic/quantization.md)

## Related reading

### [Slack Bots on Sprites: OAuth is Easy Now](https://yomu.fyi/post/slack-bots-on-sprites-oauth-is-easy-now.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Nov 6, 2026
- Type: Tutorial

Building a Slack bot with OAuth usually requires setting up a server with a public URL, handling callbacks, storing tokens, and maintaining them across restarts. Sprites simplifies this by providing an internet-accessible environment with a persistent filesystem, eliminating the need for Ngrok or separate hosting. To create the bot, users initialize npm, install @slack/bolt and express, and build an OAuth handler for Slack’s callback that stores tokens to the filesystem. The bot is registered as a Sprite service with sprite-env services create , which enables auto-start on HTTP requests, automatic request routing to port 8080, and persistent registration across hibernations. When idle, the Sprite hibernates to save costs and wakes in under a second, and users can create a checkpoint snapshot for easy restoration.


### [Run Claude Code From Your Phone With Sprites](https://yomu.fyi/post/run-claude-code-from-your-phone-with-sprites.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Nov 6, 2026
- Type: Tutorial

Sprites are on-demand, hardware-isolated Linux computers that start in under a second, persist code, packages, and state between sessions, and let you checkpoint the entire environment at any time. The sprite-mobile tool provides a chat UI for issuing commands to Claude Code in YOLO mode from a phone over Tailscale, enabling unrestricted and aggressive experimentation. Setup requires one command to create a Sprite and a one-line script that configures Claude CLI, GitHub CLI, Fly.io CLI, Sprites CLI, and Tailscale, sets up HTTPS via Tailscale Serve, and starts Tailnet Gate for waking suspended Sprites. After setup, obtain the public URL with sprite url update –auth public -s cheese\_sandwich and open it in a Tailscale-connected phone’s browser. Sprites automatically sleep after 30 seconds of inactivity, wake in under a second, and incur no charges when idle.


### [How to Run Untrusted Code Safely](https://yomu.fyi/post/how-to-run-untrusted-code-safely.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Nov 6, 2026
- Type: Tutorial

Running untrusted code requires stronger isolation than containers offer because containers share the host kernel and remain vulnerable to escapes, a threat classified by NIST SP 800-190 as critical and by MITRE ATT&CK as T1611. Network egress control must be allowlist-based rather than denylist-based, as code without filesystem access can still exfiltrate data or download payloads to attacker servers. Sprites provide hardware-isolated sandboxes with network policies enforced through an external API that code cannot modify, even with root access. They support checkpoint-based rollback, pre-installed runtimes for Python, Node, Ruby, Rust, Go, Java, Elixir, Bun, and Deno, and automatically sleep when idle. The post demonstrates creating a code-runner Sprite, applying a deny-all network policy, executing code via subprocess, and optionally permitting specific domains like api.openai.com while blocking all others.


### [Run a Local LLM on a CPU With BitNet and a Sprite](https://yomu.fyi/post/run-a-local-llm-on-a-cpu-with-bitnet-and-a-sprite.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Nov 6, 2026
- Type: Tutorial

Microsoft’s BitNet uses 1.58-bit quantization and ternary weights to run a 2.4 billion parameter model on a CPU, fitting in 1.1GB of RAM and achieving 50 tokens per second. The author deploys this model on a Sprite, a hardware-isolated compute unit that provides a URL with TLS, sleeps when idle, and wakes on request with a 1.2-second model load time. A Python HTTP server wraps llama-cli as a subprocess, exposing an OpenAI-compatible API at /v1/chat/completions, a status page, and a self-addressed client at /client.py. Checkpoints capture the entire environment atomically, allowing safe rollback when experimenting with different models, kernel parameters, or prompt formats. The Sprite registers as a service on port 8080, making the inference endpoint available to any other Sprite in the account without infrastructure changes.


### [I Build Everything In Sprites Now](https://yomu.fyi/post/i-build-everything-in-sprites-now.md)
- Company: [Fly.io](https://yomu.fyi/company/fly-io.md)
- Published: Nov 6, 2026
- Type: Problem & solution

The author recounts the chaos of local development across multiple projects, where version managers (asdf, nvm), separate databases (Postgres, MySQL), and conflicting environment variables cause constant friction. To solve this, they use Sprites, a Fly.io product that provides each project its own persistent, hardware-isolated Linux computer with 100 GB storage, incurring cost only when active. Each Sprite has dedicated versions, services, env vars, and ports, waking from sleep in seconds and avoiding all collisions. Sprites expose real public URLs with automatic TLS, eliminating the need for ngrok or tunneling for OAuth callbacks and webhooks. The author also values checkpoint snapshots for safe experimentation and the ability to maintain specialist coding agents per project, each with its own claude.md and skills, without context pollution.
