Granite 4.0 Nano: Just how small can you go?

Summary

IBM has released Granite 4.0 Nano, a collection of sub-billion and ~1.5B parameter language models tailored for edge and on-device deployments. Trained on more than 15 trillion tokens under an Apache 2.0 license, the suite includes four instruct models alongside their base counterparts. The lineup introduces hybrid-SSM architectures—specifically the ~1.5B parameter Granite 4.0 H 1B and the ~350M parameter Granite 4.0 H 350M—as well as standard transformer variants suited for runtimes like llama.cpp. According to benchmark results across general knowledge, math, code, and safety domains, these compact models demonstrated strong capability improvements relative to their parameter footprint. Furthermore, evaluation on IFEval and Berkeley's Function Calling Leaderboard v3 indicated competitive performance on instruction following and tool calling tasks.

Context

Developing powerful, useful language models for edge and on-device applications without requiring hundreds of billions of parameters, while accommodating environments where hybrid architectures lack optimized runtime support.

Approach / What changed

IBM developed Granite 4.0 Nano under an Apache 2.0 license, trained on over 15T tokens with ISO 42001 certification. The release includes four instruct models and their base counterparts: hybrid-SSM dense models (Granite 4.0 H 1B at ~1.5B parameters and Granite 4.0 H 350M at ~350M parameters) alongside traditional transformer models (Granite 4.0 1B and Granite 4.0 350M) compatible with runtimes such as vLLM, llama.cpp, and MLX.

Takeaways

• Granite 4.0 Nano provides both hybrid-SSM architectures (H 1B and H 350M) and standard transformer variants (1B and 350M) for platforms lacking hybrid runtime support.
• The models were trained on over 15 trillion tokens under an Apache 2.0 license and carry ISO 42001 certification for responsible model governance.
• In evaluations using IFEval and Berkeley's Function Calling Leaderboard v3, Granite Nano outperformed several similarly sized models on instruction following and tool calling.

Related reading

• Amazon · May 15, 2026

  Making LLMs faster without sacrificing accuracy

  Standard scaling laws optimize parameter count and training data budgets to minimize loss but ignore internal Transformer architecture configurations. To address throughput disparities among equally sized models, researchers developed a scaling framework incorporating hidden size, the multilayer perceptron to attention parameter ratio, and grouped-query attention. The method calibrates a correction surface over standard Chinchilla scaling laws to independently optimize separable architectural parameters for target accuracy. Evaluated across more than 200 models up to three billion parameters, the framework produced the Panda model family for maximum accuracy and the Surefire family for Pareto efficiency. Surefire models matched or exceeded LLaMA-3.2 accuracy while increasing inference throughput by 12% to 47% across vLLM and SGLang serving systems on modern GPUs.

  Tao Yu, Youngsuk Park

• Hugging Face · Sep 4, 2025

  Welcome EmbeddingGemma, Google's new efficient embedding model