---
title: "Granite 4.1 LLMs: How They’re Built"
description: "Granite 4.1 is a family of dense decoder-only language models in 3B, 8B, and 30B sizes, trained on approximately 15T tokens. Its five-phase pre-training pipeline moves from broad web-scale data to progressively curated mixtures, increases math and code emphasis, and extends context from 4K to 512K tokens through staged long-context training. Supervised fine-tuning uses about 4.1M curated samples evaluated with LLM-as-Judge and rule-based checks, while reinforcement learning uses on-policy GRPO with DAPO loss. The 8B instruct model matches or surpasses the earlier Granite 4.0-H-Small 32B-A9B MoE model despite its simpler dense architecture and fewer parameters, and the Granite 4.1 models are released under Apache 2.0."
---

# Granite 4.1 LLMs: How They’re Built

[Hugging Face](https://yomu.fyi/company/hugging-face) · Yousaf Shah · Apr 29, 2026

**Type:** Explainer

## Summary

Granite 4.1 is a family of dense decoder-only language models in 3B, 8B, and 30B sizes, trained on approximately 15T tokens. Its five-phase pre-training pipeline moves from broad web-scale data to progressively curated mixtures, increases math and code emphasis, and extends context from 4K to 512K tokens through staged long-context training. Supervised fine-tuning uses about 4.1M curated samples evaluated with LLM-as-Judge and rule-based checks, while reinforcement learning uses on-policy GRPO with DAPO loss. The 8B instruct model matches or surpasses the earlier Granite 4.0-H-Small 32B-A9B MoE model despite its simpler dense architecture and fewer parameters, and the Granite 4.1 models are released under Apache 2.0.

## Context

The work addresses the challenge of building high-quality small language models, emphasizing rigorous data curation rather than relying only on increased compute or model scale. The stated goals include strengthening math, coding, instruction following, general chat, tool use, and long-context performance.

## Approach / What changed

Granite 4.1 uses dense decoder-only transformers with GQA, RoPE, SwiGLU, RMSNorm, and shared input/output embeddings. Training combines five pre-training phases over approximately 15T tokens, staged long-context extension to 512K tokens, supervised fine-tuning on curated data assessed by LLM-as-Judge and rule-based filters, and multi-stage reinforcement learning using on-policy GRPO with DAPO loss.

## Takeaways

- The five pre-training phases progressively shift from broad web data toward higher-quality math, code, technical, instruction, and reasoning mixtures.
- Long-context extension proceeds through 32K, 128K, and 512K stages; the 512K stage uses 80% books and 20% code repository data for the 8B and 30B models.
- The 8B instruct model reportedly matches or surpasses Granite 4.0-H-Small, a 32B-A9B MoE model, while using a dense architecture with fewer parameters.

**Tags:** [Data Pipelines](https://yomu.fyi/topic/data-pipelines), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance)

- Source: [Hugging Face](https://huggingface.co/blog/ibm-granite/granite-4-1)
- Source URL: https://huggingface.co/blog/ibm-granite/granite-4-1
- Ingested by Yomu: 2026-08-29T01:48:35.717Z

[Read original post](https://huggingface.co/blog/ibm-granite/granite-4-1)