---
title: "Granite 4.2 LLMs: How They're Built"
description: "Granite 4.2 is IBM’s first family of dense, decoder-only reasoning language models, offered in 3B, 8B, and 30B sizes. The models use a shared design built around Grouped Query Attention, Rotary Position Embedding, SwiGLU, RMSNorm, bfloat16 precision, and separate input/output embeddings; each has a 131,072-token sequence length. Pre-training starts from scratch on approximately 15 trillion tokens across five phases, with the final phase extending the context window to 512K tokens. Supervised fine-tuning uses about 7.2 million samples spanning agentic and non-agentic data, followed by multi-stage reinforcement learning; the 8B and 30B models also receive agentic RL in sandboxed environments for tool use, code execution, terminal control, and web search. All models support thinking, non-thinking, and low-effort thinking modes, native tool calling, and Apache 2.0 licensing."
---

# Granite 4.2 LLMs: How They're Built

[Hugging Face](https://yomu.fyi/company/hugging-face) · Yousaf Shah, Swanand Kadhe, Riddhiman Moulick, Ashish Sunil Agrawal · Aug 25, 2026

**Type:** Explainer

## Summary

Granite 4.2 is IBM’s first family of dense, decoder-only reasoning language models, offered in 3B, 8B, and 30B sizes. The models use a shared design built around Grouped Query Attention, Rotary Position Embedding, SwiGLU, RMSNorm, bfloat16 precision, and separate input/output embeddings; each has a 131,072-token sequence length. Pre-training starts from scratch on approximately 15 trillion tokens across five phases, with the final phase extending the context window to 512K tokens. Supervised fine-tuning uses about 7.2 million samples spanning agentic and non-agentic data, followed by multi-stage reinforcement learning; the 8B and 30B models also receive agentic RL in sandboxed environments for tool use, code execution, terminal control, and web search. All models support thinking, non-thinking, and low-effort thinking modes, native tool calling, and Apache 2.0 licensing.

## Context

Earlier Granite releases were instruction-following assistants; Granite 4.2 adds explicit reasoning, allowing models to produce a chain of thought and choose between thinking, non-thinking, and low-effort thinking modes depending on the task.

## Approach / What changed

The models are trained from scratch on approximately 15 trillion tokens using five-phase pre-training, then supervised fine-tuned on agentic and non-agentic data and post-trained through a multi-stage, multi-environment reinforcement-learning pipeline. The 8B and 30B models additionally undergo agentic RL in sandboxed environments, while quality control normalizes, evaluates, filters, and deduplicates SFT data.

## Takeaways

- The three models use dense decoder-only transformers with Grouped Query Attention, 40 or 32 attention heads, 8 KV heads, RoPE with θ = 10,000,000, SwiGLU, RMSNorm, and bfloat16 precision.
- The SFT mixture contains approximately 7.2 million samples, with 31.6% agentic and 68.4% non-agentic data; quality checks include LLM judging, heuristic filtering, and SHA-256-based local and global deduplication.
- Native tool calling follows the OpenAI function-calling format through compatible endpoints, enabling integration with agentic harnesses; all models are released under the Apache 2.0 license.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source)

- Source: [Hugging Face](https://huggingface.co/blog/ibm-granite/granite-4-2)
- Source URL: https://huggingface.co/blog/ibm-granite/granite-4-2
- Ingested by Yomu: 2026-08-29T05:42:06.451Z

[Read original post](https://huggingface.co/blog/ibm-granite/granite-4-2)