---
title: "Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps"
description: "This guide presents a low-cost recipe for improving structured-output compliance in Liquid AI’s LFM2.5-350M, a task where parseable, schema-adherent responses determine whether an LLM can feed downstream systems. It fine-tunes the model with Group Relative Policy Optimization (GRPO) through TRL, using about 500 samples, 100 steps, three weighted rewards for format, field count, and JSON Schema validation, and a LoRA adapter training roughly 6M parameters. Prompt augmentation covers fenced-code-block instructions and top-level-array tasks, while evaluation uses IFStruct with llama.cpp on an OpenAI-compatible local server. On 2,000 IFStruct samples, the BF16 base model passes 22.6%, rising to 29.7% after tuning. JSON improves from 18.0% to 31.9%, bare-list compliance from 16.6% to 29.7%, while YAML changes little."
---

# Fine-tuning a 350M Model for Better Structured Outputs in 100 GRPO Steps

[Hugging Face](https://yomu.fyi/company/hugging-face) · Leonie Monigatti, ben burtenshaw, Sergio Paniego · Sep 3, 2026

**Type:** Tutorial

## Summary

This guide presents a low-cost recipe for improving structured-output compliance in Liquid AI’s LFM2.5-350M, a task where parseable, schema-adherent responses determine whether an LLM can feed downstream systems. It fine-tunes the model with Group Relative Policy Optimization (GRPO) through TRL, using about 500 samples, 100 steps, three weighted rewards for format, field count, and JSON Schema validation, and a LoRA adapter training roughly 6M parameters. Prompt augmentation covers fenced-code-block instructions and top-level-array tasks, while evaluation uses IFStruct with llama.cpp on an OpenAI-compatible local server. On 2,000 IFStruct samples, the BF16 base model passes 22.6%, rising to 29.7% after tuning. JSON improves from 18.0% to 31.9%, bare-list compliance from 16.6% to 29.7%, while YAML changes little.

## Context

Structured-output compliance is a common real-world LLM requirement, but the source states that many benchmarks combine it with broader reasoning or extraction scores. Valid, parseable output in the requested format and shape determines whether a model can be connected to a downstream system.

## Approach / What changed

The guide fine-tunes LFM2.5-350M with GRPO using TRL, approximately 500 structured-output samples, LoRA targeting LFM-specific modules, and rewards for format, field count, and JSON Schema validation. It augments prompts for fenced code blocks and top-level arrays, then compares the base and merged models on IFStruct using llama.cpp and an OpenAI-compatible local server.

## Takeaways

- The three rewards use weights \[1.0, 0.5, 2.0\]: format correctness, expected top-level field count, and JSON Schema validation, respectively.
- Training runs for 100 steps with eight generations per prompt group, a per-device batch size of four, gradient accumulation of eight, and a maximum completion length of 1,024.
- Average latency increased from 1,453 ms for the base model to 1,518 ms for the GRPO-tuned model in the reported IFStruct evaluations.

**Tags:** [LLMs](https://yomu.fyi/topic/llm), [LoRA](https://yomu.fyi/topic/lora), [Model Evaluation](https://yomu.fyi/topic/model-evaluation), [Reinforcement Learning](https://yomu.fyi/topic/reinforcement-learning)

- Source: [Hugging Face](https://huggingface.co/blog/grpo-with-trl-ifstruct)
- Source URL: https://huggingface.co/blog/grpo-with-trl-ifstruct
- Ingested by Yomu: 2026-09-03T12:00:52.962Z

[Read original post](https://huggingface.co/blog/grpo-with-trl-ifstruct)