GRPO fine-tuning on Red Hat OpenShift AI: Reinforcement learning from verifiable rewards with Training Hub

Summary

Supervised fine-tuning for structured reasoning tasks like tool calling often demands expensive, labor-intensive dataset curation. Group Relative Policy Optimization (GRPO) resolves this by using programmatic reward functions to score multiple candidate responses without requiring human labels or a dedicated reward model. On Red Hat OpenShift AI, developers can implement GRPO using Training Hub, Kubeflow Trainer, and the Kubeflow SDK to fine-tune models like Qwen3-4B. The setup utilizes vLLM for candidate generation alongside LoRA weight updates, sharing storage via ReadWriteMany persistent volume claims. In a practical tool-calling test, five training iterations raised task accuracy from 33% to 67%, teaching the model to emit parseable function invocations rather than fabricated conversational text.

Context

Enterprise language model deployments often require structured reasoning, such as producing valid JSON, executable SQL, or accurate tool calls. Although supervised fine-tuning can improve these capabilities, creating and curating labelled training datasets is slow and expensive.

Approach / What changed

The implementation runs a GRPO fine-tuning job on Red Hat OpenShift AI using Training Hub, Kubeflow Trainer, and the Kubeflow SDK. It fine-tunes Qwen3-4B on the Toucan-1.5M dataset with LoRA, splitting GPU memory between vLLM inference and training. Programmatic reward functions score candidate groups to update adapter weights over shared ReadWriteMany persistent storage.

Takeaways

• GRPO eliminates the need for human labelling and separate reward models by using programmatic functions to score candidate responses against verifiable criteria.
• The OpenShift AI training setup requires ReadWriteMany persistent storage so both the JupyterLab workbench and the training pod can simultaneously access datasets, models, and checkpoints.
• Fine-tuning Qwen3-4B over five GRPO iterations improved tool-calling accuracy from 33% to 67%, shifting model behavior from fabricating conversational text to generating syntactically correct function calls.

Related reading

• Red Hat · Aug 24, 2026

  Run LoRA fine-tuning on Red Hat OpenShift AI with Ray

  Red Hat OpenShift AI 3.5 integrates Ray cluster runtime images with preinstalled Training Hub packages to support distributed fine-tuning algorithms including LoRA, SFT, OSFT, and GRPO. Using the CodeFlare software development kit within a Jupyter workbench, developers can provision an elastic single-GPU Ray cluster attached to shared ReadWriteMany persistent storage containing model weights and training datasets. The tutorial demonstrates fine-tuning the Qwen2.5-1.5B-Instruct model on the public sql-create-context dataset to produce direct, bare SQL output without explanatory conversational text. After training completes, the workbench loads and merges the resulting LoRA adapter checkpoints with the base model to evaluate output format correctness against sample database schema questions. The RayCluster automatically tears down resources once the job concludes, while the shared pipeline architecture allows switching between supported fine-tuning algorithms by modifying dataset inputs and configuration parameters.

  Fiona Waters

• AWS ·