# Run LoRA fine-tuning on Red Hat OpenShift AI with Ray

[Red Hat](https://yomu.fyi/company/red-hat) · Fiona Waters · Aug 24, 2026

**Type:** Tutorial

## Summary

Red Hat OpenShift AI 3.5 integrates Ray cluster runtime images with preinstalled Training Hub packages to support distributed fine-tuning algorithms including LoRA, SFT, OSFT, and GRPO. Using the CodeFlare software development kit within a Jupyter workbench, developers can provision an elastic single-GPU Ray cluster attached to shared ReadWriteMany persistent storage containing model weights and training datasets. The tutorial demonstrates fine-tuning the Qwen2.5-1.5B-Instruct model on the public sql-create-context dataset to produce direct, bare SQL output without explanatory conversational text. After training completes, the workbench loads and merges the resulting LoRA adapter checkpoints with the base model to evaluate output format correctness against sample database schema questions. The RayCluster automatically tears down resources once the job concludes, while the shared pipeline architecture allows switching between supported fine-tuning algorithms by modifying dataset inputs and configuration parameters.

## Context

Distributed fine-tuning of large language models across enterprise infrastructure often requires complex dependency management, manual pip installations, and custom cluster provisioning across multi-stage machine learning pipelines.

## Approach / What changed

Red Hat OpenShift AI 3.5 incorporates Ray CUDA runtime images with preinstalled Training Hub libraries. Using the CodeFlare SDK from a Jupyter workbench connected to ReadWriteMany shared persistent storage, a Ray cluster is dynamically configured and launched to run a parameter-efficient LoRA fine-tuning job on Qwen2.5-1.5B-Instruct using the sql-create-context dataset.

## Takeaways

- Training Hub comes preinstalled in Red Hat OpenShift AI 3.5 Ray CUDA runtime images, enabling fine-tuning without pip installations or external dependencies in air-gapped environments.
- Shared persistent volume claims require ReadWriteMany access mode so that the workbench control plane and Ray cluster pods can concurrently access model weights, datasets, and checkpoints.
- LoRA fine-tuning on the sql-create-context dataset trains Qwen2.5-1.5B-Instruct to output direct, executable SQL queries rather than wrapping answers in conversational preamble or Markdown formatting.

**Tags:** [Kubernetes](https://yomu.fyi/topic/kubernetes), [LLMs](https://yomu.fyi/topic/llm), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Open Source](https://yomu.fyi/topic/open-source), [Python](https://yomu.fyi/topic/python)

[Read original post](https://developers.redhat.com/articles/2026/08/24/lora-fine-tuning-red-hat-openshift-ai-ray)