Personalize your product's text-to-speech voice for any language: Fine-tuning with Kubeflow Trainer on Red Hat OpenShift AI

Summary

Training an English-centered text-to-speech model for Turkish can produce fluent but hard-to-understand pronunciation for native speakers. This tutorial fine-tunes unsloth/orpheus-3b-0.1-pretrained with about 81,000 Turkish text-audio pairs on Red Hat OpenShift AI using a Kubeflow Trainer TrainJob distributed across two nodes with two A100 GPUs each. LoRA freezes the 3-billion-parameter base and trains 97.3M adapter parameters, while preprocessing converts text to Llama-3 tokenizer tokens and audio to three-layer SNAC codes, masking text labels during scoring. The merged model is evaluated against the base on ten Turkish sentences using Whisper transcription and WER/CER metrics. Mean WER falls from 2.44 to 0.35 and CER from 1.59 to 0.09, though the authors describe the small test as directional.

Takeaways

1. 1

   The TrainJob exposes the base model, dataset, epoch count, and LoRA rank as environment variables, so changing the language or training length requires configuration edits rather than training-code changes.

2. 2

   The tokenizer must match the model; a mismatch can leave training running and the loss falling while the resulting audio remains unusable.

3. 3

   Evaluation loss is calculated on a 1,000-sample held-out split, while WER and CER come from a separate ten-sentence Turkish dictation test covering registers such as directions, news, weather, and flight announcements.