OpenShift AI

Red Hat · Sep 1, 2026

Orchestrate production RAG with OpenShift AI

This guide shows how to move a streaming retrieval-augmented generation (RAG) workflow from a monolithic Ray Data script to modular AI pipelines on OpenShift AI. It addresses failure recovery and run traceability by separating parsing and chunking, ingestion, model download, and deployment into five reusable components. Parsed and chunked JSONL is stored in S3-compatible storage so KFP can rerun ingestion without repeating document processing, and each run records parameters, timestamps, logs, and status. The data chain uses Docling’s HybridChunker and Ray map_batches with ActorPoolStrategy, while the model chain downloads a Hugging Face model and deploys it through vLLM and KServe in parallel. For embedding ingestion, the workflow supports local CPU processing with granite-embedding-125m-english or a dedicated GPU vLLM service for larger collections and offers notebooks for deployment and query validation.