---
title: "Deploy NeMo Guardrails on Red Hat OpenShift AI"
description: "This article shows how to deploy an evaluated NeMo Guardrails configuration on a Red Hat OpenShift AI cluster, continuing a series on local development and dataset-based evaluation. It requires OpenShift AI 3.5 or later and namespace write access, while OpenShift AI 3.2 or newer can be configured to use the 3.5 NeMo Guardrails image through TrustyAI service-operator changes. Deployment uses a ConfigMap for the configuration, a NemoGuardrails custom resource, and an authentication annotation that protects the server route; an empty rails.co file is sufficient when no custom algorithms are defined. The deployed server accepts v1/guardrail/checks requests with a bearer token, and the demonstrated classifier permits a benign prompt, blocks a prompt injection, and rejects an invalid token. The article notes that LLM proxying and LLM-based guardrailing are outside its scope."
---

# Deploy NeMo Guardrails on Red Hat OpenShift AI

[Red Hat](https://yomu.fyi/company/red-hat) · Rob Geada · Sep 9, 2026

**Type:** Tutorial

## Summary

This article shows how to deploy an evaluated NeMo Guardrails configuration on a Red Hat OpenShift AI cluster, continuing a series on local development and dataset-based evaluation. It requires OpenShift AI 3.5 or later and namespace write access, while OpenShift AI 3.2 or newer can be configured to use the 3.5 NeMo Guardrails image through TrustyAI service-operator changes. Deployment uses a ConfigMap for the configuration, a NemoGuardrails custom resource, and an authentication annotation that protects the server route; an empty rails.co file is sufficient when no custom algorithms are defined. The deployed server accepts v1/guardrail/checks requests with a bearer token, and the demonstrated classifier permits a benign prompt, blocks a prompt injection, and rejects an invalid token. The article notes that LLM proxying and LLM-based guardrailing are outside its scope.

## Context

The work takes an evaluated NeMo Guardrails configuration from local development and makes it available on a Red Hat OpenShift AI cluster. The deployment requires OpenShift AI 3.5 or later and write access to a namespace; OpenShift AI 3.2 or newer can use the 3.5 NeMo Guardrails image after TrustyAI service-operator configuration changes.

## Approach / What changed

The configuration is uploaded to an OpenShift ConfigMap, then referenced by a NemoGuardrails custom resource managed by the TrustyAI operator. Authentication is enabled on the server route, with namespace view access or a service-account token controlling access. The deployed endpoint is tested with benign and prompt-injection requests, plus an invalid token.

## Takeaways

- An empty rails.co file satisfies NeMo Guardrails' configuration expectation when the configuration defines no custom guardrail algorithms.
- The demonstrated prompt-injection classifier returned success for a benign prompt and blocked a request asking how to break out of instructions.
- The enable-auth annotation protects the deployed route; a generated service-account token can provide shareable, revocable access and expires after 168 hours by default.

**Tags:** [Authentication](https://yomu.fyi/topic/authentication), [NeMo Guardrails](https://yomu.fyi/topic/nemo-guardrails), [OpenShift AI](https://yomu.fyi/topic/openshift-ai)

- Source: [Red Hat](https://developers.redhat.com/articles/2026/09/09/deploy-nemo-guardrails-on-red-hat-openshift-ai)
- Source URL: https://developers.redhat.com/articles/2026/09/09/deploy-nemo-guardrails-on-red-hat-openshift-ai
- Ingested by Yomu: 2026-09-09T20:05:04.454Z

[Read original post](https://developers.redhat.com/articles/2026/09/09/deploy-nemo-guardrails-on-red-hat-openshift-ai)