Red HatTarun EtikalaTutorial
LangGraph agents can call tools and follow prompts but lack safety guardrails, risking unsafe responses like explaining fraud. NeMo Guardrails acts as a transparent proxy between the agent and LLM, inspecting every request and response without requiring agent code changes. On Red Hat OpenShift AI, the guardrails service is deployed via a NemoGuardrails custom resource managed by the TrustyAI Operator, and the agent’s BASE_URL points to this service instead of the model endpoint. The example demonstrates three rail types: regex filtering for jailbreak patterns, content safety classification against S1-S13 categories, and topic safety to keep the agent within a banking domain, with optional tracing using MLflow and OpenTelemetry. Rails execute sequentially, with the first failure short-circuiting the chain, and the separation lets safety policy be owned by guardrails while the agent handles reasoning and tools.
Adding NeMo Guardrails as a proxy between a LangGraph agent and LLM on Red Hat OpenShift AI to enforce safety policies without modifying agent code.
Deploy NeMo Guardrails as a proxy service using the TrustyAI Operator’s NemoGuardrails custom resource on OpenShift AI. Point the LangGraph agent’s BASE_URL at the guardrails service. Configure three rail types (regex, content safety, topic safety) in config.yaml and prompts.yml. Optionally enable tracing with MLflow and OpenTelemetry.
NeMo Guardrails uses a proxy pattern that sits between the agent and LLM, inspecting requests and responses without requiring changes to the agent’s code.
Rails execute sequentially; the first failure short-circuits the chain, so a blocked input rail prevents further processing and returns a configured refusal.
On OpenShift AI, the guardrails service is deployed via a NemoGuardrails custom resource managed by the TrustyAI Operator, and the agent’s BASE_URL is pointed at the guardrails service instead of the model endpoint.
NeMo Guardrails sits between the agent and LLM, inspecting traffic in both directions. It passes system prompts and tool calls through unchanged but checks user messages and LLM responses against configured rails. Allowed requests reach the inference endpoint; blocked requests receive a configured refusal.
The example includes three rail types: regex filtering for jailbreak patterns (no LLM call), content safety classification against S1-S13 categories (violence, criminal planning, profanity), and topic safety classification to keep the agent within a banking domain.
Red Hat OpenShift AI provides two complementary views: agent-level tracing in MLflow shows user queries, tool calls, and responses, while rail-level tracing with OpenTelemetry shows per-rail spans, which rail blocked, and timing for each layer.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Arabic is not a single target for a language model. This explainer maps its four parts — dialect surface, data pipeline, model, and evaluation — so you can tell where an Arabic feature really succeeds or fails.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
