Loading…
Topic
NeMo Guardrails
4 posts about NeMo Guardrails. Every summary links to the original.
- 4 posts
- 1 company
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Add NeMo Guardrails to a LangGraph agent on OpenShift AI
LangGraph agents can call tools and follow prompts but lack safety guardrails, risking unsafe responses like explaining fraud. NeMo Guardrails acts as a transparent proxy between the agent and LLM, inspecting every request and response without requiring agent code changes. On Red Hat OpenShift AI, the guardrails service is deployed via a NemoGuardrails custom resource managed by the TrustyAI Operator, and the agent’s BASE_URL points to this service instead of the model endpoint. The example demonstrates three rail types: regex filtering for jailbreak patterns, content safety classification against S1-S13 categories, and topic safety to keep the agent within a banking domain, with optional tracing using MLflow and OpenTelemetry. Rails execute sequentially, with the first failure short-circuiting the chain, and the separation lets safety policy be owned by guardrails while the agent handles reasoning and tools.
Deploy NeMo Guardrails on Red Hat OpenShift AI
This article shows how to deploy an evaluated NeMo Guardrails configuration on a Red Hat OpenShift AI cluster, continuing a series on local development and dataset-based evaluation. It requires OpenShift AI 3.5 or later and namespace write access, while OpenShift AI 3.2 or newer can be configured to use the 3.5 NeMo Guardrails image through TrustyAI service-operator changes. Deployment uses a ConfigMap for the configuration, a NemoGuardrails custom resource, and an authentication annotation that protects the server route; an empty rails.co file is sufficient when no custom algorithms are defined. The deployed server accepts v1/guardrail/checks requests with a bearer token, and the demonstrated classifier permits a benign prompt, blocks a prompt injection, and rejects an invalid token. The article notes that LLM proxying and LLM-based guardrailing are outside its scope.
Evaluating LLM guardrail configs locally with EvalHub
This installment evaluates local NeMo Guardrails configurations against large-scale risk datasets using EvalHub, following earlier manual testing. EvalHub provides a unified interface for AI evaluation, with provider integrations, reusable benchmarks, Hugging Face dataset inputs, and both local and Kubernetes deployment options. The benchmark configuration maps dataset prompt and label columns to block and pass labels, while balanced safe and risky examples prevent an always-block guardrail from appearing perfect. On the prompt-injection benchmark, the nuanced regex configuration achieved 51.1% accuracy with 13.7 ms median latency; DeBERTa reached 88.93% accuracy with 45.2 ms median latency. The conclusion favors layered guardrails: targeted regex checks can provide speed, with DeBERTa handling other cases, and EvalHub supplies reproducible measurements of the latency–efficacy trade-off.
Developing LLM guardrail configs locally with NeMo Guardrails
The article presents a local workflow for developing and testing NeMo Guardrails configurations in a Jupyter notebook, avoiding repeated Kubernetes deployments and requiring no LLM, GPU, or cloud resources. It defines rails, configurations, and servers, then starts with a case-insensitive regex rail that blocks the exact phrase “ignore all previous instructions”; the test catches a direct attack but allows an accented “Ignóre” variant. A broader regex configuration is followed by a DeBERTa-based prompt-injection classifier and a manually assembled set of 13 prompts, producing 12 correct decisions out of 13, with one subtle attack allowed. The article recommends iterating locally, potentially layering targeted regexes before classifiers for fail-fast, defense-in-depth filtering, while noting that 13 prompts cannot meaningfully measure performance and pointing to standardized EvalHub evaluations as a next step.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.