Developing LLM guardrail configs locally with NeMo Guardrails

Summary

The article presents a local workflow for developing and testing NeMo Guardrails configurations in a Jupyter notebook, avoiding repeated Kubernetes deployments and requiring no LLM, GPU, or cloud resources. It defines rails, configurations, and servers, then starts with a case-insensitive regex rail that blocks the exact phrase “ignore all previous instructions”; the test catches a direct attack but allows an accented “Ignóre” variant. A broader regex configuration is followed by a DeBERTa-based prompt-injection classifier and a manually assembled set of 13 prompts, producing 12 correct decisions out of 13, with one subtle attack allowed. The article recommends iterating locally, potentially layering targeted regexes before classifiers for fail-fast, defense-in-depth filtering, while noting that 13 prompts cannot meaningfully measure performance and pointing to standardized EvalHub evaluations as a next step.

Context

Deploying NeMo Guardrails to a Kubernetes cluster for every configuration change is impractical, while manual testing of a small prompt set is not scalable or rigorous enough to measure guardrail performance.

Approach / What changed

The workflow runs NeMo Guardrails locally from a Jupyter notebook, testing progressively broader regex configurations and a DeBERTa-based prompt-injection classifier. It also demonstrates sequentially layering a targeted regex rail before the classifier so specific matches fail fast and more complex cases fall through to classification.

Takeaways

• A case-insensitive regex blocking “ignore all previous instructions” stopped the direct attack but allowed the obfuscated “Ignóre” variant.
• The DeBERTa classifier made 12 correct decisions on a manually assembled set of 13 prompts, allowing one subtle prompt-injection attack.
• NeMo input rails run sequentially, enabling targeted regexes to precede classifiers for fail-fast filtering and defense-in-depth.

Related reading