Red HatclobnerTutorial
Red Hat AI Inference 3.5 extends its Model-as-a-Service (MaaS) governance layer to non-OpenShift Kubernetes distributions, including Azure Kubernetes Service (AKS). MaaS adds multi-tenant API governance on top of the inference platform, controlling authentication, rate limiting, and model access via Authorino and Limitador from Red Hat Connectivity Link (Kuadrant). A model is first deployed with a single LLMInferenceService YAML, creating a vLLM server and router-scheduler, but the endpoint is open. Applying a MaaSSubscription and MaaSAuthPolicy activates governance: Authorino validates API keys or JWTs, and Limitador enforces token rate limits. The post demonstrates three enforcement scenarios on an AKS cluster with a T4 GPU running Qwen3-0.6B: 401 for missing or invalid keys, 200 for authenticated requests, and 429 when rate limits are exceeded, blocking requests before they reach vLLM.
The post addresses the governance challenge of securing AI model endpoints, authenticating developers, and preventing runaway GPU costs on public cloud Kubernetes clusters. It builds on a previous article about Red Hat AI Inference’s inference layer on non-OpenShift managed Kubernetes (xKS) and focuses on the MaaS governance layer deployed on Azure Kubernetes Service (AKS) with a cluster of 4 nodes (3 CPU, 1 GPU with NVIDIA T4) running Kubernetes 1.34.8.
Deploy Red Hat AI Inference 3.5 with MaaS enabled on AKS. Install the governance layer components: RHCL/Kuadrant (Authorino for authentication, Limitador for rate limiting), MaaS controller, MaaS API, and MaaS gateway. Deploy a model using the LLMInferenceService CRD. Then apply a MaaSSubscription to define access tiers and a MaaSAuthPolicy to require API key authentication. The MaaS controller reconciles these CRDs into Kuadrant AuthPolicies and RateLimitPolicies. API keys are minted via the MaaS API using Kubernetes service account tokens.
MaaS governance on AKS requires external tools like Helm, cert-manager, and an OIDC provider (Azure AD) because vanilla Kubernetes lacks OpenShift’s built-in OLM, CA, and OAuth server.
Without MaaSSubscription and MaaSAuthPolicy, the model endpoint is open to anyone; applying these two YAML files activates authentication and rate limiting.
Rate limiting is enforced by Limitador at the gateway before requests reach vLLM, preventing wasted GPU compute on rejected requests.
The request returns 401 Unauthorized because Authorino rejects requests without a valid API key or JWT.
Authenticate with a Kubernetes service account token (e.g., `kubectl create token default -n models-as-a-service`) and call the MaaS API endpoint `/v1/api-keys` with the token to mint a one-time `sk-oai-*` key.
The inference gateway serves models directly without authentication; the MaaS gateway adds policy enforcement (authentication and rate limiting) and uses a different path format (`/v1/chat/completions` with namespaced model identifier).
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
A model deployed on AKS is open by default. See how MaaS policies insert Authorino authentication and Limitador rate limits in front of vLLM, so bad keys and over-quota requests stop at the gate.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
