---
title: "From token consumer to token provider: Building your org's AI API"
description: "Enterprise AI usage can become costly and fragmented when teams rely on separate provider accounts, personal keys, and independently operated inference stacks, leaving spending, workloads, and governance difficult to see. The post presents Model-as-a-Service (MaaS) in Red Hat OpenShift AI 3.4 as a shared service in which a platform team deploys approved models and exposes them through a standard, OpenAI-compatible API. Administrators can use the model catalog, distributed inference through llm-d on NVIDIA L40s, scoped API keys, rate limits, and dashboards for token use, latency, errors, GPU utilization, and queue length. Developers obtain self-service endpoints and can connect tools such as Claude Code, OpenCode, or Continue with existing clients. The architecture combines OpenShift, vLLM, an AI gateway, Prometheus, Grafana, and Jaeger to centralize inference, governance, and cost attribution across local and external model providers."
---

# From token consumer to token provider: Building your org's AI API

[Red Hat](https://yomu.fyi/company/red-hat) · Markell Rawls · Sep 10, 2026

**Type:** Explainer

## Summary

Enterprise AI usage can become costly and fragmented when teams rely on separate provider accounts, personal keys, and independently operated inference stacks, leaving spending, workloads, and governance difficult to see. The post presents Model-as-a-Service (MaaS) in Red Hat OpenShift AI 3.4 as a shared service in which a platform team deploys approved models and exposes them through a standard, OpenAI-compatible API. Administrators can use the model catalog, distributed inference through llm-d on NVIDIA L40s, scoped API keys, rate limits, and dashboards for token use, latency, errors, GPU utilization, and queue length. Developers obtain self-service endpoints and can connect tools such as Claude Code, OpenCode, or Continue with existing clients. The architecture combines OpenShift, vLLM, an AI gateway, Prometheus, Grafana, and Jaeger to centralize inference, governance, and cost attribution across local and external model providers.

## Context

AI adoption has left many organizations with separate provider accounts, unauthorized keys, independently deployed inference stacks, limited cost visibility, and insufficient governance over models processing internal data. The post also describes model deprecations breaking applications and cites a finding that 70 to 80% of employees using AI at work rely on employer-unapproved tools.

## Approach / What changed

The proposed Model-as-a-Service pattern gives a platform team responsibility for validated model deployments and exposes them through a shared, OpenAI-compatible gateway. OpenShift AI provides a model catalog, self-service endpoints, scoped API keys, rate limits, observability, and routing to locally served or external models, with OpenShift, vLLM, llm-d, Prometheus, Grafana, and Jaeger supporting the architecture.

## Takeaways

- MaaS exposes approved models through an OpenAI-compatible /v1/chat/completions endpoint and can route requests to locally served vLLM models or external providers such as AWS Bedrock, Azure, and Anthropic.
- API keys are generated for team access, scoped to policy, and revocable; admin-defined rate limits return HTTP 429 responses that standard clients can handle with backoff.
- Prometheus, Grafana, and Jaeger support tracking token consumption, request rates, latency, errors, GPU utilization, and distributed traces.

**Tags:** [Model Serving](https://yomu.fyi/topic/model-serving), [Observability](https://yomu.fyi/topic/observability), [OpenShift AI](https://yomu.fyi/topic/openshift-ai), [vLLM](https://yomu.fyi/topic/vllm)

- Source: [Red Hat](https://developers.redhat.com/articles/2026/09/10/from-token-consumer-to-token-provider-building-your-orgs-ai-api)
- Source URL: https://developers.redhat.com/articles/2026/09/10/from-token-consumer-to-token-provider-building-your-orgs-ai-api
- Ingested by Yomu: 2026-09-10T14:58:52.619Z

[Read original post](https://developers.redhat.com/articles/2026/09/10/from-token-consumer-to-token-provider-building-your-orgs-ai-api)