Grab AI Gateway: Connecting Grabbers to multiple GenAI providers

Summary

Grab AI Gateway is a central access layer connecting Grabbers’ applications to multiple GenAI providers, including OpenAI, Azure, AWS Bedrock, and Google VertexAI. Built as a set of reverse proxies, it centralizes provider authentication, authorization, rate limiting, auditing, and capacity management while presenting a unified OpenAI-style API; users can switch models by changing the model parameter. Exploration keys support short-lived staging experiments, while reviewed use cases can use longer-term service keys, and the gateway exposes embedding, image, audio, fine-tuning, search, and in-house open source models. Shared capacity, dynamic routing, monitoring, cost attribution, and usage records support governance and efficiency, although batch traffic can interfere with latency-sensitive services and provider-specific SDK edge cases require integration testing. More than 3,000 Grabbers have requested exploration keys, and the gateway powers applications including ride-safety audio analysis, content moderation, SQL generation, incident automation, and Slack support.

Takeaways

1. 1

   Exploration keys are short-lived, limited to staging, subject to stricter rate limits, and available through a Slack bot; more than 3,000 Grabbers have requested them.

2. 2

   The unified interface translates OpenAI-scheme requests into provider-specific payloads and responses, allowing provider changes, fallback logic, and dynamic routing without rewriting application logic.

3. 3

   Shared capacity pools and routing across models or regions improve utilization and help reduce rate-limiting, but batch workloads can interfere with latency-sensitive online services.