Loading…
Architecture
173 posts about Architecture. Every summary links to the original.
Slack ·
Managing context in long-run agentic applications
Long-running multi-agent systems struggle with context management because accumulating raw message histories degrades inference quality, increases latency, and exceeds context window limits. In a collaborative security investigation platform, passing unrestricted history can also introduce confirmation bias across specialized agents. To maintain coherence across unbounded rounds of investigation, the system eliminates raw message history carryover between invocations. Instead, it coordinates agents through three structured context channels: a Director's Journal for orchestration memory, a Critic's Review that scores findings to filter hallucinations, and a Critic's Timeline of validated chronological events. This architecture provides agents with tailored context without overwhelming their inference capacity.
Dominic MarksDropbox ·
Improving storage efficiency in Magic Pocket, our immutable blob store
After introducing a direct erasure-coding service, Dropbox encountered severe storage fragmentation in its immutable blob store, Magic Pocket, due to a large number of under-filled volumes. Because their legacy compaction strategy was only designed to top off near-full volumes, the team created new multi-strategy compaction methods to aggressively consolidate sparse volumes. These approaches leverage dynamic programming and streaming pipelines to rapidly reclaim unused disk capacity at exabyte scale.
Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra,Kazuaki Okumura,Mike White,Kevin Altschuler,Facundo AgrielSlack ·
How Slack Rebuilt Notifications 📣
Slack redesigned its legacy notification architecture to address user confusion, conflicting desktop and mobile preference systems, and tight coupling between notification content and push delivery. The engineering team unified four separate preference models into a single framework with three channel options: All new posts, Mentions, and Mute. To ensure safe rollback and backwards compatibility without direct database mutations, Slack implemented a read-time translation strategy alongside a new desktop_push_enabled preference and auto-saving cross-platform React UI components. This decoupling of activity from push delivery resulted in a fivefold sustained increase in settings engagement, reduced reliance on per-channel overrides, and eliminated synchronization mismatches across mobile and desktop clients.
Frances CoronelGrab ·
From firefighting to building: How AI agents restored our team’s core productivity
Grab's Analytics Data Warehouse team spent roughly 40% of their engineering bandwidth answering repetitive questions, tracing data lineage, and handling basic pipeline enhancement requests across more than 15,000 tables. To eliminate these manual investigative bottlenecks, the team implemented a multi-agent AI architecture using FastAPI, LangGraph, Redis, and PostgreSQL. Incoming requests route through two dedicated pathways: an enhancement pipeline for generating code changes and an investigation pipeline for diagnosing data anomalies. Specialized agents interact with underlying engines like Trino, GitLab, and observability platforms to query data, trace transformations, and check ongoing incidents before synthesizing findings. This system automates the context-gathering process within minutes while maintaining human-in-the-loop review for merge requests and production changes.
Sneh AgrawalGrab ·
From deployment slop to production reality: How BriX bridges the gap with enterprise-grade AI infrastructure
Internal AI prototypes frequently fail enterprise rollouts due to diverging versions, security oversights, hardcoded credentials, and infrastructure bottlenecks. BriX addresses this deployment gap by turning AI rollout into a configuration-driven platform rather than an engineering rewrite. Built on a synchronous streaming architecture, it routes user prompts through a React frontend using Server-Sent Events, a FastAPI gateway, and LangGraph orchestration. The platform integrates model switching, centralized prompt locks, and standardized Model Context Protocols for governed enterprise data access.
Sneh AgrawalGrab ·
How Grab is accelerating growth with real-time personalization using Customer Data Platform scenarios
Grab previously relied on daily batch attribute updates in its Customer Data Platform, which created engineering bottlenecks and hindered time-sensitive engagement opportunities. To address this limitation, the team introduced Scenarios, a self-serve real-time personalization capability embedded within the platform. The architecture processes real-time event triggers from Grab's Scribe platform using Apache Flink, enriches incoming events with historical context from StarRocks, and evaluates pre-trained machine learning classifiers. Computed outputs sync to Kafka streams or Amphawa, an internal feature store powered by AWS DynamoDB, maintaining end-to-end latencies under fifteen seconds. Across more than a dozen production deployments, including real-time interventions for subscription abandonment within fifteen minutes, the platform achieved over a 3% conversion uplift compared to batch campaigns.
Saubhagya AwaneeshSlack ·
Streamlining Security Investigations with Agents
Slack's Security Engineering team handles billions of daily security events and needed a reliable way to streamline on-call alert triage. An initial prototype relying on a single 300-word prompt produced inconsistent results and frequently reached spurious conclusions without properly challenging assumptions. To gain precise control, the team decomposed the workflow into chained model invocations with structured JSON outputs organized across three agent personas: a Director, four domain experts, and a Critic. Domain experts gather raw evidence through tool calls, the Critic evaluates finding quality and synthesizes a timeline, and the Director steers investigation phases using tiered model costs. The multi-agent system enables engineers to supervise investigations via a real-time dashboard while uncovering emergent issues like credential exposures across process ancestry chains.
Dominic MarksGrab ·
SpellVault’s evolution: Beyond LLM apps, towards the agentic future
Grab developed SpellVault as an internal no-code platform to democratize the creation of AI applications backed by Retrieval-Augmented Generation (RAG) and plugin integrations. To advance beyond static retrieval and linear input-output processing, the platform transitioned from its legacy executor to a graph-based execution model supporting branching, looping, and ReAct agent patterns. Capabilities like Python code execution and internal repository searching were unbundled from the prompt builder and consolidated alongside user plugins into unified Native and Community Built Tools. The platform also introduced a drag-and-drop deterministic workflow designer, automated task scheduling, and support for the Model Context Protocol (MCP).
Felix Haryanto LieGrab ·
Modernising Grab’s model serving platform with NVIDIA Triton Inference Server
Grab's machine learning serving platform, Catwalk, experienced mounting technical debt, elevated latency, and rising costs from maintaining disparate inference engines for various frameworks. To address these limitations, the team adopted NVIDIA Triton Inference Server to establish a unified engine, starting with an ONNX migration. They built a custom Triton manager component featuring a proxy layer to translate legacy API requests alongside a server manager that handles model downloads, verification, configuration, and health checks. Within ten days, over half of online deployments migrated seamlessly without requiring client code modifications. The transition reduced p90 latency from 120ms to 20ms on large transformer models and cut average infrastructure spend by approximately 20 percent across evaluated services.
Daniel TaiGrab ·
User foundation models for Grab
Traditional recommendation systems at Grab relied on siloed, manually engineered features that struggled to capture sequential interaction data and cross-service user intent across its superapp ecosystem. To resolve these limitations, Grab developed a custom user foundation model built on a transformer architecture designed to learn from both tabular profile data and time-series clickstream interactions. The architecture unifies diverse data modalities—including categorical identifiers, numerical values, text, and geographical coordinates—by framing inputs as key-value token pairs. Custom positional embeddings and attention masks ensure the model processes tabular attributes as unordered sets while maintaining the sequential order of time-series events. The pre-trained model generates holistic user embeddings and supports direct fine-tuning across downstream tasks such as fraud detection, churn prediction, and advertisement optimization.
Abhinav RaiGrab ·
Data mesh at Grab part I: Building trust through certification
Rapid business growth across multiple verticals led Grab's centralized data engineering model to become an unscalable bottleneck, resulting in duplicate pipelines, ambiguous ownership, and broken downstream dependencies. To resolve these issues, the organization initiated a data mesh journey called Signals Marketplace that decentralizes data management and treats data as a product. A central data certification system establishes formal data contracts covering schemas, SLAs, freshness, and retention, while assigning clear Business Data Owners and Technical Data Owners. Breaches in contract guarantees automatically generate Data Production Incident tickets to enforce accountability and root-cause fixes. Consequently, 75% of internal queries now target certified assets, redundant tables saw a 400% year-over-year deprecation increase, and the total number of top-used datasets dropped by over 58%.
Chun Rong PhangGrab ·
Grab's service mesh evolution: From Consul to Istio
Grab operated over 1,000 microservices across hybrid infrastructure using Consul alongside a fallback mechanism called Catcher. Single-point-of-failure vulnerabilities in Consul servers and limited support for multi-cluster operations prompted an evaluation of alternative mesh technologies, ultimately leading to the selection of Istio. Grab avoided the standard single-control-plane-per-cluster pattern by deploying multiple external control planes in dedicated Kubernetes clusters arranged in active-active pairs. Migration began in Q4 2024, shifting traffic across AWS and GCP while handling both HTTP and gRPC protocols with gradual traffic-shifting and rollback mechanisms.
Hilman KurniawanGrab ·
DispatchGym: Grab’s reinforcement learning research framework
Applying reinforcement learning to dispatch systems is often hindered when the chosen control levers exert weak influence over reward functions. To streamline research, Grab built DispatchGym, a framework that connects reinforcement learning algorithms to a dispatch process simulation via the Gymnasium API. The simulation emphasizes directional accuracy over absolute precision, allowing researchers to evaluate relative metric shifts across supply and demand scenarios. Built in modular Python and accelerated with Numba, the system allows data scientists to test code locally and launch distributed Spark executions with a single command-line call. The framework has been used to evaluate various contextual bandit models and action sampling strategies for tuning dispatch hyperparameters.
Tan Sien YiGrab ·
The complete stream processing journey on FlinkSQL
Grab previously relied on Apache Zeppelin notebooks for interactive stream processing exploration, but faced lagging Flink version upgrades, five-minute cluster cold starts, and poor integration with internal platforms. To address these limitations, the team migrated to a shared FlinkSQL gateway architecture structured into compute, integration, and query layers. The new setup uses a Hive Metastore catalog to expose Kafka topics as relational tables, while a custom control plane handles authentication and headless REST APIs over Flink's native interface. For production workflows, a configuration-based portal accepts SQL logic and automatically provisions and deploys Flink pipelines within ten minutes. This transition reduced ad-hoc query response times to under one minute and eliminated the need to maintain version adapter shims.
Calvin TranGrab ·
Effortless enterprise authentication at Grab: Dex in action
Grab needed a centralised system to simplify identity management, satisfy audit requirements, and standardise authentication across internal and external tools like Databricks and Datadog. The engineering team selected OpenID Connect as their standard protocol and adopted Dex, an open-source CNCF identity aggregator. Dex acts as an intermediary between applications and multiple identity providers to issue standardised OIDC tokens. To secure service-to-service communication, Grab implemented token exchange with trusted peer relationships rather than relying on privileged service accounts. Dex also provides a kill-switch mechanism that can route authentication traffic to an alternate provider during identity provider outages.
Kah Wei LeeGrab ·
Streamlining RiskOps with the SOP agent framework
Manual Account Takeover (ATO) investigations in Risk Operations traditionally demand intensive cross-referencing across systems, manual SQL execution, and high-pressure decision-making prone to human error. To resolve these bottlenecks, an SOP-driven LLM agent framework models investigative workflows as natural-language tree structures with explicit function notations like @function_name. Execution is coordinated between an SOP planner, which traverses the tree using a Depth-First Search strategy, and a Worker Agent that parses JSON-formatted steps to invoke database queries and APIs. Once all steps evaluate their decision criteria, the framework synthesizes the collected data into an actionable summary report. Implementing this architecture automated 87% of ATO cases and dropped average ticket handling time from 22 minutes to 3 minutes.
Fujiao LiuGrab ·
Introducing the SOP-driven LLM agent frameworks
Standard operating procedure (SOP) driven Large Language Model agent frameworks address common generative AI challenges such as hallucinations, non-standard output formats, and branching navigation errors. Workflows are represented as hierarchical trees where nodes encapsulate actions or decision points that can be created using a visual editor and annotated with explicit external function calls. Execution relies on a tripartite architecture consisting of a Depth-First Search planner module with backtracking, an adaptive worker agent that limits API exposure and compresses context, and a multilingual user agent. Supporting tools include a Graph Retrieval-Augmented Generation pipeline, a plugin system integrating Python and SQL, and a state stack for pausing workflows during human intervention. In production deployments for fraud and account takeover investigations, the framework automated up to 87% of cases while cutting handling times substantially.
Fujiao LiuGrab ·
Evaluating performance impact of removing Redis-cache from a Scylla-backed service
Grab operates a high-throughput Rust read service that aggregates counter metrics from Scylla tables across minutely, hourly, and daily granularities. The service initially cached aggregated responses in Redis using keys rounded to 15-minute intervals alongside a five-minute TTL. Because incoming queries predominantly requested recent data, transitions between 15-minute windows caused simultaneous cache misses across active configurations, resulting in severe Scylla traffic spikes, latency surges, and timeouts. To resolve the load imbalance, engineers proposed removing the Redis cache entirely and relying directly on Scylla's native internal caching. The rollout was staged in production by deterministically disabling Redis caching for specific counter configurations using mathematical operations on configuration IDs.
Md RiyadhGrab ·
Grab AI Gateway: Connecting Grabbers to multiple GenAI providers
Grab built the AI Gateway to centralize access, cost control, and security across external and open-source Generative AI providers such as OpenAI, Azure, AWS, and Google. Designed as a set of lightweight reverse proxies, the gateway manages authentication, rate limiting, and authorization while translating payloads into a unified OpenAI-compatible interface. The platform archives request metadata and calculated per-call costs into a central data lake for auditing and showback, dynamically routing traffic across shared reserved capacity and regions to mitigate quota throttling. Supporting over 300 internal use cases, the system integrates directly with internal development notebooks and deployment tools to power applications ranging from real-time audio safety analysis to automated content moderation.
Bjorn JeeGrab ·
Embracing passwordless authentication with Grab’s Passkey
Grab introduced Passkey to replace vulnerable traditional passwords and cumbersome multi-factor methods with a seamless, phishing-resistant alternative based on the FIDO standard. The architecture relies on an authenticator located on the user's device, a frontend client, and a backend storing only public keys and metadata. During registration and login, the frontend invokes WebAuthn APIs such as navigator.credentials.create and navigator.credentials.get using server-generated challenges to prevent replay attacks. Passkeys synchronize across ecosystems via Google Password Manager and Apple iCloud Keychain, allowing users to authorize logins with their device lock screen. This implementation improves user experience, eliminates the need to store secrets in backend databases, and cuts third-party communication costs associated with OTP delivery.
Ocean Nguyen