# Machine Learning
> 67 posts about Machine Learning, summarised, each linking to the original.

## Articles

### [GenPage: Towards End-to-End Generative Homepage Construction at Netflix](https://yomu.fyi/post/genpage-towards-end-to-end-generative-homepage-construction-at-netflix.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Jun 29, 2026

Netflix traditionally constructs its structured two-dimensional homepage through a complex, multi-stage recommender pipeline that separates candidate generation and ranking across rows and entities. To simplify this architecture and optimize directly for whole-page user satisfaction, Netflix developed GenPage, an end-to-end generative transformer model that autoregressively builds the entire homepage from raw tokenized context. The system relies on a domain-specific tokenizer to compress engagement history, context injection for cold start, hybrid row decoding to minimize decoding steps, and reinforcement learning post-training. In online A/B testing against the production baseline, GenPage delivered statistically significant gains in core user engagement metrics while cutting end-to-end serving latency by 20 percent. Offline evaluations further showed that enriching context prompts improved recommendation quality more effectively than increasing model capacity in the current operating regime.


### [How we used DSPy to turn AI evaluations into better responses in Dash chat](https://yomu.fyi/post/how-we-used-dspy-to-turn-ai-evaluations-into-better-responses-in-dash.md)
- Company: [Dropbox](https://yomu.fyi/company/dropbox.md)
- Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon
- Published: Jun 25, 2026

Dropbox improved its Dash chat agent by establishing an automated optimization loop powered by DSPy and LLM-as-judge evaluations. Engineers first calibrated their LLM judges against human-annotated interaction traces, then used those judges to systematically optimize the agent's system prompts via offline counterfactual replay. This automated workflow reduced incomplete responses by 26% while decreasing overall token consumption.


### [Toward More Controllable AI Video Editing: An Early Research Exploration at Netflix](https://yomu.fyi/post/toward-more-controllable-ai-video-editing-an-early-research-exploratio.md)
- Company: [Netflix](https://yomu.fyi/company/netflix.md)
- Author: Netflix Technology Blog
- Published: Jun 23, 2026

Generative video editing tools often regenerate entire video clips, which inadvertently modifies untouched scene details or violates physical continuity during object deletion. To provide artists with precise control, two research prototypes were developed: Vera and VOID. Vera uses a layered video diffusion framework with a Mixture-of-Transformers architecture to jointly generate separate edit and alpha matte layers, compositing them with original footage while leaving unchanged pixels untouched. VOID addresses object deletion by conditioning on interaction-aware quadmasks and using a second-pass appearance refiner to reconstruct scenes with plausible physical continuity. In a user study evaluated across 75 real-world scenarios, reviewers selected VOID 64.8% of the time over six baselines.


### [When history fails you, borrow from geography](https://yomu.fyi/post/when-history-fails-you-borrow-from-geography.md)
- Company: [Airbnb](https://yomu.fyi/company/airbnb.md)
- Author: Harrison Katz
- Published: Jun 2, 2026

When historical travel data became uninformative during the asynchronous post-COVID recovery, Airbnb redesigned its demand forecasting to borrow signals across geographies rather than waiting for local data. Using a hierarchical Bayesian framework, the team propagated updated posterior estimates from early-recovering corridors as informative priors for structurally similar, later-recovering corridors. This allowed accurate corridor-level demand forecasting in near real time during periods of high disruption and uneven market reopening.


### [How we optimized Dash's relevance judge with DSPy](https://yomu.fyi/post/how-we-optimized-dash-s-relevance-judge-with-dspy.md)
- Company: [Dropbox](https://yomu.fyi/company/dropbox.md)
- Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra,Kazuaki Okumura,Mike White,Kevin Altschuler,Facundo Agriel,Ishan Mishra,Eric Wang,Dmitriy Meyerzon
- Published: Mar 17, 2026

Dropbox Dash optimized its LLM-as-a-judge relevance scoring system using DSPy to migrate from expensive proprietary models to cheaper open-weight alternatives. By establishing automated feedback loops based on human agreement and strict JSON format validation, the team systematically generated robust prompts for new models. This reduced human-score disagreement by 45% and slashed model adaptation time from weeks to days while enabling 10x to 100x more data labeling.


### [Using LLMs to amplify human labeling and improve Dash search relevance](https://yomu.fyi/post/using-llms-to-amplify-human-labeling-and-improve-dash-search-relevance.md)
- Company: [Dropbox](https://yomu.fyi/company/dropbox.md)
- Author: Ilya Yakovlev,Andrew Cheung,Binoy Dash,Simran Jumani,Dmitriy Meyerzon,Mark Breitenbach,Ishan Mishra,Kazuaki Okumura,Mike White,Kevin Altschuler,Facundo Agriel,Ishan Mishra,Eric Wang,Dmitriy Meyerzon,Dmitriy Meyerzon
- Published: Feb 26, 2026

Dropbox Dash uses large language models (LLMs) to amplify human labeling efforts for training its search relevance and ranking models. By validating and optimizing LLM evaluators against a small set of human-labeled internal data, Dropbox creates massive, high-quality training datasets offline for production rankers like XGBoost without incurring high latency or latency costs at query time.


### [How Grab is accelerating growth with real-time personalization using Customer Data Platform scenarios](https://yomu.fyi/post/how-grab-is-accelerating-growth-with-real-time-personalization-using-c.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Saubhagya Awaneesh
- Published: Dec 18, 2025

Grab previously relied on daily batch attribute updates in its Customer Data Platform, which created engineering bottlenecks and hindered time-sensitive engagement opportunities. To address this limitation, the team introduced Scenarios, a self-serve real-time personalization capability embedded within the platform. The architecture processes real-time event triggers from Grab's Scribe platform using Apache Flink, enriches incoming events with historical context from StarRocks, and evaluates pre-trained machine learning classifiers. Computed outputs sync to Kafka streams or Amphawa, an internal feature store powered by AWS DynamoDB, maintaining end-to-end latencies under fifteen seconds. Across more than a dozen production deployments, including real-time interventions for subscription abandonment within fifteen minutes, the platform achieved over a 3% conversion uplift compared to batch campaigns.


### [How we built a custom vision LLM to improve document processing at Grab](https://yomu.fyi/post/how-we-built-a-custom-vision-llm-to-improve-document-processing-at-gra.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Jia Chen
- Published: Nov 4, 2025

Document processing for identity verification across Southeast Asia presents challenges due to varied layouts and non-Latin scripts. Traditional OCR and off-the-shelf vision models struggle with accuracy, high latency, or lack of regional language training data. Grab addressed this by creating synthetic regional datasets, using an automated labeling pipeline named Documint, and evaluating open-source multimodal architectures. After initial LoRA fine-tuning failed on complex scripts like Thai and Vietnamese, full-parameter fine-tuning of Qwen2-VL 2B yielded substantial gains. To optimize deployment costs and latency, the team constructed a custom 1B parameter model pairing a Qwen2-VL vision encoder with a Qwen2.5 0.5B language decoder, achieving performance within 3 percentage points of the 2B model at significantly lower latency.


### [Machine-learning predictive autoscaling for Flink](https://yomu.fyi/post/machine-learning-predictive-autoscaling-for-flink.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Minh Nhat Nguyen
- Published: Oct 30, 2025

Managing resource allocations for expanding Apache Flink stream-processing workloads often leads to over-provisioning or unstable reactive scaling. Reactive autoscaling with Kubernetes Horizontal Pod Autoscaling triggers severe restart spikes because pipelines must reprocess backlogged records from checkpoints, inducing feedback loops and scaling fluctuations. To prevent these spirals, a predictive vertical autoscaling system uses upstream Kafka message throughput as an independent workload metric. The architecture forecasts future workload using time-series models and maps the projected throughput to required TaskManager CPU allocations through a trained regression model. A custom controller vertically scales TaskManager CPU resources before traffic changes arrive, avoiding restart-induced latency spikes.


### [Modernising Grab’s model serving platform with NVIDIA Triton Inference Server](https://yomu.fyi/post/modernising-grab-s-model-serving-platform-with-nvidia-triton-inference.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Daniel Tai
- Published: Oct 21, 2025

Grab's machine learning serving platform, Catwalk, experienced mounting technical debt, elevated latency, and rising costs from maintaining disparate inference engines for various frameworks. To address these limitations, the team adopted NVIDIA Triton Inference Server to establish a unified engine, starting with an ONNX migration. They built a custom Triton manager component featuring a proxy layer to translate legacy API requests alongside a server manager that handles model downloads, verification, configuration, and health checks. Within ten days, over half of online deployments migrated seamlessly without requiring client code modifications. The transition reduced p90 latency from 120ms to 20ms on large transformer models and cut average infrastructure spend by approximately 20 percent across evaluated services.


### [User foundation models for Grab](https://yomu.fyi/post/user-foundation-models-for-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Abhinav Rai
- Published: Sep 26, 2025

Traditional recommendation systems at Grab relied on siloed, manually engineered features that struggled to capture sequential interaction data and cross-service user intent across its superapp ecosystem. To resolve these limitations, Grab developed a custom user foundation model built on a transformer architecture designed to learn from both tabular profile data and time-series clickstream interactions. The architecture unifies diverse data modalities—including categorical identifiers, numerical values, text, and geographical coordinates—by framing inputs as key-value token pairs. Custom positional embeddings and attention masks ensure the model processes tabular attributes as unordered sets while maintaining the sequential order of time-series events. The pre-trained model generates holistic user embeddings and supports direct fine-tuning across downstream tasks such as fraud detection, churn prediction, and advertisement optimization.


### [The evolution of Grab's machine learning feature store](https://yomu.fyi/post/the-evolution-of-grab-s-machine-learning-feature-store.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Daniel Tai
- Published: Jul 24, 2025

Grab redesigned its initial machine learning feature store, Amphawa, to address high-dimensional data, complex entity retrieval, and versioning challenges during feature updates. The new architecture adopts a feature-table model where data scientists output Parquet datasets to Amazon S3 using Spark, which are then atomically ingested into Amazon Aurora PostgreSQL via a reverse ETL workflow. To prevent noisy-neighbor contention and optimize infrastructure costs, the platform utilizes Aurora's distributed storage to separate reads from writes. Grab pairs Aurora Serverless on writer nodes to scale up during daily batch ingestion with Provisioned instances on read replicas for steady serving traffic.


### [DispatchGym: Grab’s reinforcement learning research framework](https://yomu.fyi/post/dispatchgym-grab-s-reinforcement-learning-research-framework.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Tan Sien Yi
- Published: Jul 7, 2025

Applying reinforcement learning to dispatch systems is often hindered when the chosen control levers exert weak influence over reward functions. To streamline research, Grab built DispatchGym, a framework that connects reinforcement learning algorithms to a dispatch process simulation via the Gymnasium API. The simulation emphasizes directional accuracy over absolute precision, allowing researchers to evaluate relative metric shifts across supply and demand scenarios. Built in modular Python and accelerated with Numba, the system allows data scientists to test code locally and launch distributed Spark executions with a single command-line call. The framework has been used to evaluate various contextual bandit models and action sampling strategies for tuning dispatch hyperparameters.


### [From failure to success: The birth of GrabGPT, Grab’s internal ChatGPT](https://yomu.fyi/post/from-failure-to-success-the-birth-of-grabgpt-grab-s-internal-chatgpt.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Wenbo Wei
- Published: May 19, 2025

Grab's machine learning platform team initially faced overwhelming volumes of repetitive user inquiries across their internal support channels. An initial attempt to automate answers using the open-source chatbot-ui framework and GPT-3.5-turbo failed to scale because the 8,000-token context limit could not accommodate extensive documentation, and embedding search proved inadequate. The project then pivoted to create an internal conversational AI platform called GrabGPT by wiring chatbot-ui with Google authentication and Grab's catwalk model-serving infrastructure. The resulting internal service rapidly expanded across the organization, providing auditable interactions, multi-model support across OpenAI, Claude, and Gemini, and private network routing to safeguard corporate data.


### [Streamlining RiskOps with the SOP agent framework](https://yomu.fyi/post/streamlining-riskops-with-the-sop-agent-framework.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Fujiao Liu
- Published: May 8, 2025

Manual Account Takeover (ATO) investigations in Risk Operations traditionally demand intensive cross-referencing across systems, manual SQL execution, and high-pressure decision-making prone to human error. To resolve these bottlenecks, an SOP-driven LLM agent framework models investigative workflows as natural-language tree structures with explicit function notations like @function\_name. Execution is coordinated between an SOP planner, which traverses the tree using a Depth-First Search strategy, and a Worker Agent that parses JSON-formatted steps to invoke database queries and APIs. Once all steps evaluate their decision criteria, the framework synthesizes the collected data into an actionable summary report. Implementing this architecture automated 87% of ATO cases and dropped average ticket handling time from 22 minutes to 3 minutes.


### [Introducing the SOP-driven LLM agent frameworks](https://yomu.fyi/post/introducing-the-sop-driven-llm-agent-frameworks.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Fujiao Liu
- Published: Apr 25, 2025

Standard operating procedure (SOP) driven Large Language Model agent frameworks address common generative AI challenges such as hallucinations, non-standard output formats, and branching navigation errors. Workflows are represented as hierarchical trees where nodes encapsulate actions or decision points that can be created using a visual editor and annotated with explicit external function calls. Execution relies on a tripartite architecture consisting of a Depth-First Search planner module with backtracking, an adaptive worker agent that limits API exposure and compresses context, and a multilingual user agent. Supporting tools include a Graph Retrieval-Augmented Generation pipeline, a plugin system integrating Python and SQL, and a state stack for pausing workflows during human intervention. In production deployments for fraud and account takeover investigations, the framework automated up to 87% of cases while cutting handling times substantially.


### [Grab AI Gateway: Connecting Grabbers to multiple GenAI providers](https://yomu.fyi/post/grab-ai-gateway-connecting-grabbers-to-multiple-genai-providers.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Bjorn Jee
- Published: Feb 19, 2025

Grab built the AI Gateway to centralize access, cost control, and security across external and open-source Generative AI providers such as OpenAI, Azure, AWS, and Google. Designed as a set of lightweight reverse proxies, the gateway manages authentication, rate limiting, and authorization while translating payloads into a unified OpenAI-compatible interface. The platform archives request metadata and calculated per-call costs into a central data lake for auditing and showback, dynamically routing traffic across shared reserved capacity and regions to mitigate quota throttling. Supporting over 300 internal use cases, the system integrates directly with internal development notebooks and deployment tools to power applications ranging from real-time audio safety analysis to automated content moderation.


### [Metasense V2: Enhancing, improving and productionisation of LLM powered data governance](https://yomu.fyi/post/metasense-v2-enhancing-improving-and-productionisation-of-llm-powered.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Nick Buhrer
- Published: Nov 14, 2024

Grab scaled Metasense to automate metadata generation, column-level classification, and sensitivity tiering across its entire data lake. Post-rollout analysis revealed that high tag volumes, wide tables, and mixed content—such as nested JSON and customer communications—strained model capacity and led to missed Personally Identifiable Information (PII). To overcome these limitations, the team split the classification process into distinct PII and non-PII tasks, reduced prompt word counts, and partitioned tables with over 150 columns into smaller units. Integrating LangChain and LangSmith modernized the architecture by enabling direct prompt experimentation, custom metric tracking, and version-controlled deployments. The updated pipeline achieved low misclassification rates alongside automated alert thresholds to trigger model improvement protocols if errors increase.


### [LLM-assisted vector similarity search](https://yomu.fyi/post/llm-assisted-vector-similarity-search.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Md Riyadh
- Published: Oct 23, 2024

Vector similarity search often struggles with conceptual nuances, negations, and complex constraints despite its efficiency in high-dimensional semantic retrieval. To address this limitation, a two-step retrieval pipeline combines Facebook AI Similarity Search (FAISS) using OpenAI's text-embedding-ada-002 embeddings with a large language model re-ranking step powered by GPT-4o. The initial FAISS search extracts a candidate shortlist of matches, which GPT-4o then re-ranks based on contextual relevance and query intent. Evaluations on structured datasets showed that while simple queries yielded comparable results across methods, the LLM-assisted approach effectively resolved negations and conceptual matching where raw vector search failed. Internal production trials on relational datasets exceeding 4,500 rows demonstrated similar relevance improvements, with additional query latency being the primary operational trade-off.


### [Evolution of Catwalk: Model serving platform at Grab](https://yomu.fyi/post/evolution-of-catwalk-model-serving-platform-at-grab.md)
- Company: [Grab](https://yomu.fyi/company/grab.md)
- Author: Vishal Sharma
- Published: Oct 1, 2024

Grab developed and scaled Catwalk, an internal machine learning model serving platform, to address operational bottlenecks, low resource utilization, and deployment friction between data scientists and backend engineers. The platform transitioned from an admin-managed TensorFlow Serving setup into a low-code self-service system supporting PyTorch and ONNX, before replacing complex Helm charts with Kubernetes Custom Resource Definitions for declarative, blue-green deployment orchestration. To support complex business workflows and multi-model applications, Grab subsequently introduced Catwalk Orchestrator with bundled deployments that allow individual services to scale independently. Across two years, the orchestrator architecture expanded to 200 deployed applications serving approximately 1,400 production machine learning models.


[Newer posts](https://yomu.fyi/topic/machine-learning.md) · [Older posts](https://yomu.fyi/topic/machine-learning/page/3.md)