Elasticsearch

5 posts about Elasticsearch. Every summary links to the original.

Grab · Sep 26, 2024
Enabling conversational data discovery with LLMs at Grab
Grab faced major bottlenecks in dataset discovery across its 200,000 data lake tables due to low documentation coverage and vanilla Elasticsearch limitations in its internal tool, Hubble. To resolve this, engineers first tuned Elasticsearch by boosting frequently queried P80 tables and certified schemas while deboosting deprecated datasets. They then deployed a GPT-4 pipeline that generated documentation from table schemas and sample data, raising P80 documentation coverage from 20% to roughly 90%. Finally, the team developed HubbleIQ, an LLM-powered assistant built via Glean Apps and integrated into Slack and Hubble search to answer semantic queries. These initiatives increased search click-through rates to 94% and lifted the proportion of users finding discovery easy to 73%.
Shreyas Parbat
Grab · Feb 11, 2022
How Grab built a scalable, high-performance ad server
Grab transitioned from an off-the-shelf MVP to an in-house ad serving system to accommodate business scale, hyperlocal requirements, and machine learning personalization. The architecture orchestrates core microservices and data pipelines across sequential steps: targeting, capping, pacing, scoring, ranking, pricing, and tracking. ElasticSearch serves as the targeting ads repository, while ScyllaDB acts as the high-throughput stats store fed by Kafka streams and data pipelines. The system operates on key engineering principles including parallelization and tuned latency limits, graceful fallbacks for slow dependency calls, and a unified server serving all ad types across the superapp.
Anthony McCallum
Grab · May 24, 2021
Building a Hyper Self-Service, Distributed Tracing and Feedback System for Rule & Machine Learning (ML) Predictions
Grab's Trust, Identity, Safety, and Security team processes billions of daily rule and machine learning decisions for fraud detection, safety, and identity checks. Earlier logging approaches using plain text Kibana logs and the ActionTrace library lacked structured formats, dynamic entity customization, and fine-grained access controls. To resolve these limitations, the team built Archivist, a centralized tracing, statistics, and feedback system. Archivist ingests events through an SDK into Kafka streams, buffers and routes data into Elasticsearch indices and Amazon S3, and provides a role-based user portal. The platform handles 80 million daily logs across roughly 50 business scenarios, reducing scenario onboarding times from days to minutes.
Warren Zhou
Grab · Jul 31, 2019
How We Built a Logging Stack at Grab
Grab needed a scalable logging platform to replace slow, fragmented systems that hindered debugging across their growing service fleet. Generating 25TB of daily logs, the team built a horizontally scalable Elasticsearch cluster configured via Ansible and monitored with Datadog. Although the initial proof of concept assigned all node roles (ingest, coordinator, master, and data) to every machine, operating at scale introduced major challenges with JVM heap exhaustion and cluster stability. The team resolved memory pressure and performance bottlenecks by tuning circuit breakers, lowering field data cache limits, adjusting shard allocations based on segment memory, and disabling translog compression during shard transfers.
Daniel Kasen
Grab · Mar 5, 2019
Structured Logging: The Best Friend You’ll Want When Things Go Wrong
Grab redesigned its backend logging approach to address mounting vendor costs, query language limitations, and debugging difficulties across a growing microservices ecosystem. Most services previously emitted syslog-style key-value logs almost entirely at the INFO level, which made volume reduction difficult and lacked causal ordering and automated correlation. The engineering team migrated to a self-managed Elastic stack backend and built a structured logging library in Go from the ground up. This framework introduces dynamic log-level adjustment at runtime, automatic trace-based log correlation via Grab-Kit, and the Common Grab Log Schema to enforce consistent JSON formatting without Elasticsearch indexing conflicts.
Aditya Praharaj

Enabling conversational data discovery with LLMs at Grab

Grab faced major bottlenecks in dataset discovery across its 200,000 data lake tables due to low documentation coverage and vanilla Elasticsearch limitations in its internal tool, Hubble. To resolve this, engineers first tuned Elasticsearch by boosting frequently queried P80 tables and certified schemas while deboosting deprecated datasets. They then deployed a GPT-4 pipeline that generated documentation from table schemas and sample data, raising P80 documentation coverage from 20% to roughly 90%. Finally, the team developed HubbleIQ, an LLM-powered assistant built via Glean Apps and integrated into Slack and Hubble search to answer semantic queries. These initiatives increased search click-through rates to 94% and lifted the proportion of users finding discovery easy to 73%.

Shreyas Parbat

How Grab built a scalable, high-performance ad server

Grab transitioned from an off-the-shelf MVP to an in-house ad serving system to accommodate business scale, hyperlocal requirements, and machine learning personalization. The architecture orchestrates core microservices and data pipelines across sequential steps: targeting, capping, pacing, scoring, ranking, pricing, and tracking. ElasticSearch serves as the targeting ads repository, while ScyllaDB acts as the high-throughput stats store fed by Kafka streams and data pipelines. The system operates on key engineering principles including parallelization and tuned latency limits, graceful fallbacks for slow dependency calls, and a unified server serving all ad types across the superapp.

Anthony McCallum

Building a Hyper Self-Service, Distributed Tracing and Feedback System for Rule & Machine Learning (ML) Predictions

Grab's Trust, Identity, Safety, and Security team processes billions of daily rule and machine learning decisions for fraud detection, safety, and identity checks. Earlier logging approaches using plain text Kibana logs and the ActionTrace library lacked structured formats, dynamic entity customization, and fine-grained access controls. To resolve these limitations, the team built Archivist, a centralized tracing, statistics, and feedback system. Archivist ingests events through an SDK into Kafka streams, buffers and routes data into Elasticsearch indices and Amazon S3, and provides a role-based user portal. The platform handles 80 million daily logs across roughly 50 business scenarios, reducing scenario onboarding times from days to minutes.

Warren Zhou

How We Built a Logging Stack at Grab

Grab needed a scalable logging platform to replace slow, fragmented systems that hindered debugging across their growing service fleet. Generating 25TB of daily logs, the team built a horizontally scalable Elasticsearch cluster configured via Ansible and monitored with Datadog. Although the initial proof of concept assigned all node roles (ingest, coordinator, master, and data) to every machine, operating at scale introduced major challenges with JVM heap exhaustion and cluster stability. The team resolved memory pressure and performance bottlenecks by tuning circuit breakers, lowering field data cache limits, adjusting shard allocations based on segment memory, and disabling translog compression during shard transfers.

Daniel Kasen

Structured Logging: The Best Friend You’ll Want When Things Go Wrong

Grab redesigned its backend logging approach to address mounting vendor costs, query language limitations, and debugging difficulties across a growing microservices ecosystem. Most services previously emitted syslog-style key-value logs almost entirely at the INFO level, which made volume reduction difficult and lacked causal ordering and automated correlation. The engineering team migrated to a self-managed Elastic stack backend and built a structured logging library in Go from the ground up. This framework introduces dynamic log-level adjustment at runtime, automatic trace-based log correlation via Grab-Kit, and the Common Grab Log Schema to enforce consistent JSON formatting without Elasticsearch indexing conflicts.

Aditya Praharaj