Bloom Filters: The Cheap Gate Before the Expensive Question
A Bloom filter is a small bit array plus a few hash functions that answers "has this been seen?" with no false negatives and a false-positive rate you choose up front.
Loading…
Topic
Search
185 posts about Search. Every summary links to the original.
- 185 posts
- 31 companies
- 3 explainers
See the idea
Search explainers
Apache Lucene Scoring: BM25 Relevance vs StableTfl Consistency
A comparison of Apache Lucene's BM25 scoring and StableTfl for paginated search: BM25 maximizes relevance, while StableTfl trades a small relevance drop for consistent ranking across replicas and pages.
AI Search: Speed vs. Relevance Trade-off
AI Search balances retrieval speed and result quality by choosing between dense embeddings, sparse lexical search, and hybrid approaches. Understand which trade-off fits your workload.
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
Newest first
Posts
Pharos - Searching Nearby Drivers on Road Network at Scale
Ride-hailing allocation requires identifying nearby drivers based on actual routing distance and estimated time of arrival rather than straight-line haversine distance. Grab developed Pharos, a distributed in-memory spatial microservice that evaluates road networks to match fast-moving drivers with pick-up requests. The system partitions OpenStreetMap road graphs by city and vehicle vertical while indexing driver positions and edge-based nodes using Adaptive Radix Trees. To locate candidates, Pharos projects driver coordinates onto road segments as phantom nodes and runs Incremental Network Expansion to compute isochrone reaches. Read-write concurrency is managed by switching tree root snapshots, allowing concurrent reads while updates produce new roots for subsequent lookups.
Xendit Built a Counter-Fraud Watchlist for the Fintech Industry
Payment processor Xendit needed a fast, cost-effective counter-fraud watchlist system to screen transactions against international sanctions lists in real time without relying on expensive providers like Worldcheck or Refinitiv. Leveraging their existing familiarity with Postgres, the team loaded parsed sanctions lists from the United Nations and the Indonesian government into Supabase. They implemented full-text search using the Postgres Trigram extension, enabling relevance scoring across queried records. A dedicated database function handles the search queries directly via Xendit's Python clients, allowing the entire solution to reach production in under one week. The deployment has operated reliably in production for nine months without issues, with potential future iterations planned around machine learning techniques.
How We’re Solving Data Discovery Challenges at Shopify
Shopify describes data discovery as a response to growing numbers of data assets and the difficulty of finding, governing, and using them across the data lifecycle. At Shopify, pre-Artifact discovery relied on people, Slack, GitHub, and job logs, while teams lacked granular ownership, change-impact visibility, and documented reporting context. They built Artifact, a search-and-browse tool backed by a centralized metadata model, ingestion jobs, Elasticsearch indexes, GraphQL APIs, and a graph database for filterable upstream and downstream lineage. Its landing page supports browsing and plain-English search, while details pages expose documentation, schema, descriptions, usage, ownership, and lineage; initial scope prioritized frequently used, high-impact, queryable assets and downstream reports and dashboards. Since launch in early 2020, weekly Data team usage exceeded 30% and monthly retention exceeded 50%, above expectations, with future work planned for additional assets, integrations, stewardship, change management, notifications, and metadata APIs.
Five Common Data Stores and When to Use Them
The article compares five common data stores—relational databases, NoSQL databases, key-value stores, full-text search engines, and message queues—and connects each to appropriate design needs. Relational databases use schema-bound tables, primary and foreign keys, SQL, and ACID transactions for durable business-critical data, while NoSQL databases trade strict schemas for flexibility with document and wide-column models. In-memory stores such as Redis and Memcached provide fast access and configurable eviction for caching; Redis additionally supports richer data structures and disk persistence. Full-text search engines target word and substring searches, whereas Kafka uses a distributed append-only log and producer-consumer model to move data reliably between services. The article concludes that the right choice depends on whether data must be permanent, flexible, quickly accessed, searched, or transferred.
Save Your Place with Grab!
Grab introduced Saved Places across Southeast Asia to eliminate the friction of repeatedly typing addresses and prevent selection errors between similarly named locations. Data analysis across transport and food orders revealed that consumers consistently visit only five to seven unique locations and order food to one or two addresses. To streamline repeat bookings, cross-functional teams designed a feature allowing users to bookmark locations under custom labels such as Home and Work. Following usability testing and release, more than 14 million users stored nearly 45 million addresses across the platform. Platform metrics also showed that while office destinations clustered in central districts across major cities, residential distributions varied significantly between markets like Singapore and Jakarta.
Related topics
All topics →Weekly digest
The best engineering reads, once a week.