--- title: "Latest reads" description: "The engineering internet, summarised so you can actually read it." --- # Latest reads > The engineering internet, summarised so you can actually read it. ## Articles ### [Migrating from Role to Attribute-based Access Control](https://yomu.fyi/post/migrating-from-role-to-attribute-based-access-control.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Minh Khoi Nguyen - Published: Mar 9, 2023 Grab's streaming data platform team migrated the Kafka Control Plane from Role-Based Access Control to Attribute-Based Access Control to eliminate operational bottlenecks and manual permission management. The previous model required defining hundreds of roles, permissions, and group mappings in an internal IAM service, leading to approval delays and stale memberships. Under the new architecture, user attributes sync from the HRMS and token payloads, while resource attributes are tagged upon creation or backfilled to reflect department and team ownership. Open Policy Agent evaluates access requests defined in Rego via middleware by comparing user attributes with resource metadata. This transition eliminated over 200 roles, 200 permissions, and roughly 3,000 unused IAM resources while automating access provisioning for new joiners. ### [Securing GitOps pipelines](https://yomu.fyi/post/securing-gitops-pipelines.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Thang Le - Published: Mar 1, 2023 Grab's real-time data platform team transitioned from an Atlantis-driven Terraform workflow to an in-house GitOps platform called Khone to manage streaming infrastructure resources like Kafka topics and Flink pipelines. The earlier setup suffered from coarse-grained access controls, required manual merge request comments, and lacked flexible validation capabilities within native configuration files. Khone derives environment parameters directly from standardized directory paths and uses Python with the python-hcl2 library to inspect and validate resource definitions before executing Terraform stages in parallel. To prevent configuration tampering in merge requests, CI/CD pipeline definitions and execution scripts are isolated in a separate administrative repository and fetched during job runs using shallow Git clones. ### [New zoom freezing feature for Geohash plugin](https://yomu.fyi/post/new-zoom-freezing-feature-for-geohash-plugin.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Maria Mitisor - Published: Feb 21, 2023 Grab introduced a zoom freeze capability to its Geohash plugin for the Java OpenStreetMap Editor (JOSM). Previously, zooming in or out forced the plugin to calculate and render new geohashes automatically, causing visual clutter when users needed to focus on a specific region. The updated interface introduces a toggle label allowing editors to switch between enabling and disabling zoom freezing. While frozen, the current geohash boundaries remain locked during map scaling, and options to show smaller or larger geohashes are disabled. Disabling the feature instantly updates the display with the appropriate geohash subdivisions for the current zoom level. ### [Catching Anomalies Early in Mobile App Releases](https://yomu.fyi/post/catching-anomalies-early-in-mobile-app-releases.md) - Company: [Nextdoor](https://yomu.fyi/company/nextdoor.md) - Author: Walt Leung - Published: Jan 11, 2023 Nextdoor deploys weekly mobile updates across iOS and Android using phased rollouts starting at 1% adoption to minimize blast radius. Standard aggregate observability cannot reliably detect early regressions because early adopters skew significantly more active than average users, obscuring silent drops in overall metric noise. To overcome this selection bias, Nextdoor built App Release Anomaly Detection using difference-in-differences causal inference. The methodology verifies pre-adoption trends with standard deviation bounds and fits a linear regression model to estimate the effect against an unobserved counterfactual. During a rollout of iOS version v1.234.5, this model identified a statistically significant app session decline at 1% adoption, isolating the regression 10 days earlier than week-over-week metrics. ### [Building Obsidian, Tinder’s Design System](https://yomu.fyi/post/building-obsidian-tinder-s-design-system.md) - Company: [Tinder](https://yomu.fyi/company/tinder.md) - Author: Tinder - Published: Jan 9, 2023 As Tinder scaled over a decade, maintaining visual consistency across its client applications became unsustainable and increased engineering overhead. To address this, the UI Platform team developed Obsidian, a unified design system combining design tokens, reusable components, and documentation. Designers define base and context tokens in Figma using the Tokens Studio plugin, which are then transformed by Style Dictionary into native artifacts for iOS, Android, and Web. Web developers integrate these tokens using Atomizer to generate static stylesheets, while a Zeroheight documentation portal and a custom token lookup tool assist legacy migrations. The ongoing migration has streamlined collaboration between designers and engineers, resolved styling ambiguities, and improved color contrast accessibility across platforms. ### [Graph service platform](https://yomu.fyi/post/graph-service-platform.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Wenxiang Lu - Published: Jan 5, 2023 Grab's GrabDefence team required a dedicated graph infrastructure to proactively identify mobile fraud patterns, such as multiple accounts operating on shared physical devices and suspicious financial loops. To address this, the team built a four-layer Platform as a Service that encapsulates graph database operations behind uniform RESTful APIs for OLTP search and OLAP analysis. The architecture utilizes Amazon S3 for raw data files, Amazon Neptune for graph storage, DynamoDB for schema and metadata configurations, and Kafka for streaming ingestion. When users trigger data loading tasks, the service validates entity attributes against schemas stored in DynamoDB before importing records into Neptune. This infrastructure allows investigators to traverse adjacent account IDs and visualize complex entity relationships without managing underlying database runtimes. ### [Zero trust with Kafka](https://yomu.fyi/post/zero-trust-with-kafka.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Fabrice Harbulot - Published: Dec 7, 2022 Grab's real-time data platform team transitioned their large-scale Kafka infrastructure from basic network access controls to a zero-trust architecture. The platform implements mutual Transport Layer Security (mTLS) for offline peer authentication and encryption, driven by HashiCorp Vault's PKI engine and Strimzi on Kubernetes. Policy-Based Access Control is enforced using dedicated Open Policy Agent deployments per cluster, backed by GitOps workflows where topic owners approve JSON authorization rules. To simplify client integration, the team enhanced their Go SDK to handle ephemeral in-memory certificates, automatic renewals, and configurable retries. While the security posture improved, the Java encryption and decryption overhead caused a drop in streaming throughput. ### [How KartaCam powers GrabMaps](https://yomu.fyi/post/how-kartacam-powers-grabmaps.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Shuangquan Hou - Published: Dec 1, 2022 Grab's Geo team required an efficient, low-cost way to collect fresh street-level imagery across Southeast Asia, where professional mapping equipment is prohibitively expensive and smartphone crowdsourcing yields inconsistent quality. To address this, Grab built KartaCam, a custom mapping device equipped with a 12MP sensor, dual-band GNSS, 4G LTE, and onboard edge AI. Edge machine learning models evaluate scene suitability, check image quality, filter for map-relevant objects, and blur personal data directly on the device prior to upload. Deploying a four-camera KartaCam 360 array delivers panoramic coverage and point-of-interest data comparable to commercial mapping rigs at roughly one-twentieth the hardware cost. ### [Graph for fraud detection](https://yomu.fyi/post/graph-for-fraud-detection.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Min Chen - Published: Nov 24, 2022 Rapid expansion across multiple business verticals introduced diverse, evolving fraud patterns at Grab, making traditional rule engines and decision trees inadequate due to their heavy reliance on extensive feature engineering and labeled data. To address cold starts and detect unknown fraud, the engineering team implemented a semi-supervised Relational Graph Convolutional Network (RGCN) across millions of connected ecosystem entities. The model propagates information along neighborhood nodes through graph convolutional layers to produce node embeddings and output fraud probabilities. Trained on graphs where only a small percentage of nodes are labeled, the RGCN achieved an AUROC close to 1 and distinctly separated fraudulent embeddings from genuine ones. The approach also offers high explainability by surfacing dense clusters of shared physical devices and addresses ongoing challenges in real-time prediction and noisy connections. ### [Query expansion based on user behaviour](https://yomu.fyi/post/query-expansion-based-on-user-behaviour.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Shuailong Liang - Published: Nov 16, 2022 Grab developed a query expansion framework to resolve search issues such as typos, multi-language terms, missing alternatives, and word spacing. The corpus is generated through manual annotation for the top thousand fat-head queries and session-based data mining of user rewrite patterns. Offline pipelines extract adjacent query rewrites occurring within 30 seconds where only the second query generated a click, limiting candidates to three per query for serving efficiency. In production, expanded keywords pass through a query understanding, multi-recall, and result fusion flow with automated pruning of underperforming pairs. Online A/B testing across six Southeast Asian countries showed improvements in click-through and conversion rates, with data-mined expansions outperforming manual annotations alone. ### [How to Categorize and Prevent Risks of Sensitive Links in URLScan](https://yomu.fyi/post/how-to-categorize-and-prevent-risks-of-sensitive-links-in-urlscan.md) - Company: [Tinder](https://yomu.fyi/company/tinder.md) - Author: Tinder - Published: Nov 7, 2022 Security integrations with URLScan can accidentally index sensitive links when API calls are executed with public visibility. Tinder Security Labs constructed targeted queries to uncover exposed links across file-sharing services, enterprise ticketing platforms, and single sign-on portals. Investigations revealed that exposed links included document magic links containing contracts and bank details, as well as active new-hire registration links that allowed attackers to complete registration workflows and configure two-factor authentication. In one SSO query alone, researchers discovered 1,300 indexed invitation links, with 20 to 50 new links appearing daily. The researchers collaborated with URLScan to purge the exposed indexes from public searches and advised organizations to configure security scanning tools to use private or unlisted visibility settings. ### [Using mobile sensor data to encourage safer driving](https://yomu.fyi/post/using-mobile-sensor-data-to-encourage-safer-driving.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Laiyi Lin - Published: Oct 25, 2022 Dedicated vehicular inertial measurement units are prohibitively expensive to install across large fleets, while customer feedback lacks the coverage required to assess driving safety comprehensively. To monitor driver behavior at scale, Grab processes smartphone sensor data collected via a mobile telematics SDK at high sampling frequencies. A custom orientation algorithm calculates Euler roll and pitch angles from gravity and resolves yaw through principal component analysis to align phone measurements with vehicle axes. The pipeline applies low-pass or Kalman filtering to handle varying noise levels, followed by threshold peak-finding algorithms to detect harsh acceleration, braking, and cornering. Speeding events are identified by comparing GPS speeds against known map limits or aggregated speed profiles derived from free-flowing traffic. ### [Automatic rule backtesting with large quantities of data](https://yomu.fyi/post/automatic-rule-backtesting-with-large-quantities-of-data.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Chao Wang - Published: Sep 8, 2022 Evaluating new or modified risk rules previously required Grab analysts to run slow offline Presto queries, manually construct payloads, or run rules in shadow mode for days. To standardize and accelerate this workflow, Grab developed an automated backtesting system powered by an AWS EMR Spark pipeline. Historical events are continuously ingested via Kafka and a Kubernetes stream pipeline into S3 using Snappy-compressed Parquet. Users configure replay intervals and rule definitions directly in the rule engine UI, which triggers asynchronous Spark jobs through Amazon SQS and Lambda. This automated simulation replaces multi-week shadow mode runs and generates downloadable aggregation metrics on transactions, user counts, and treatment outcomes. ### [Supabase is SOC2 compliant](https://yomu.fyi/post/supabase-is-soc2-compliant.md) - Company: [Supabase](https://yomu.fyi/company/supabase.md) - Author: inian,joel - Published: Aug 17, 2022 Supabase reports achieving SOC2 Type 1 compliance and later updates the announcement to say it is SOC2 Type 2 compliant. The post explains SOC2’s five Trust Services Criteria and distinguishes Type 1’s point-in-time audit from Type 2’s observation period, while framing customer expectations and data stewardship as the main reasons for pursuing certification. Supabase used Vanta to monitor controls and collect evidence, selected an auditor familiar with SaaS companies and Vanta, and formalized practices covering device security, access control, policies, logging, monitoring, vendor reviews, and repository protection. The process still required substantial manual evidence, and the company found that audit scope and auditor expectations mattered more than achieving a perfect automated-tool score. Supabase also launched a security center and identified HIPAA certification as a next step. ### [supabase-js v2](https://yomu.fyi/post/supabase-js-v2.md) - Company: [Supabase](https://yomu.fyi/company/supabase.md) - Author: inian,alaister - Published: Aug 16, 2022 Supabase announces supabase-js v2, a release focused on developer quality-of-life improvements and designed around incremental changes rather than a large rewrite. Its type system uses CLI-generated database definitions at the client level, while authentication adds explicit signInWithPassword() and signInWithOtp() methods, makes most Auth operations asynchronous, and moves server-side methods under auth.admin. Data mutations return minimal by default unless select() is appended, constructor options are scoped by library, and Edge Functions automatically detect request and response content types. The release also introduces typed errors, a channel() interface for presence and PostgreSQL change subscriptions, and a preparatory path toward multiplayer features; older realtime syntax remains supported for now. Developers can install v2, consult updated documentation and migration guidance, while v1 continues receiving security fixes and maintenance patches for three months. ### [Supabase CLI v1 and Management API Beta](https://yomu.fyi/post/supabase-cli-v1-and-management-api-beta.md) - Company: [Supabase](https://yomu.fyi/company/supabase.md) - Author: soedirgo,qiao - Published: Aug 15, 2022 Supabase CLI v1 is moving out of beta with capabilities for managing database migrations and generating TypeScript types, alongside a beta Management API. The CLI can manage organizations and projects, link projects locally, run schema and manual migration workflows, support CI/CD through local and GitHub Actions workflows, and generate types from local or remote databases. Migration changes include improved manually written migrations, an experimental migra-backed db diff that is faster and produces more concise DDL, and local branching for testing DDL without data loss; the source notes that generated scripts are not perfect. The REST-based Management API, previously called the Admin API, is consumed by the CLI and exposes programmatic management of organizations, projects, Edge Functions, and more, with endpoints intended for CI/CD and test-environment creation. ### [How we store and process millions of orders daily](https://yomu.fyi/post/how-we-store-and-process-millions-of-orders-daily.md) - Company: [Grab](https://yomu.fyi/company/grab.md) - Author: Xi Chen - Published: Aug 15, 2022 The Grab Order Platform processes millions of food and mart transactions daily, requiring high throughput, fault tolerance, and reduced cloud costs across transactional and analytical workloads. To meet these demands, the engineering team decoupled their database architecture by using Amazon DynamoDB for critical OLTP queries and MySQL RDS for historical OLAP queries. DynamoDB handles online order lifecycles with strong consistency, utilizing sparse Global Secondary Indexes for ongoing orders and TTL configurations to limit storage growth. Updates propagate asynchronously to MySQL RDS through a Kafka ingestion pipeline backed by Amazon SQS retries and timestamp-based version checks. This dual-database approach isolated core transaction availability from analytical queries and delivered significant cloud cost savings. ### [Supabase Series B](https://yomu.fyi/post/supabase-series-b.md) - Company: [Supabase](https://yomu.fyi/company/supabase.md) - Author: Paul Copplestone - Published: Aug 12, 2022 Supabase announced an $80 million Series B in May, bringing total funding to $116 million, and reported growth to more than 150,000 hosted PostgreSQL databases and 110,000 developers. Community metrics also increased, with Discord membership rising from 4,000 to 8,000, Twitter followers from 18,000 to 37,000, and GitHub stars from 19,000 to 36,000. The round is led by Felicis, with Coatue, Lightspeed, and Square Peg Capital joining, while Supabase describes its open-source support through contributor payments, maintainer hiring, sponsorship, and community collaboration. The company is moving into its third phase, focused on cloud-native PostgreSQL capabilities including branching, scalable storage, distribution, ephemeral compute, snapshots, and time travel. It invested $500,000 in OrioleDB and hired a developer to work on it, while stating that OrioleDB is not currently used on the Supabase platform and that the company will not run a PostgreSQL fork. ### [Launch Week 5 Hackathon](https://yomu.fyi/post/launch-week-5-hackathon.md) - Company: [Supabase](https://yomu.fyi/company/supabase.md) - Author: Ant Wilson - Published: Aug 10, 2022 Supabase’s Launch Week 5 Hackathon invites individuals and teams of up to five to build a new open-source project using Supabase during a 10-day event. It starts at 8:00am PT on Friday 12 August 2022, with submissions due by 11:59pm PT on Sunday 21 August; projects may use any language or framework and must be created during the event. The best overall project receives $1,500 in GitHub sponsorship paid as $500 per month for three months, while the listed category prizes include Gold Supabase Caps, with winner and runner-up prizes and caps for every winning team member. Supabase-js v2.0 and Flutter SDK v1.0 are announced during Launch Week, and using them earns extra credit; the supabase-js release candidate is installable with npm install @supabase/supabase-js@rc. ### [Slack Consolidate: a slackbot built with Python and Supabase](https://yomu.fyi/post/slack-consolidate-a-slackbot-built-with-python-and-supabase.md) - Company: [Supabase](https://yomu.fyi/company/supabase.md) - Author: Rodrigo Mansueli - Published: Aug 9, 2022 Slack Consolidate is a Python Slackbot that watches configured Slack channels and reposts messages to destination channels according to routing rules stored in Supabase. The tutorial sets up Slack app permissions from a manifest, installs the Python SDKs, configures environment variables for Supabase and Slack tokens, and creates slack\_channels and slack\_watcher PostgreSQL tables. Its polling loop loads channel mappings, fetches up to 20 recent messages, converts Slack timestamps to ISO-compatible values, inserts messages into the watcher table, builds message links, and posts through a bot client; it waits 1.4 seconds between channel processing and rescans channel configuration hourly. Public-channel posts contain a link, while private-channel posts include the message text as well. The project also provides a Google Colab notebook for trying the example. [Newer posts](https://yomu.fyi/page/39.md) · [Older posts](https://yomu.fyi/page/41.md)