GrabAlex ProkofievProblem & solution
Troubleshooting critical issues in the Grab app was hindered by incomplete user-journey data, because manually instrumenting hundreds of UI components left interaction gaps and delayed diagnosis. AutoTrack is a new SDK that records application state, user interactions, and UI screens in real time, creating an end-to-end session chronicle from launch through navigation. It tracks lifecycle transitions and launch mechanisms such as deeplinks and push notifications, while UIViewController, Activity, and Fragment lifecycle events identify the displayed screen. Screen metadata includes the full hierarchy as JSON; after personal information is redacted for GDPR/PDPA compliance, that data is used to train an AI model that generates an HTML representation of the screen and interactions. AutoTrack supports reproducing elusive bugs, automated issue assignment, UI test generation, and analytics-trigger validation, with future work adding network and performance signals, privacy guardrails, and automated triage.
Manual click-stream instrumentation depended on developers covering potentially hundreds of UI components. Missed interactions broke user journeys, while outdated or fragmented data made troubleshooting reactive, delayed Mean Time To Resolution, and increased operational costs and customer dissatisfaction.
AutoTrack unifies real-time application-state, user-interaction, and UI-screen tracking. It records lifecycle transitions and launch mechanisms, identifies screens through iOS and Android lifecycle events, captures screen hierarchy JSON with personal information redacted for GDPR/PDPA compliance, and uses that data to generate HTML representations through an AI model.
AutoTrack records application lifecycle states and launch mechanisms, including explicit launches, deeplinks, and push-notification launches.
UIViewController, Activity, and Fragment lifecycle events identify the active screen, while captured hierarchy JSON adds detailed UI context for analysis.
Recorded sessions can support bug reproduction, automatic issue routing, UI test generation, and validation of analytics event triggers.
Summaries are generated from the original by a language model. Check the original before relying on it.
See the idea
Android's on-device machine learning stack lets apps run inference, image recognition, and text processing locally without sending data to servers, reducing latency and protecting privacy.
Keep reading
The Good Stuff
Paid picks we use ourselves. Links carry a referral code — a subscription through one earns us a commission at no extra cost to you.
One useful email. Unsubscribe anytime.
