How We Improved Agent Chat Efficiency with Machine Learning

Summary

Agent typing time represented a large portion of Grab's chat support journey, and 85% of messages were still free typed because agents customized static templates to fit their personal style. To accelerate typing across multilingual markets without robotic templates, Grab built SmartChat, a machine learning feature that provides contextual sentence completion. The team opted for a lightweight seq2seq architecture using single-layered GRU encoder-decoders in TensorFlow instead of bulky attention models to keep model latency under 100ms. The user interface was implemented in React using a content-editable div with inline typeahead suggestions activated via keyboard shortcuts.

Takeaways

1. 1

   Presenting the top three suggestions slowed agents down because reading through unhelpful options created a distraction compared to a single inline recommendation.

2. 2

   To achieve an overall UI refresh rate under the 200ms human reaction threshold, the machine learning model had to generate predictions in under 100ms.

3. 3

   Attention-based seq2seq models were avoided in favor of a single-layer GRU recurrent neural network to minimize computational overhead and latency.