# How We Improved Agent Chat Efficiency with Machine Learning

[Grab](https://yomu.fyi/company/grab) · Suman Anand · Apr 19, 2021

**Type:** Problem & solution

## Summary

Agent typing time represented a large portion of Grab's chat support journey, and 85% of messages were still free typed because agents customized static templates to fit their personal style. To accelerate typing across multilingual markets without robotic templates, Grab built SmartChat, a machine learning feature that provides contextual sentence completion. The team opted for a lightweight seq2seq architecture using single-layered GRU encoder-decoders in TensorFlow instead of bulky attention models to keep model latency under 100ms. The user interface was implemented in React using a content-editable div with inline typeahead suggestions activated via keyboard shortcuts.

## Context

Agent typing time represented a large portion of the chat support journey, and 85% of messages were still free typed because support agents preferred personalized writing styles over rigid canned templates.

## Approach / What changed

Grab built SmartChat, an internal machine learning model using a single-layer GRU encoder-decoder seq2seq architecture with contextual features (past messages, user type, entrypoint, time of day) integrated into a React content-editable chat input using a Smart Compose-style interaction.

## Takeaways

- Presenting the top three suggestions slowed agents down because reading through unhelpful options created a distraction compared to a single inline recommendation.
- To achieve an overall UI refresh rate under the 200ms human reaction threshold, the machine learning model had to generate predictions in under 100ms.
- Attention-based seq2seq models were avoided in favor of a single-layer GRU recurrent neural network to minimize computational overhead and latency.

**Tags:** [Architecture](https://yomu.fyi/topic/architecture), [Developer Experience](https://yomu.fyi/topic/developer-experience), [Machine Learning](https://yomu.fyi/topic/machine-learning), [Microservices](https://yomu.fyi/topic/microservices), [Performance](https://yomu.fyi/topic/performance)

[Read original post](https://engineering.grab.com/how-we-improved-agent-chat-efficiency-with-ml)