# 2.3x faster using the Go plugin to replace Lua virtual machine

[Grab](https://yomu.fyi/company/grab) · Yonghao Hu · May 15, 2023

**Type:** Problem & solution

## Summary

Talaria, an open-source distributed time-series database developed at Grab, previously allowed users to run custom data transformation scripts during ingestion using a Lua virtual machine. Launching and executing Lua scripts caused significant performance overhead when processing large volumes of events. To resolve this bottleneck, the team replaced the Lua VM with Go plugins compiled as Linux shared libraries (.so files). Benchmarks revealed that calling Go plugins achieves performance on par with native Go functions, executing roughly 2.3 times faster and consuming 2.3 times less memory than cached Lua VMs. Both execution methods conform to a unified Handler interface to load and run custom transformations.

## Context

Executing custom data processing scripts via Lua virtual machines during high-volume event ingestion introduced significant performance and memory overhead in Talaria.

## Approach / What changed

Replaced Lua VM execution with Go plugins compiled into Linux shared libraries (.so files) implementing a common Handler interface.

## Takeaways

- Go plugin calls perform almost identically to native Go functions, running approximately 2.3 times faster and using 2.3 times less memory than cached Lua VM executions.
- Go plugins are compiled using the -buildmode=plugin flag, loaded via plugin.Open, and mapped to execution targets with the Lookup method.
- The Go version used to compile a plugin must precisely match the Go version of the host application, which can be managed using Docker builds.

**Tags:** [Go](https://yomu.fyi/topic/go), [Open Source](https://yomu.fyi/topic/open-source), [Performance](https://yomu.fyi/topic/performance)

[Read original post](https://engineering.grab.com/faster-using-the-go-plugin-to-replace-Lua-VM)