When is JIT Faster Than A Compiler?

Summary

JIT compilation can outperform ahead-of-time compilation or interpretation for highly dynamic languages such as Ruby by optimizing runtime behavior under assumptions that static approaches cannot safely make. Ruby permits redefining operators, equality, iteration, hashing, and other behavior at any time, even from a background thread, forcing an interpreter to check whether definitions changed and limiting AOT code to conservative behavior. YJIT addresses this by compiling frequently used methods with assumptions about current definitions, then invalidating and discarding compiled versions when those assumptions become false; later calls can run interpreted code until a new version is compiled. The post presents this speculative optimization and de-optimization model as the reason a JIT can be faster, while noting that YJIT had known bugs in Ruby 3.1 and was expected to be closer to production-ready for some uses in Ruby 3.2.

Context

Ruby's dynamic behavior allows operators, equality, iteration, hashing, and other definitions to be changed during execution, including from a background thread. An ahead-of-time compiler must produce code that remains correct for any later behavior, while an interpreter must continually check whether behavior has changed.

Approach / What changed

YJIT compiles frequently used Ruby methods at runtime using assumptions about the current meanings of operations and other redefinable behavior. When a change invalidates an assumption, YJIT discards the affected compiled methods through de-optimization; subsequent calls can use interpretation until a new compiled version is generated.

Takeaways

• YJIT can assume that rarely changed behavior, such as the meaning of +, remains stable while compiled code runs, allowing faster specialized machine code.
• When Ruby behavior is redefined, compiled methods based on the old definition are invalidated and discarded rather than continuing to execute incorrect code.
• A sufficiently sophisticated interpreter that pre-builds machine code for current assumptions and invalidates it when they change would effectively be a JIT.

Related reading