Implement Lua 5.4 to-be-closed

[arnodel]

To-be-closed variables are a new feature of Lua 5.4: https://www.lua.org/manual/5.4/manual.html#3.3.8

See added tests in locals.lua for working examples and thread.lua for coroutine tests, including the use of coroutine.close().

The feature is under-specified somewhat in the Lua docs.

Need to support

• errors
• coroutines (that includes implementing coroutine.close() which allows closing a suspended coroutine without completing it, cleaning up all pending to-be-closed variables)

Approach

I try to give a succinct explanation of the implementation approach.

Height of a lexical scope

During AST -> IR compilation, a height is assigned to each lexical scope. If L2 is a lexical scope with parent L1, then

height(L2) = height(L1) + n, where n is the number of to-be-closed variables that L2 defines

If L is a root lexical scope (i.e. it is the outermost block in a function body) then

height(L) = 0

Code generation

There are 2 new opcodes:

• clpush <reg>: push the contents of <reg> onto the close stack
• cltrunc h: truncate the close stack to height h (h is encoded in the opcode and must be known at compile time)

This is how the opcodes are inserted

• Whenever a to-be-closed varable is defined (local x <close> = val), a clpush r1 instruction is emitted, where r1 is the register containing val.
• Whenever a lexical scope L is exited, a cltrunc h instruction is emitted where h is the height of the parent scope of L
• Whenever a Jump is emitted, just before the jump is emitted a cltrunc h instruction is emitted where h is the height of the lexical scope of the jump destination.

Runtime

There are two aspects to the runtime machinery - normal execution of Lua continutaions and error handling

Normal execution of Lua continuations

Each Lua continuation maintains a close stack, which is a stack of Lua values. It starts empty and is modified as follows

• clpush <reg> pushes the value of <reg> on top of the close stack
• cltrunc h pops values from the top of the close stack and executes their __close metamethod until the height of the close stack is at most h.
• When returning from a Lua function (return OR tail-call), a cltrunc 0 instruction is executed.

Error handling

If there is an error, then all close stacks in the "call stack" should be called until the error is caught. This is achieve by adding a Cleanup() method to the runtime.Cont interface, which must be implemented by each continuation type. The runtime loop will call those in turn down the call stack when an error is encountered. For a Lua continuation, Cleanup() is roughly equivalent to cltrunc 0.

[codecov]

Codecov Report

Merging #45 (620cdfd) into lua5.4 (f13d22f) will increase coverage by 0.26%.
The diff coverage is 89.34%.

@@            Coverage Diff             @@
##           lua5.4      #45      +/-   ##
==========================================
+ Coverage   86.08%   86.35%   +0.26%     
==========================================
  Files          90       90              
  Lines        9827     9920      +93     
==========================================
+ Hits         8460     8566     +106     
+ Misses       1076     1063      -13     
  Partials      291      291              

Impacted Files	Coverage Δ
runtime/cont.go	22.58% <ø> (ø)
runtime/thread.go	85.14% <80.76%> (+6.52%)	⬆️
runtime/luacont.go	90.05% <90.32%> (+0.82%)	⬆️
lib/coroutine/coroutine.go	85.24% <100.00%> (+2.22%)	⬆️
parsing/parser.go	92.29% <100.00%> (+0.62%)	⬆️
runtime/error.go	86.41% <100.00%> (+3.08%)	⬆️
runtime/gocont.go	92.20% <100.00%> (+0.10%)	⬆️
runtime/termination.go	78.46% <100.00%> (+3.85%)	⬆️
runtime/value.go	86.91% <0.00%> (+0.62%)	⬆️
... and 4 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f13d22f...620cdfd. Read the comment docs.

[cderici]

This looks great to me so far!