Implementation of the local builtin with support for dynamic local scopes

[JohnoKing]

This is the eventual result of on/off work on the local builtin planned for inclusion in 1.1. (A declare builtin has also been added because it's a simple addtion when local is present.) Initially a port of ksh93v-'s code, this builtin was rewritten to implement local scoping in POSIX functions (i.e. name() { syntax). It's still experimental and may have bugs, but it's now stable enough to run the ksh and modernish regression test suites without failing.

Changes:

• Implemented local and declare builtins, which both default to using dynamic local scoping regardless of the type of function. local cannot be used outside of functions, unlike declare which has no such requirement.

• Added typeset -D, which allows typeset to use dynamic local scoping.

• Added typeset -P/local -P, which allows for the use of static local scoping in both POSIX and KornShell functions (the flag used will likely be changed).

• Completely overhauled POSIX function handling to run them from sh_funscope() instead of b_dot_cmd(). This made the implementation of local scoping much easier and also inadvertently fixed an out-of-bounds bug without adding a costly malloc call (cf. Array 'argv[0]' accessed at index -1, which is out of bounds att/ast#1423). (Note that KornShell functions (function name { syntax) that are executed via the . command will ignore scoping.) At present POSIX functions still use sh.dot_depth for measuring function depth.

• b_dot_cmd(): Removed DOTMAX, as it's just a duplicate of MAXDEPTH.

• b_dot_cmd(): Removed superfluous double setting of sh.st.lineno to 1.

• Removed the (surprisingly useless) sh.posix_fun pointer and implemented a more versatile sh.infunction variable for identifying the current running function type, alongside helper FUN_POSIX, FUN_KSH and FUN_KSHDOT macros.

• For scoping purposes, the NV_DYNAMIC and NV_STATSCOPE bitflags have been added (there's also NV_SCOPES, which is a macro for (NV_GLOBAL|NV_DYNAMIC|NV_STATSCOPE). np->nvflag was expanded to uint32_t with the help of the new NV_OPENMASK to allow it to store NV_DYNAMIC.

• Added some (incomplete) scoping documentation to typeset --man.

• Various updates to the regression tests, including the addition of a large set of scoping regression tests at ./src/cmd/ksh93/tests/local.sh. (The copyright date does accurately represent the age of that script. This pull request has been stuck in development hell for a LONG time.)

TODO List:

• Add documentation to the ksh93(1) man page, NEWS file, etc.
• Bug test on more platforms than just Linux x86_64 (there's some questionable alignment code for Namval_t that could cause bugs).
• Decide on whether POSIX functions should use sh.fn_depth, sh.dot_depth or something else. I'll just keep it as sh.dot_depth; changing it isn't worth the risk of introducing yet more bugs.
• Add regression tests for using invocation local scopes with POSIX functions (cf. `+=' operator picks up global value of variable inside function even when variable is declared locally without assigning a value #533).
• Decide how scoping should work in POSIX functions nested within KornShell functions, preferably making it backward compatible.
• I have recently realized the current -c flag conflicts with ksh93v-. The static scoping flag for POSIX functions should be changed to something else. Changed to -P.
• Fix Implementation of the local builtin with support for dynamic local scopes #703 (comment).

[dannyweldon]

local cannot be used inside of functions

Did you actually mean "local cannot be used outside of functions"?

[JohnoKing]

Whoops, that was a typo. It has been fixed.

[McDutchie]

Thank you very much for all your work on this! I will have to take some time to fully understand the code.

Unfortunately, I already found a bug in dynamic scoping. Test script:

fun1() { local foo=set_by_fun1; fun2; echo $foo; }
fun2() { foo=changed_by_fun2; }
fun1

Expected output: changed_by_fun2; actual output: set_by_fun1.

[JohnoKing]

That bug is caused by a limitation in the current scoping code, which interacts rather poorly with NV_NOSCOPE (the current code just copies the variable from the parent function and otherwise lacks a viewport to that scope). I'm considering reimplementing the ksh93v- sh.st.var_local pointer as a possible fix for that, but so far all of my attempts in that direction break static scoping in one way or another.

[ormaaj]

Dynamic local?! I must be dreaming. :D

edit: aah and even works in posix functions! nice.