ksh93: "$*" joins positional parameters on the first byte of $IFS instead of first character

[stephane-chazelas]

$ ksh -c 'IFS=é; set : :; echo "$*"' | hd
00000000  3a c3 3a 0a                                       |:.:.|
00000004
$ echo é | hd
00000000  c3 a9 0a                                          |...|
00000003
$ locale charmap
UTF-8

Expected :é: (3a c3 a9 3a 0a)

POSIX says it must be the first character not byte http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_05_02

bash and zsh use the first character.

[McDutchie]

On the release version (2012-08-01), expanding "$*" while IFS contains a UTF-8 character corrupts ksh93's shell-quoting mechanism, so that anything that produces output suitable for re-entry into the shell has the final quote missing -- even after IFS is restored.

This appears to be fixed on the current beta (2014-12-24), at least as compiled on Linux.

On the release version, the behaviour is as follows:

$ ksh -c 'i=$IFS; IFS=é; set : :; echo "$*"; IFS=$i; trap "echo end" EXIT; trap'
:?:
trap -- 'echo end EXIT
end

Note the missing quote after trap -- 'echo end. You get the same with the output of export -p, alias, etc.

Even using a subshell doesn't avoid the corruption (yay non-forking subshells). However, setting and then unsetting LC_ALL=C seems to be an effective workaround:

$ ksh -c 'IFS=é; set : :; echo "$*"; trap "echo end" EXIT; LC_ALL=C; unset LC_ALL; trap'
:?:
trap -- 'echo end' EXIT
end