Editorial: Improve specification of [RegExp]IdentifierNames

[jmdyck]

Currently, the algorithm for StringValue of IdentifierName has the step:

Let _idTextUnescaped_ be the result of
  replacing any occurrences of `\\` |UnicodeEscapeSequence| in _idText_
  with the code point represented by the |UnicodeEscapeSequence|.

(And the algorithm for CapturingGroupName of RegExpIdentifierName has something quite similar.)

In #1552 (comment), @michaelficarra said of this step:

This language is a little less precise than I would prefer. Could we define a syntax-directed operation [to specify "the code point represented by the |UnicodeEscapeSequence|"]

This PR does so.

The first commit refactors the grammar slightly, and the second introduces 4 SDOs. The first makes the second easier, but it also has benefits of its own. I think they're easier to understand as separate commits.

The third commit just moves some pre-existing prose. The connection between the prose and the Early Error rules + SDOs could maybe be improved, but that wasn't my focus.

Note that the second commit uses list-concatenation in two spots, so if PR #2382 isn't accepted, I'll have to do something else there.

[jmdyck]

(force-pushed to rebase to master + resolve merge conflicts from #2411)

[michaelficarra]

I think you should combine the IdentifierCodePoint(s) SDOs with their RegExp variants. Then you could also take advantage of the chain rule to eliminate some of the definitions currently in RegExpIdentifierCodePoint.

[michaelficarra]

LGTM otherwise. This is a big improvement IMO.

[jmdyck]

I think you should combine the IdentifierCodePoint(s) SDOs with their RegExp variants.

Yeah, there are some parallels between the two, so it's tempting to look for ways to combine them. But I'm doubtful it would be a net benefit.

E.g., consider [RegExp]IdentifierCodePoints:

• the algorithm for IdentifierName :: IdentifierStart refers to IdentifierStart, and
• the algorithm for RegExpIdentifierName :: RegExpIdentifierStart refers to RegExpIdentifierStart,

so the two rules can't be combined; the best you can do is put them next to each other.

With [RegExp]IdentifierCodePoint, it's slightly better: a couple of the algorithms are the same, so you could join the <emu-grammar>s (e.g., the ones for IdentifierStart :: IdentifierStartChar and RegExpIdentifierStart :: IdentifierStartChar). However, with the other rules, you'd again just be moving them closer together.

I'm not saying there's no benefit to moving two similar rules close together -- if, in future, you need to change one, there's a fair chance you need to change the other as well, and you're more likely to recognize that if they're right next to each other. On the other hand, in this case, it seems like a low probability they'll need to change. Moreover, bringing them together means that at least one of them is taken from its 'home', which is a tradeoff.

(The above assumes you're not talking about merging productions (changing the grammar). That would allow more merging of SDOs, but it would have problems of its own.)

Then you could also take advantage of the chain rule to eliminate some of the definitions currently in RegExpIdentifierCodePoint.

Hm, I don't follow. Can you give an example?

[michaelficarra]

@jmdyck Looks like I got IdentifierStart and IdentifierStartChar mixed up in some of the definitions of (RegExp)IdentifierCodePoint and thought the chain rule would apply. After looking again, I don't see it.

If there's no benefit from the chain rule applying, I agree with your point about moving the relevant definitions of the SDO further from where it belongs. At that point, the minor benefit of reducing or grouping together the mostly-duplicated algorithms isn't enough to warrant the change.