Switch IdentityEscape and Atom parsing behaviour to Annex B

[adrianheine]

parseIdentityEscape currently is broken since it passes a string to isIdentifierPart instead of a char code. This commit fixes that and switches to Annex B at the same time. There are some failing tests, though, and I'd like some help with them. I suspect there are some more hidden bugs somewhere else.

[jviereck]

@adrianheine, thanks for providing this. I can take a look over the weekend in case no one else jumps up.

[adrianheine]

That would be great! I also started working on the remainder of #90, but wanted to get this done first to minimize (my) confusion.

[jviereck]

Taking a closer look at this, I am not so sure yet what exactly the issue is. Here what I could figure out so far. Basically it boils down to the question what /[\c0]/ is supposed to match. This is a valid RegExp in Firefox and Chrome but I have no real clue what this should match right now.

Applying the current PR version and running npm test, this is the first error I got (after improving the output a bit):

Error: Failure parsing string

  [\c0001d-G]

Got:

{"type":"error","name":"SyntaxError","message":"classEscape at position 2
    [\\c0001d-G]
      ^","input":"[\\c0001d-G]"}

Expected:

{"type":"error","name":"SyntaxError","message":"invalid range in character class at position 7: d-G
    [\\c0001d-G]
           ^","input":"[\\c0001d-G]"}

Note how after applying this patch, the parsing error changes. Before, the error was due to the invalid range in /[d-G]/. Now, the parser fails at the /[\c0001]/ part.

From playing in the browser,

/[\c312]/.test('3') -> False
/[\c012]/.test('0') -> False
/[\c012]/.test('1') -> True
/[\c012]/.test('2') -> True

So, it seems the parsing behavior is like the \c3 etc get parsed into whatever and then the 1 and 2 are parsed as possible entries in the group.

Does this parsing behavior make sense to someone?

[mathiasbynens]

https://tc39.github.io/ecma262/#sec-regular-expression-patterns-semantics says:

The production ClassAtomNoDash :: \ [lookahead = c] evaluates as follows:

Return the CharSet containing the single character \ U+005C (REVERSE SOLIDUS).

NOTE: This production can only be reached from the sequence \c within a character class where it is not followed by an acceptable control character.

[adrianheine]

Edit: never mind, I got it:

/\c0/.test("\u0010") // => false
/[\c0]/.test("\u0010") // => true
/\ca/.test("\u0001") // => true
/[\ca]/.test("\u0001") // => true

Also:

> /[\c]/.test("\\")
true
> /[\c]/.test("c")
true
> /\c/.test("\\c")
true

See tc39/ecma262@421a6ae and tc39/ecma262@f48dab5.

[adrianheine]

This is good to merge from my point of view.

[mathiasbynens]

@adrianheine You figured it out. FWIW, I have a table here: https://mathiasbynens.be/notes/javascript-escapes#control