EntityParser can't handle encoded emoji #67

aduth · 2018-11-15T15:36:19Z

While the tokenizer will gracefully decode most encoded characters:

⇒ node
> var Tokenizer = require( 'simple-html-tokenizer' );
undefined
> Tokenizer.tokenize( '&amp;' )[ 0 ].chars === '&'
true

It doesn't handle characters whose encodings exceed 16 bits (e.g. emoji):

⇒ node
> var Tokenizer = require( 'simple-html-tokenizer' );
undefined
> Tokenizer.tokenize( '&#128517;' )[ 0 ].chars === '😅'
false

It may be that EntityParser should use String.fromCodePoint in place of String.fromCharCode instead, or an equivalent polyfill?

rwjblue · 2018-11-20T15:21:06Z

Seems reasonable to me...

@krisselden what do you think?

krisselden · 2018-11-21T13:09:23Z

Yes

locks assigned chadhietala and wycats Nov 20, 2018

CvX linked a pull request Jul 1, 2019 that will close this issue

Handle encoded emoji #70

Draft

Provide feedback