Fix building without ICU #14489

TimothyGu · 2017-07-26T01:08:29Z

Corresponding documentation changes for encoding will be folded into #14486.

Checklist

make -j4 test (UNIX), or vcbuild test (Windows) passes
tests and/or benchmarks are included
commit message follows commit guidelines

Affected core subsystem(s)

encoding, test

jasnell · 2017-07-27T17:39:38Z

Argh... we seriously need to get building without ICU enabled by default as part of the normal CI run.

bnoordhuis

LGTM. Only lightly reviewed, mostly rubber-stamp.

@jasnell That's nodejs/build#419.

TimothyGu · 2017-07-30T01:23:06Z

@jasnell I assume your comment was a LGTM?

TimothyGu · 2017-07-30T11:06:18Z

I modified this PR to offer a TextDecoder with reduced functionalities even when built without ICU. @jasnell @bnoordhuis please review.

CI: https://ci.nodejs.org/job/node-test-pull-request/9407/

jasnell · 2017-07-31T19:05:04Z

doc/api/util.md


 * `encoding` {string} Identifies the `encoding` that this `TextDecoder` instance
  supports. Defaults to `'utf-8'`.
 * `options` {Object}
  * `fatal` {boolean} `true` if decoding failures are fatal. Defaults to
-    `false`.
+    `false`. This options is only supported when ICU is enabled (see


s/options/option

jasnell · 2017-07-31T19:07:09Z

test/parallel/test-readline-position.js

@@ -14,16 +14,23 @@ const ctrlU = { ctrl: true, name: 'u' };
    prompt: ''
  });

-  for (const [cursor, string] of [
+  const tests = [


This change does not look related?

This is necessary to fix building (and testing) on ICU, since the JS implementation can only distinguish between wide/narrow.

jasnell

Code change looks good, tho I'd prefer a bit less code duplication on the two TextDecoder variants.

refack

Looks good.
I've hit that "ICU missing" wall several times... good riddance.

refack · 2017-08-01T02:47:17Z

test/parallel/test-whatwg-encoding-textencoder.js

+
+{
+  const fn = TextEncoder.prototype[inspect];
+  fn.call(new TextEncoder(), Infinity, {});


Don't you want to assert anything about the output string? Or at least wrap in assert.doesNotThrow just to be explicit.

Wrapped in doesNotThrow. We don't really care about the inspection results as long as it works 😉

refack · 2017-08-01T02:48:17Z

test/parallel/test-whatwg-encoding-textencoder.js

+
+  [{}, [], true, 1, '', new TextDecoder()].forEach((i) => {
+    assert.throws(() => fn.call(i, Infinity, {}),
+                  common.expectsError({


Could you add the Type of the expected Error.

refack · 2017-08-01T02:49:24Z

test/parallel/test-whatwg-encoding-textdecoder.js

+
+{
+  const fn = TextDecoder.prototype[inspect];
+  fn.call(new TextDecoder(), Infinity, {});


Same comments as in test-whatwg-encoding-textencoder.js#L24 & L28

refack · 2017-08-01T02:51:41Z

test/parallel/test-whatwg-encoding-textdecoder.js

+                    code: 'ERR_ENCODING_INVALID_ENCODED_DATA',
+                    type: TypeError,
+                    message:
+                      /^The encoded data was not valid for encoding utf-8$/


You could use a simple string, should be semantically the same as an anchored RegExp.

refack · 2017-08-01T02:59:20Z

lib/internal/errors.js

@@ -118,6 +118,7 @@ E('ERR_HTTP_HEADERS_SENT',
 E('ERR_HTTP_INVALID_STATUS_CODE', 'Invalid status code: %s');
 E('ERR_HTTP_TRAILER_INVALID',
  'Trailers are invalid with this transfer encoding');
+E('ERR_ICU_NOT_SUPPORTED', '%s is not supported on Node.js built without ICU');


Could you add a test asserting the expected message, to test/parallel/test-internal-errors.js, as per the new tweaks in https://github.com/nodejs/node/blob/master/doc/guides/using-internal-errors.md#testing-new-errors

Thanks for the reminder, I realized that I also forgot to add the new error code to the docs...

Actually, the test file only contains more complicated error code processing, like ERR_INVALID_ARG_TYPE. I'll still add the missing docs though.

The reasoning was "test the message once then assume it's ok everywhere else" but, this will probably get reworked anyway after we finish migrating all the Error, so this is non-blocking.

obsolete

TimothyGu · 2017-08-02T00:38:39Z

@jasnell Changed so that there's less duplication.

Tried to kick off a CI w/o ICU: ~~https://ci.nodejs.org/job/node-test-commit/11502/~~ https://ci.nodejs.org/job/node-test-commit/11503/

Also split up the tests.

TimothyGu · 2017-08-02T04:23:22Z

After the V8 6.0 update, the V8 console is now installed on the global object regardless of whether inspector is actually enabled, and in the case of no-ICU inspector is always disabled, which leads to the test failures :(

Pushed one more fix to address this situation.

New no ICU CI: https://ci.nodejs.org/job/node-test-commit/11505/

TimothyGu · 2017-08-05T08:41:13Z

Landed in 34d1b11...17547c4.

PR-URL: #14489 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>

Also split up the tests. PR-URL: #14489 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>

PR-URL: #14489 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>

addaleax · 2017-08-10T16:34:28Z

This doesn’t land cleanly on 8.x; if you can, please follow the guide and raise a backport PR.

Note that the resulting package does not work yet due to nodejs/node#14489 not being in 8.3.0.

Also split up the tests. PR-URL: nodejs#14489 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>

TimothyGu · 2017-08-12T09:27:23Z

@addaleax Done: #14786. Only the second commit needs backporting, as the first and third commits have been backported already.

Also split up the tests. Backport-PR-URL: #14786 Backport-Reviewed-By: Anna Henningsen <[email protected]> PR-URL: #14489 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>

MylesBorins · 2017-08-16T08:35:37Z

Is this needed on v6.x?

MylesBorins · 2017-09-19T17:20:25Z

ping. this would need to be manually backported

TimothyGu added encoding Issues and PRs related to the TextEncoder and TextDecoder APIs. test Issues and PRs related to the tests. labels Jul 26, 2017

TimothyGu mentioned this pull request Jul 26, 2017

doc: cross-link util.TextDecoder and intl.md #14486

Closed

2 tasks

bnoordhuis previously approved these changes Jul 28, 2017

View reviewed changes

TimothyGu mentioned this pull request Jul 29, 2017

build: fix build break when omitting icu #14533

Closed

2 tasks

TimothyGu force-pushed the encoding-no-icu branch 4 times, most recently from 626437c to 67493e8 Compare July 30, 2017 11:04

addaleax added the i18n-api Issues and PRs related to the i18n implementation. label Jul 30, 2017

jasnell reviewed Jul 31, 2017

View reviewed changes

jasnell approved these changes Jul 31, 2017

View reviewed changes

refack reviewed Aug 1, 2017

View reviewed changes

TimothyGu force-pushed the encoding-no-icu branch from 67493e8 to fde1f4b Compare August 1, 2017 05:50

refack approved these changes Aug 1, 2017

View reviewed changes

TimothyGu force-pushed the encoding-no-icu branch from d2b9f5d to ad06098 Compare August 2, 2017 00:18

TimothyGu added 2 commits August 2, 2017 09:33

test: fix test-readline-position w/o ICU

939818f

encoding: rudimentary TextDecoder support w/o ICU

e07d268

Also split up the tests.

TimothyGu force-pushed the encoding-no-icu branch from ad06098 to e07d268 Compare August 2, 2017 01:34

inspector: fix console with inspector disabled

27b5568

TimothyGu added the inspector Issues and PRs related to the V8 inspector protocol label Aug 2, 2017

TimothyGu closed this Aug 5, 2017

TimothyGu deleted the encoding-no-icu branch August 5, 2017 08:41

TimothyGu added a commit that referenced this pull request Aug 5, 2017

test: fix test-readline-position w/o ICU

3b0ef0b

PR-URL: #14489 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>

TimothyGu added a commit that referenced this pull request Aug 5, 2017

encoding: rudimentary TextDecoder support w/o ICU

365b2e3

Also split up the tests. PR-URL: #14489 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>

TimothyGu added a commit that referenced this pull request Aug 5, 2017

inspector: fix console with inspector disabled

17547c4

PR-URL: #14489 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>

addaleax pushed a commit that referenced this pull request Aug 7, 2017

test: fix test-readline-position w/o ICU

28b9c7a

PR-URL: #14489 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>

addaleax pushed a commit that referenced this pull request Aug 7, 2017

inspector: fix console with inspector disabled

deea68c

PR-URL: #14489 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>

addaleax added the backport-requested-v8.x label Aug 10, 2017

fornwall added a commit to termux/termux-packages that referenced this pull request Aug 11, 2017

nodejs-current: Update from 8.2.1 to 8.3.0

552d700

Note that the resulting package does not work yet due to nodejs/node#14489 not being in 8.3.0.

TimothyGu added a commit to TimothyGu/node that referenced this pull request Aug 12, 2017

encoding: rudimentary TextDecoder support w/o ICU

2ab406e

Also split up the tests. PR-URL: nodejs#14489 Reviewed-By: James M Snell <[email protected]> Reviewed-By: Refael Ackermann <[email protected]>

TimothyGu mentioned this pull request Aug 12, 2017

[v8.x backport] encoding: rudimentary TextDecoder support w/o ICU #14786

Closed

4 tasks

addaleax added backported-to-v8.x and removed backport-requested-v8.x labels Aug 12, 2017

addaleax mentioned this pull request Aug 13, 2017

v8.4.0 proposal #14811

Merged

MylesBorins added the lts-watch-v6.x label Aug 16, 2017

MylesBorins added backport-requested-v6.x and removed lts-watch-v6.x labels Sep 19, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix building without ICU #14489

Fix building without ICU #14489

TimothyGu commented Jul 26, 2017

jasnell commented Jul 27, 2017

bnoordhuis left a comment

TimothyGu commented Jul 30, 2017

TimothyGu commented Jul 30, 2017

jasnell Jul 31, 2017

jasnell Jul 31, 2017

TimothyGu Aug 1, 2017

jasnell left a comment

refack left a comment

refack Aug 1, 2017

TimothyGu Aug 1, 2017

refack Aug 1, 2017

refack Aug 1, 2017

refack Aug 1, 2017

refack Aug 1, 2017

TimothyGu Aug 1, 2017

TimothyGu Aug 1, 2017

refack Aug 1, 2017

TimothyGu commented Aug 2, 2017 •

edited

Loading

TimothyGu commented Aug 2, 2017 •

edited

Loading

TimothyGu commented Aug 5, 2017

addaleax commented Aug 10, 2017

TimothyGu commented Aug 12, 2017

MylesBorins commented Aug 16, 2017

MylesBorins commented Sep 19, 2017

Fix building without ICU #14489

Fix building without ICU #14489

Conversation

TimothyGu commented Jul 26, 2017

Checklist

Affected core subsystem(s)

jasnell commented Jul 27, 2017

bnoordhuis left a comment

Choose a reason for hiding this comment

TimothyGu commented Jul 30, 2017

TimothyGu commented Jul 30, 2017

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jasnell left a comment

Choose a reason for hiding this comment

refack left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TimothyGu commented Aug 2, 2017 • edited Loading

TimothyGu commented Aug 2, 2017 • edited Loading

TimothyGu commented Aug 5, 2017

addaleax commented Aug 10, 2017

TimothyGu commented Aug 12, 2017

MylesBorins commented Aug 16, 2017

MylesBorins commented Sep 19, 2017

TimothyGu commented Aug 2, 2017 •

edited

Loading

TimothyGu commented Aug 2, 2017 •

edited

Loading