buffer: add Buffer.copyFrom(...) #46500

jasnell · 2023-02-04T22:04:14Z

This is a variation on Buffer.from(arrayBufferView) that copies the bytes and does not truncate when using a multibyte TypedArray. For instance,

Buffer.from(new Uint16Array([0xffff]))
<Buffer ff>

Buffer.copyBytesFrom(new Uint16Array([0xffff]))
<Buffer ff ff>

Open to suggestions on a better name.

Fixes: #43862

lib/buffer.js

doc/api/buffer.md

lib/buffer.js

doc/api/buffer.md

addaleax · 2023-02-05T00:39:08Z

Counter-suggestion: Mark Buffer as a legacy API? There's no good reason for not using standardized equivalents in 2023.

jasnell · 2023-02-05T00:59:35Z

Mark Buffer as a legacy API? There's no good reason for not using standardized equivalents in 2023.

As much as I do prefer standardized APIs, Buffer clearly is never ever going to decline in use and is being actively implemented and used in at least three other runtimes right now. It's not going anywhere and there's really no benefit I think to changing it's status

LiviaMedeiros · 2023-02-05T11:21:49Z

Is there a reason to work with TypedArray and not ArrayBufferView (which also includes DataView)?

TypedArray.from is basically a variant of Array.from and therefore requires iterable value as first parameter, but it should be irrelevant for this method.

jasnell · 2023-02-05T15:56:29Z

@LiviaMedeiros ... the implementation does use ArrayBufferView. I've updated the doc change to reflect that.

aduh95 · 2023-02-05T18:51:05Z

@LiviaMedeiros ... the implementation does use ArrayBufferView. I've updated the doc change to reflect that.

How do we interpret length for DataViews? As the number of 8-bit chunks? Or 32-bit chunks? Same questions for offset. Whatever we chose, it would feel quite arbitrary to me, I’d rather have us only support TypedArray, and maybe DataView only if length and offset are not supplied.

jasnell · 2023-02-05T19:03:56Z

Yeah, I realized that the DataView wouldn't work with the current implementation anyway. Let's limit it to TypedArray. I don't really like the idea of supporting DataView only part of the time, and would rather just make it consistent no matter what other arguments are passed in.

nodejs-github-bot · 2023-02-05T19:09:36Z

CI: https://ci.nodejs.org/job/node-test-pull-request/49410/

LiviaMedeiros · 2023-02-05T19:22:53Z

How do we interpret length for DataViews? As the number of 8-bit chunks? Or 32-bit chunks? Same questions for offset.

It might be reasonable to interpret both as number of bytes, and length can become byteLength.
This would also remove limitation of having to copy full chunks.

I'm not quite sure if there are usecases of copy where counting elements is preferred over counting bytes and at the same time it wouldn't be better to process elements on copy (i.e. by using TypedArray.from(arrayLike, mapFn) which also can be ported to Buffer).

lib/buffer.js

doc/api/buffer.md

aduh95

The commit message needs to be amended to use the new name.

lib/buffer.js

test/parallel/test-buffer-from.js

Fixes: nodejs#43862

nodejs-github-bot · 2023-02-26T17:52:54Z

CI: https://ci.nodejs.org/job/node-test-pull-request/50028/ 💛

Fixes: #43862 PR-URL: #46500 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>

jasnell · 2023-02-26T22:10:41Z

Landed in 311fb04

Fixes: #43862 PR-URL: #46500 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>

Notable changes: buffer: * (SEMVER-MINOR) add Buffer.copyBytesFrom(...) (James M Snell) #46500 doc: * add marco-ippolito to collaborators (Marco Ippolito) #46816 events: * (SEMVER-MINOR) add listener argument to listenerCount (Paolo Insogna) #46523 lib: * (SEMVER-MINOR) add AsyncLocalStorage.bind() and .snapshot() (flakey5) #46387 src: * (SEMVER-MINOR) add `fs.openAsBlob` to support File-backed Blobs (James M Snell) #45258 tls: * (SEMVER-MINOR) support automatic DHE (Tobias Nießen) #46978 url: * (SEMVER-MINOR) implement URLSearchParams size getter (James M Snell) #46308 wasi: * (SEMVER-MINOR) add support for version when creating WASI (Michael Dawson) #46469 worker: * (SEMVER-MINOR) add support for worker name in inspector and trace_events (Debadree Chatterjee) #46832 PR-URL: #47086

Notable changes: buffer: * (SEMVER-MINOR) add Buffer.copyBytesFrom(...) (James M Snell) #46500 doc: * add marco-ippolito to collaborators (Marco Ippolito) #46816 events: * (SEMVER-MINOR) add listener argument to listenerCount (Paolo Insogna) #46523 lib: * (SEMVER-MINOR) add AsyncLocalStorage.bind() and .snapshot() (flakey5) #46387 src: * (SEMVER-MINOR) add `fs.openAsBlob` to support File-backed Blobs (James M Snell) #45258 tls: * (SEMVER-MINOR) support automatic DHE (Tobias Nießen) #46978 url: * (SEMVER-MINOR) implement URLSearchParams size getter (James M Snell) #46308 wasi: * (SEMVER-MINOR) add support for version when creating WASI (Michael Dawson) #46469 worker: * (SEMVER-MINOR) add support for worker name in inspector and trace_events (Debadree Chatterjee) #46832 PR-URL: #47087

ethanresnick · 2023-03-18T03:37:16Z

doc/api/buffer.md

+
+```js
+const u16 = new Uint16Array([0, 0xffff]);
+const buf = Buffer.copyBytesFrom(u16, 0, 1);


@jasnell Was the offset in this example supposed to be 1?

Fixes: #43862 PR-URL: #46500 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>

Notable changes: * buffer: * (SEMVER-MINOR) add Buffer.copyBytesFrom(...) (James M Snell) #46500 * doc: * add marco-ippolito to collaborators (Marco Ippolito) #46816 * add debadree25 to collaborators (Debadree Chatterjee) #46716 * add deokjinkim to collaborators (Deokjin Kim) #46444 * events: * (SEMVER-MINOR) add listener argument to listenerCount (Paolo Insogna) #46523 * lib: * (SEMVER-MINOR) add AsyncLocalStorage.bind() and .snapshot() (flakey5) #46387 * (SEMVER-MINOR) add aborted() utility function (Debadree Chatterjee) #46494 * src: * (SEMVER-MINOR) allow optional Isolate termination in node::Stop() (Shelley Vohr) #46583 * (SEMVER-MINOR) allow embedder control of code generation policy (Shelley Vohr) #46368 * stream: * (SEMVER-MINOR) add abort signal for ReadableStream and WritableStream (Debadree Chatterjee) #46273 * tls: * (SEMVER-MINOR) support automatic DHE (Tobias Nießen) #46978 * url: * (SEMVER-MINOR) implement URLSearchParams size getter (James M Snell) #46308 * worker: * (SEMVER-MINOR) add support for worker name in inspector and trace_events (Debadree Chatterjee) #46832 PR-URL: TBD

Notable changes: * buffer: * (SEMVER-MINOR) add Buffer.copyBytesFrom(...) (James M Snell) #46500 * doc: * add marco-ippolito to collaborators (Marco Ippolito) #46816 * add debadree25 to collaborators (Debadree Chatterjee) #46716 * add deokjinkim to collaborators (Deokjin Kim) #46444 * events: * (SEMVER-MINOR) add listener argument to listenerCount (Paolo Insogna) #46523 * lib: * (SEMVER-MINOR) add AsyncLocalStorage.bind() and .snapshot() (flakey5) #46387 * (SEMVER-MINOR) add aborted() utility function (Debadree Chatterjee) #46494 * src: * (SEMVER-MINOR) allow optional Isolate termination in node::Stop() (Shelley Vohr) #46583 * (SEMVER-MINOR) allow embedder control of code generation policy (Shelley Vohr) #46368 * stream: * (SEMVER-MINOR) add abort signal for ReadableStream and WritableStream (Debadree Chatterjee) #46273 * tls: * (SEMVER-MINOR) support automatic DHE (Tobias Nießen) #46978 * url: * (SEMVER-MINOR) implement URLSearchParams size getter (James M Snell) #46308 * worker: * (SEMVER-MINOR) add support for worker name in inspector and trace_events (Debadree Chatterjee) #46832 PR-URL: #47502

Notable changes: * buffer: * (SEMVER-MINOR) add Buffer.copyBytesFrom(...) (James M Snell) #46500 * doc: * add marco-ippolito to collaborators (Marco Ippolito) #46816 * add debadree25 to collaborators (Debadree Chatterjee) #46716 * add deokjinkim to collaborators (Deokjin Kim) #46444 * events: * (SEMVER-MINOR) add listener argument to listenerCount (Paolo Insogna) #46523 * lib: * (SEMVER-MINOR) add AsyncLocalStorage.bind() and .snapshot() (flakey5) #46387 * (SEMVER-MINOR) add aborted() utility function (Debadree Chatterjee) #46494 * src: * (SEMVER-MINOR) allow optional Isolate termination in node::Stop() (Shelley Vohr) #46583 * (SEMVER-MINOR) allow embedder control of code generation policy (Shelley Vohr) #46368 * (SEMVER-MINOR) add initial support for single executable applications (Darshan Sen) #45038 * stream: * (SEMVER-MINOR) add abort signal for ReadableStream and WritableStream (Debadree Chatterjee) #46273 * tls: * (SEMVER-MINOR) support automatic DHE (Tobias Nießen) #46978 * url: * (SEMVER-MINOR) implement URLSearchParams size getter (James M Snell) #46308 * worker: * (SEMVER-MINOR) add support for worker name in inspector and trace_events (Debadree Chatterjee) #46832 PR-URL: #47502

Notable changes: Add initial support for single executable applications Compile a JavaScript file into a single executable application: ```console $ echo 'console.log(`Hello, ${process.argv[2]}!`);' > hello.js $ cp $(command -v node) hello $ npx postject hello NODE_JS_CODE hello.js \ --sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2 $ npx postject hello NODE_JS_CODE hello.js \ --sentinel-fuse NODE_JS_FUSE_fce680ab2cc467b6e072b8b5df1996b2 \ --macho-segment-name NODE_JS $ ./hello world Hello, world! ``` Contributed by Darshan Sen in #45038 Replace url parser with Ada Node.js gets a new URL parser called Ada that is compliant with the WHATWG URL Specification and provides more than 100% performance improvement to the existing implementation. Contributed by Yagiz Nizipli in #46410 Other notable changes: * buffer: * (SEMVER-MINOR) add Buffer.copyBytesFrom(...) (James M Snell) #46500 * doc: * add marco-ippolito to collaborators (Marco Ippolito) #46816 * add debadree25 to collaborators (Debadree Chatterjee) #46716 * add deokjinkim to collaborators (Deokjin Kim) #46444 * events: * (SEMVER-MINOR) add listener argument to listenerCount (Paolo Insogna) #46523 * lib: * (SEMVER-MINOR) add AsyncLocalStorage.bind() and .snapshot() (flakey5) #46387 * (SEMVER-MINOR) add aborted() utility function (Debadree Chatterjee) #46494 * src: * (SEMVER-MINOR) allow optional Isolate termination in node::Stop() (Shelley Vohr) #46583 * (SEMVER-MINOR) allow embedder control of code generation policy (Shelley Vohr) #46368 * stream: * (SEMVER-MINOR) add abort signal for ReadableStream and WritableStream (Debadree Chatterjee) #46273 * tls: * (SEMVER-MINOR) support automatic DHE (Tobias Nießen) #46978 * url: * (SEMVER-MINOR) implement URLSearchParams size getter (James M Snell) #46308 * worker: * (SEMVER-MINOR) add support for worker name in inspector and trace_events (Debadree Chatterjee) #46832 PR-URL: #47502

jasnell added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 4, 2023

nodejs-github-bot added buffer Issues and PRs related to the buffer subsystem. needs-ci PRs that need a full CI run. labels Feb 4, 2023

anonrig reviewed Feb 4, 2023

View reviewed changes

lib/buffer.js Outdated Show resolved Hide resolved

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 4, 2023

This comment was marked as outdated.

Sign in to view

aduh95 reviewed Feb 4, 2023

View reviewed changes

lib/buffer.js Outdated Show resolved Hide resolved

lib/buffer.js Outdated Show resolved Hide resolved

lib/buffer.js Outdated Show resolved Hide resolved

lib/buffer.js Outdated Show resolved Hide resolved

aduh95 reviewed Feb 4, 2023

View reviewed changes

doc/api/buffer.md Outdated Show resolved Hide resolved

doc/api/buffer.md Outdated Show resolved Hide resolved

doc/api/buffer.md Outdated Show resolved Hide resolved

lib/buffer.js Outdated Show resolved Hide resolved

github-actions bot mentioned this pull request Feb 5, 2023

CI Reliability 2023-02-05 nodejs/reliability#495

Open

22 tasks

addaleax reviewed Feb 5, 2023

View reviewed changes

doc/api/buffer.md Outdated Show resolved Hide resolved

LiviaMedeiros mentioned this pull request Feb 5, 2023

buffer: add from(ArrayBufferView) support #43863

Closed

jasnell force-pushed the buffer-fromtypedarray branch 2 times, most recently from 9b2f1e7 to ce45b9f Compare February 5, 2023 15:55

jasnell changed the title ~~buffer: add Buffer.fromTypedArray(...)~~ buffer: add Buffer.copyFrom(...) Feb 5, 2023

jasnell added semver-minor PRs that contain new features and should be released in the next minor version. request-ci Add this label to start a Jenkins CI on a PR. labels Feb 5, 2023

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 5, 2023

This comment was marked as outdated.

Sign in to view

jasnell force-pushed the buffer-fromtypedarray branch from ce45b9f to fe0bc1c Compare February 5, 2023 19:02

jasnell added the request-ci Add this label to start a Jenkins CI on a PR. label Feb 5, 2023

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 5, 2023

aduh95 reviewed Feb 5, 2023

View reviewed changes

lib/buffer.js Outdated Show resolved Hide resolved

lib/buffer.js Show resolved Hide resolved

doc/api/buffer.md Outdated Show resolved Hide resolved

jasnell requested a review from addaleax February 11, 2023 15:09

anonrig approved these changes Feb 11, 2023

View reviewed changes

aduh95 reviewed Feb 11, 2023

View reviewed changes

lib/buffer.js Outdated Show resolved Hide resolved

test/parallel/test-buffer-from.js Outdated Show resolved Hide resolved

test/parallel/test-buffer-from.js Outdated Show resolved Hide resolved

buffer: add Buffer.copyBytesFrom(...)

6b337c2

Fixes: nodejs#43862

jasnell force-pushed the buffer-fromtypedarray branch from 339dc32 to 6b337c2 Compare February 25, 2023 00:35

jasnell added request-ci Add this label to start a Jenkins CI on a PR. author ready PRs that have at least one approval, no pending requests for changes, and a CI started. labels Feb 25, 2023

github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Feb 25, 2023

This comment was marked as outdated.

Sign in to view

aduh95 approved these changes Feb 25, 2023

View reviewed changes

jasnell added a commit that referenced this pull request Feb 26, 2023

buffer: add Buffer.copyBytesFrom(...)

311fb04

Fixes: #43862 PR-URL: #46500 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>

jasnell closed this Feb 26, 2023

targos pushed a commit that referenced this pull request Mar 13, 2023

buffer: add Buffer.copyBytesFrom(...)

7eb8cef

Fixes: #43862 PR-URL: #46500 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>

targos pushed a commit that referenced this pull request Mar 14, 2023

buffer: add Buffer.copyBytesFrom(...)

2fece54

Fixes: #43862 PR-URL: #46500 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>

targos mentioned this pull request Mar 14, 2023

v19.8.0 release proposal #47087

Merged

ethanresnick reviewed Mar 18, 2023

View reviewed changes

danielleadams pushed a commit that referenced this pull request Apr 11, 2023

buffer: add Buffer.copyBytesFrom(...)

c6e722a

Fixes: #43862 PR-URL: #46500 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Antoine du Hamel <[email protected]>

danielleadams mentioned this pull request Apr 11, 2023

v18.16.0 release proposal #47502

Merged

sankalp1999 mentioned this pull request May 10, 2023

Buffer.copyBytesFrom should accept a DataView as well. #47924

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

buffer: add Buffer.copyFrom(...) #46500

buffer: add Buffer.copyFrom(...) #46500

jasnell commented Feb 4, 2023 •

edited

Loading

This comment was marked as outdated.

addaleax commented Feb 5, 2023

jasnell commented Feb 5, 2023

LiviaMedeiros commented Feb 5, 2023

jasnell commented Feb 5, 2023

This comment was marked as outdated.

aduh95 commented Feb 5, 2023 •

edited

Loading

jasnell commented Feb 5, 2023

nodejs-github-bot commented Feb 5, 2023

LiviaMedeiros commented Feb 5, 2023

aduh95 left a comment

This comment was marked as outdated.

nodejs-github-bot commented Feb 26, 2023 •

edited by jasnell

Loading

jasnell commented Feb 26, 2023

ethanresnick Mar 18, 2023

buffer: add Buffer.copyFrom(...) #46500

buffer: add Buffer.copyFrom(...) #46500

Conversation

jasnell commented Feb 4, 2023 • edited Loading

This comment was marked as outdated.

addaleax commented Feb 5, 2023

jasnell commented Feb 5, 2023

LiviaMedeiros commented Feb 5, 2023

jasnell commented Feb 5, 2023

This comment was marked as outdated.

aduh95 commented Feb 5, 2023 • edited Loading

jasnell commented Feb 5, 2023

nodejs-github-bot commented Feb 5, 2023

LiviaMedeiros commented Feb 5, 2023

aduh95 left a comment

Choose a reason for hiding this comment

This comment was marked as outdated.

nodejs-github-bot commented Feb 26, 2023 • edited by jasnell Loading

jasnell commented Feb 26, 2023

ethanresnick Mar 18, 2023

Choose a reason for hiding this comment

jasnell commented Feb 4, 2023 •

edited

Loading

aduh95 commented Feb 5, 2023 •

edited

Loading

nodejs-github-bot commented Feb 26, 2023 •

edited by jasnell

Loading