Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use experimental worker_threads to run tasks #29

Merged
merged 1 commit into from
Jul 16, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
language: node_js
node_js:
- "6"
- "10.5"
13 changes: 9 additions & 4 deletions History.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
2.0.0-alpha.1 / 2018-07-16
==================

* use experimental worker_threads to run tasks

1.6.1 / 2018-06-20
==================

Expand All @@ -14,7 +19,7 @@
==================

* message-pack streaming (@noam-almog)

1.4.0 / 2017-05-29
==================

Expand All @@ -25,12 +30,12 @@

* Minor performance fix: getting rid of array destructuring as it prevents code runtime optimization
in current v8

1.3.0 / 2017-01-23
==================

* Workers do not accept work when they are in process of booting up (@mheiniger)

1.2.1 / 2016-12-19
==================

Expand All @@ -46,7 +51,7 @@
==================

* Added <code>useCompression</code> option

1.0.0 / 2016-11-01
==================

Expand Down
32 changes: 17 additions & 15 deletions Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,16 +4,18 @@

# worker-nodes

A node.js library to run cpu-intensive tasks in a separate processes and to not block the event loop.
*THIS IS AN EXPERIMENTAL BRANCH, IT USES [worker_threads](https://nodejs.org/api/worker_threads.html) WHICH IS NOT STABLE YET*

A node.js library to run cpu-intensive tasks in a separate thread and to not block the event loop.


## Installation

```bash
$ npm install worker-nodes
$ npm install worker-nodes@next
```

Node.js greater than 6.6.0 highly recommended.
Node.js greater than 10.5.0 is *required*

# API Reference

Expand Down Expand Up @@ -206,18 +208,18 @@ Example results:
```bash
results for 100 executions

name time: total [ms] time usr [ms] time sys [ms] worker usr [ms] worker sys [ms] mem rss [MB] worker rss [MB] errors
------------------ ---------------- ------------- ------------- --------------- --------------- ------------ --------------- ------
no-workers 150 239 42 0 0 98 0 0
worker-nodes@1.2.0 1521 646 528 641 367 272 119 0
[email protected] 12055 7356 5726 896 212 731 74 0
worker-farm@1.3.1 12124 6711 5501 1577 446 689 74 0
[email protected] 12348 6866 5474 1696 458 698 76 0
worker-pool@3.0.2 14029 7633 5604 2285 649 769 104 0

os : Darwin / 15.5.0 / x64
cpu : Intel(R) Core(TM) i7-4578U CPU @ 3.00GHz × 4
node : 6.9.1 / v8: 5.1.281.84
name time: total [ms] time usr [ms] time sys [ms] worker usr [ms] worker sys [ms] mem rss [MB] worker rss [MB] errors
------------------ ---------------- ------------- ------------- --------------- --------------- ------------ --------------- ------
no-workers 162 250 38 0 0 101 0 0
worker-nodes@next 458 573 186 571 185 210 207 0
[email protected] 1503 670 395 991 337 292 87 0
workerpool@2.3.0 1684 1511 688 292 82 155 49 0
[email protected] 2508 1435 511 1247 368 104 59 0
process-pool@0.3.4 2571 1537 517 1333 376 105 61 0
[email protected] 15939 15984 5632 1946 546 86 79 0

os : Darwin / 17.5.0 / x64
cpu : Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz × 8
```

## See also
Expand Down
186 changes: 186 additions & 0 deletions benchmark/package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion benchmark/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,6 @@
},
"scripts": {
"pretest": "mkdir -p fixtures; cd fixtures; base64 /dev/urandom | head -c 1048576 > output.dat; base64 /dev/urandom | head -c 524288 > input.dat;",
"test": "node reporter.js --repeats 100 ./html-renderer/*"
"test": "node --experimental-worker reporter.js --repeats 100 ./html-renderer/*"
}
}
4 changes: 4 additions & 0 deletions lib/util/queue.js
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,10 @@ class Queue {
return this.storage.forEach(predicate);
}

map(predicate) {
return this.storage.map(predicate);
}

filter(predicate) {
return this.storage.filter(predicate);
}
Expand Down
20 changes: 8 additions & 12 deletions lib/worker/child-loader.js
Original file line number Diff line number Diff line change
@@ -1,22 +1,18 @@
const net = require('net');
const { parentPort } = require('worker_threads');

const { Request, Response } = require('./message');
const Transport = require('./transport');

let $module;
let $cmdTransport = process;
let $dataTransport;

function setupModule({ modulePath }) {
// load target module
$module = require(modulePath);

// setup data transport channel
$dataTransport = new Transport(new net.Socket({ fd: 3 }));
$dataTransport.on('message', handleCall);
// setup data channel
parentPort.on('message', handleCall);

// report readiness
$cmdTransport.send('ready');
parentPort.postMessage('ready');
}

function handleCall(requestData) {
Expand All @@ -33,7 +29,7 @@ function handleCall(requestData) {
})
.then(result => {
response.setResult(result);
$dataTransport.send(response);
parentPort.postMessage(response);
})
.catch(err => {
const error = {
Expand All @@ -45,15 +41,15 @@ function handleCall(requestData) {
Object.keys(err).forEach(key => error[key] = err[key]);
response.error = error;

$dataTransport.send(response);
parentPort.postMessage(response);
});
}

$cmdTransport.on('message', function ({ cmd = 'call', data }) {
parentPort.on('message', function ({ cmd = 'call', data }) {
switch (cmd) {
case 'start':
return setupModule(data);
case 'exit':
return process.exit(0);
}
});
});
Loading