[Proposal] - Thead Safe data structures

[HiImGiovi]

What is the problem this feature will solve?

What I'd like to achieve is an implementation of shared (between worker threads) and thread safe data structures.
I want something easier to work with when doing multithread nodejs. Sharing data between threads using SharedArrayBuffer can be hard sometimes.

What is the feature you are proposing to solve the problem?

I'll show you an example of what the API usage could look like for a data structure like LRU Cache:

This is the code in the main thread

// main 
import {LRUCache} from "shared-ds"
import {Worker} from "worker_threads"

const cache = new LRUCache();

cache.set("test", { p1: "property1", p2: "property2" })

const worker = new Worker("./worker.js", {
    workerData: {
          lruCache: cache
    }
}

this is the worker code:

import { workerData } from "worker_threads";
import { LRUCache } from "shared-ds";


const cache = LruCache.fromWorkerData(workerData.lruCache);

const testValue = cache.get("test");
console.log(testValue); // this outputs { p1: "property1", p2: "property2" }

I started working into an implementation that uses extensively SharedArrayBuffer and Atomics to reach thread safety but there are lots of problems to take into consideration like:

• Serialization of data into SharedArrayBuffer
• atomics operations ( I still did not figure out how I can read multiple 'bytes atomically' avoiding multiple Atomic.load calls)

Do you guys think it's a useful feature and worth implementing or taking it into consideration?

What alternatives have you considered?

No response

[Jamesernator]

There is a proposal for shared objects.

There's an older prototype of that proposal (without the struct syntax) behind the flag --harmony-struct, use new SharedStructType(["key1", "key2"]) to create constructors.

( I still did not figure out how I can read multiple 'bytes atomically' avoiding multiple Atomic.load calls)

This is impossible with atomics, they can only read a word (8 bytes or less) at a time. Just use a lock/mutex (the prototype has Atomics.Mutex).

[avivkeller]

Workers are, IIUC, a bit different from web workers. Workers are executed from a subprocess of node (a separate command), so transferring data isn't as straightforward as one might think.

Edit: My knowledge of workers isn't very accurate 😅

[joyeecheung]

Workers are executed from a subprocess of node (a separate command)

No, they are executed on a different thread but it's still in the same process. In terms of data sharing, it's not too different from Web workers. When V8 ships shared structs, they will also be usable in Node.js workers (maybe requires some additional integration patches to work properly, though we do already have some experimental integration for shared structs behind flags #47706)