Store graph edges in SharedArrayBuffer

[lettertwo]

Store graph edges in SharedArrayBuffer

This changes the low level storage mechanism for graph adjacency lists from multiple JS Maps to a custom hashmap (😅 ) implemented on top of SharedArrayBuffer.

---

Background: Why would we want to implement our own hashmap?

Parallelism

Parcel has the ability to run many operations in parallel, but Node does not have any way to share data between threads except by serializing data and sending it to each worker, where it is then deserialized by each worker before work commences. Even for modestly sized Parcel projects, this overhead is significant.

JavaScript does have support for sharing memory between threads via SharedArrayBuffer, but unfortunately, there is currently nothing higher level (like a SharedMap or something) in JS or Node that can take advantage of this in a significant way.

Thus, having a custom data structure built on top of SharedArrayBuffer means that Parcel can build a very large graph and share some significant portion of that graph with workers for free.

Caching

Parcel caches build state frequently, and that involves serializing data to be written to cache. an ArrayBuffer of data serializes more quickly than a Map.

Other Solutions Considered

We evaluated other options, such as Protocol Buffers and FlatBuffers (either of which may still be viable for storing node data in the future), but ruled them out for this work for the following reasons:

• Any transport format still requires serialization and deserialization, which is inherently less efficient than sharing memory
• These solutions generally incur a build step to generate bindings for reading and writing data to the underlying transport format
• The adjacency list only stores integers (node ids, edge types) which makes it good candidate for storage in an ArrayBuffer

---

We've added a small set of unit tests to exercise the basic assumptions about our implementation, but we are mostly relying on the existing suites of tests to verify this work.

We have also done extensive benchmarking to measure the impact of these changes in a real-world, significantly sized app.

---

EDIT: Refactoring based on feedback seems to have improved the overall performance enough to eliminate most of the regressions!

Benchmarks: Testing the impact of these changes in a real app

---

These benchmarks were taken on a Parcel app with a bundle graph containing ~92,421 nodes and ~255,538 edges and a request graph containing ~163,633 nodes and ~1,351,038 edges.

Some definitions:

Cold Build

A parcel production build with no cache.

Warm Build

A parcel production build from cache.

Cold Start

A parcel dev start with no cache.

Warm Start

A parcel dev start from cache.

Total {Serialization,Deserialization} Time

How much time is spent serializing or deserializing the graphs (including node data)

Edge {Serialization,Deserialization} Time

How much time is spent serializing or deserializing the adjacency list (just the edge data)

Cold Build	v2	this PR	change	change (%)
Total Memory	3397.57 mb	1755.89 mb	-1641.68 mb	48% less
Total Serialization Time	4941 ms	3309 ms	-1632 ms	33% faster
Edge Serialization Time	2372 ms	1456 ms	-916 ms	39% faster
Total Deserialization Time	5738 ms	4123 ms	-1615 ms	28% faster
Edge Deserialization Time	2994 ms	518 ms	-2476 ms	83% faster
init	1 ms	1 ms	+0 ms	16% slower
build	163023 ms	135714 ms	-27309 ms	17% faster
shutdown	2991 ms	4517 ms	+1525 ms	51% slower
✨ Built in	146.39 s	135.69 s	-10.70 s	7% faster

Warm Build	v2	this PR	change	change (%)
Total Memory	1304.12 mb	946.77 mb	-357.35 mb	27% less
Total Serialization Time	6469 ms	5379 ms	-1090 ms	17% faster
Edge Serialization Time	3109 ms	1438 ms	-1671 ms	54% faster
Total Deserialization Time	6223 ms	4563 ms	-1660 ms	27% faster
Edge Deserialization Time	3028 ms	495 ms	-2533 ms	84% faster
init	1994 ms	744 ms	-1250 ms	63% faster
build	4007 ms	3304 ms	-703 ms	18% faster
shutdown	2678 ms	3901 ms	+1223 ms	46% slower
✨ Built in	4.95 s	3.79 s	-1.16 s	23% faster

Cold Start	v2	this PR	change	change (%)
Total Memory	2337.65 mb	1384.07 mb	-953.58 mb	41% less
Total Serialization Time	4110 ms	3049 ms	-1061 ms	26% faster
Edge Serialization Time	2358 ms	1329 ms	-1029 ms	44% faster
Total Deserialization Time	6441 ms	3311 ms	-3130 ms	49% faster
Edge Deserialization Time	3141 ms	356 ms	-2785 ms	89% faster
init	1 ms	2 ms	+1 ms	41% slower
build	79695 ms	69485 ms	-10211 ms	13% faster
save	628 ms	415 ms	-213 ms	34% faster
shutdown	2565 ms	3490 ms	+925 ms	36% slower
✨ Built in	76.64 s	69.47 s	-7.17 s	19% faster

Warm Start	v2	this PR	change	change (%)
Total Memory	1195.51 mb	807.47 mb	-388.04 mb	32% less
Total Serialization Time	6756 ms	4198 ms	-2558 ms	38% faster
Edge Serialization Time	2326 ms	1498 ms	-828 ms	36% faster
Total Deserialization Time	6254 ms	4073 ms	-2181 ms	35% faster
Edge Deserialization Time	2897 ms	501 ms	-2396 ms	83% faster
init	1934 ms	907 ms	-1028 ms	53% faster
build	3382 ms	2418 ms	-964 ms	29% faster
save	999 ms	464 ms	-535 ms	54% faster
shutdown	2564 ms	3067 ms	+503 ms	20% slower
✨ Built in	2.57 s	2.42 s	-0.15 s	6% faster

---

Summary

• Total memory usage is down significantly across all tested conditions
• Serialization and deserialization are both significantly faster in all tested conditions
• With warm cache, builds are on par or faster in all tested conditions
• Shutdown times seem to be slower, which we did not expect, and requires further investigation.

While we have seen some regression with cold build times, and not much improvement with warm times, We have seen no regression (and even some improvement!) in build times, and we have also seen significant improvement in memory usage and serialization and deserialization times in all build scenarios. However, shutdown times are slower than expected.

The main reasons we think these are acceptable trade-offs are:

• We believe we have not significantly regressed the most common DX scenarios
• These results reflect one part of the graph improvements we are working toward. Our hope and expectation is that we will ultimately improve all of these metrics in future by backing the graph nodes in shared buffers like we have done for the edges here.

For this work, we did not address data stored in the graph nodes, focusing solely on the edges. Since there are generally many more edges than nodes in most of Parcel's graphs, this change represents a measurable improvement, even though there is still more to do to improve the efficiency of the graph structure.

[lettertwo]

This is the now the only scenario where we generate values for iteration. All other traversals are now using arrays.

[lettertwo]

This method appears to be unused.

[lettertwo]

Note that we don't actually add an entry to the node map until it is connected to another node id in addEdge.

[lettertwo]

If this edge is the first of this type for either of the nodes, we add a new entry to the node map.

[lettertwo]

The implementation described here was inspired by This dissection of the v8 Map internals

[lettertwo]

There would be significant memory saved if we used DataView instead, which would allow us to store the type field in 1 byte (instead of 4), but it also would complicate the implementation significantly.

The savings would come from the following:

• each node adds an entry for every unique edge type it is connected to, so in the worst case (every node had at least one edge of every type), that's n * t * 3 bytes saved, where n is the number of nodes and t is the number of unique edge types in the graph.
• each edge keeps its type in its entry, which would be e * 3 bytes saved, where e is the number of edges in the graph.

Additionally, typed arrays are technically not portable, since they use the originating system's endianess

Some of the reasons we haven't implemented this yet:

• DataView is likely not as well optimized as type arrays
• memory allocation is more complex, since we cannot rely on the uniform item size of a typed array
• reads and writes must now be aware of endianness
• In practice, most systems are LE, so the non-portable nature of typed array is probably a non-issue
• memory savings from this implementation are already significant compared to v8 Maps

[thebriando]

parcel/packages/core/core/src/BundleGraph.js

Lines 1210 to 1213 in 8aa9c25

	this._graph
	.getNodeIdsConnectedFrom(nodeId, bundleGraphEdgeTypes.references)
	.reverse(),
	});

I wonder if creating a new function to do this would have any benefit since we can traverse backwards with the doubly linked list now.

[devongovett]

🥳 🎉