You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently @watchable/queue is used by nevermore to sequence job settlements. The implementation of @watchable/queue employs immutable arrays (meaning every insertion copies items into a new array). This could lead to a significant performance impact when there is a backlog of e.g. 1000-plus items. It may explain why bulk tests slow down after e.g. 10000 items
A better design would be to employ a FIFO queue employing a linked list with head and tail pointers. This has an O(1) insertion and removal cost regardless of the length of the queue.
The text was updated successfully, but these errors were encountered:
cefn
changed the title
Consider removing watchable/queue from nevermore
Consider removing @watchable/queue from nevermore
Jun 15, 2024
Another related intervention might be to switch @watchable/queue to use FIFO internally and find a different way to introduce immutability (e.g. with a queue subscription that can snapshot the sequence for debugging purposes).
Currently @watchable/queue is used by nevermore to sequence job settlements. The implementation of @watchable/queue employs immutable arrays (meaning every insertion copies items into a new array). This could lead to a significant performance impact when there is a backlog of e.g. 1000-plus items. It may explain why bulk tests slow down after e.g. 10000 items
A better design would be to employ a FIFO queue employing a linked list with head and tail pointers. This has an O(1) insertion and removal cost regardless of the length of the queue.
The text was updated successfully, but these errors were encountered: