Performance: scattergl lines slower than scatter lines?

[mitja-p]

Hi,

everyone recommends scattergl for many points as it should perform better. I am however having troubles understanding why for rendering 100.000 points scatter is much faster than scattergl. Is this a bug?

Timings on my computer:
scattergl 1.5s
scatter 0.2s

At 15.000 points scattergl is already twice slower than scatter.

Example (open console to see the timings):
https://jsfiddle.net/tdr90ebm/

[etpinard]

Thanks for writing in.

Looks like for this particular dataset, the scatter line decimation algorithm works very well, leading to better performance than its scattergl counterpart. Moreover, you're using react in all the calls, so the very first call has an handicap - as it has to draw the graph's skeleton framework something that the subsequent calls do not have to repeat.

Notice that if you add mode: 'markers' to both the scattergl and scatter traces, the results are switched (by a lot): scattergl renders in ~150ms whereas scatter needs at least 5000ms to render https://jsfiddle.net/h61am47y/

---

One (easy) thing we could try to improve scattergl line performance would be to make it pass through the scatter line decimation algorithm behind some user-facing flag (e.g. line.simplify: true).

[mitja-p]

I tried to turn off line decimation (at least I think I did):
arr.forEach(p=>{p.type='scatter', p.line={simplify:false}});
Timings are still 1,5s (scattergl) vs 0,5s (scatter).

[mitja-p]

I will leave here another comment regarding performance.

1. If data is provided as Float32Array, performance is significantly lower than when using regular Array. It is wise to always convert to Array via Array.from(FloatArray).

2. Could we add a flag (for example persistGlLayer) to options, to keep the gl container alive when removing all gl series as initializing gl 2d scene takes time? Implementation is simple:
   file: src\plots\gl2d\index.js
   function exports.clean
   if (subplotData.length === 0 && !gd._context.persistGlLayer)

[veggiesaurus]

1. If data is provided as Float32Array, performance is significantly lower than when using regular Array. It is wise to always convert to Array via Array.from(FloatArray).

@panchyo0 it looks like we're not the only ones to discover this 🙈

@etpinard any idea why typed arrays have to go through a "data clean" step each plot refresh? That's what seems to cause the issue. Is this a D3 bug?

[archmoj]

1. If data is provided as Float32Array, performance is significantly lower than when using regular Array. It is wise to always convert to Array via Array.from(FloatArray).

@panchyo0 it looks like we're not the only ones to discover this

@etpinard any idea why typed arrays have to go through a "data clean" step each plot refresh? That's what seems to cause the issue. Is this a D3 bug?

@veggiesaurus scattergl traces are based on WebGL/regl not SVG/D3.
I kind of agree with you that we could skip data cleanup for typed arrays.

[archmoj]

A simple demo to help investigate the performance issue(s).

[leeoniya]

btw, if any plotly devs have advice for improving the scattergl bench [1], i'd be glad to implement it.

[1] https://github.com/leeoniya/uPlot#performance

[archmoj]

closed via #5632.

[archmoj]

#5632 addressed part of this.

[gvwilson]

Hi - we are trying to tidy up the stale issues and PRs in Plotly's public repositories so that we can focus on things that are still important to our community. Since this one has been sitting for several years, I'm going to close it; if it is still a concern, please add a comment letting us know what recent version of our software you've checked it with so that I can reopen it and add it to our backlog. Thanks for your help - @gvwilson