Stardust: GPU-Based Visualization Library | Hacker News Reader

Stardust: GPU-Based Visualization Library

stardustjs.github.io

233 pointsborisjabes36 comments

This runs like magic on an iPhone, but for some reason Chrome, Safari and FF on desktop (mac, recent mbp) it's sluggish as hell. Burning through CPU, 15fps for the force-graph.

I feel like something's wrong, but I don't even know where to start debugging this. Do I need to enable Chrome feature flags? Or do mbps just have less powerful gpus than iPhones?

Chrome and Firefox on Android, on my Pixel XL, don't rise above 32 FPS. Chrome is a bit more performant than FF but not by much. I think you're right in your assessment; the iPhone's GPU is much stronger than you might think.

On my desktop (Nvidia 750 Ti , Windows 10) I get 60fps with Chrome, Firefox, and Edge on everything I've tried of the examples.

Gave that a shot on the (Intel Graphics) Macbook Pro and wow, didn't even manage 10fps. Safari was worst of the lot.

Most examples are 60 fps, but on the last one I get 60 fps in MS Edge, but only ~35 in Chrome. IE won't even load it for whatever reason. Quadro k620

2015 MB Air, stock Chrome, silky smooth and no heat.

i'm getting 15 fps also with the default settings, Google Chrome 58.0.3029.81 (Official Build) beta (64-bit), ubuntu 16.04.2, Intel(R) Core(TM) i5-6200U CPU @ 2.30GHz, Device: Mesa DRI Intel(R) HD Graphics 520 (Skylake GT2) (0x1916)

intel_gpu_top :

                   render busy:  99%:                    render space: 58/16384

                          task  percent busy
                            CS:  98%:     vert fetch: 422129727 (6178461/sec)
                           GAM:  97%:     prim fetch: 140710713 (2059497/sec)
                          GAFS:  90%:    VS invocations: 422116511 (6178270/sec)
                            VS:  89%:    GS invocations: 0 (0/sec)
                            CL:  89%:         GS prims: 0 (0/sec)
                            VF:  89%:    CL invocations: 140703527 (2059406/sec)
                            SF:  89%:         CL prims: 140659556 (2059072/sec)
                           SDE:   3%:                     PS invocations: 33276080884 (482451768/sec)
                          GAFM:   0%:                      PS depth pass: 35922946732 (507385276/sec)
                           SVG:   0%:                      
                           TDG:   0%:                      
                            RS:   0%:                      
                           TSG:   0%:                      
                           VFE:   0%:

This one seems to be doing more stuff with particles and runs much more smoothly :

https://lab.cheron.works/webgl-gpgpu-particles/

I'm getting 0.5fps with Safari Technology Preview, and a smooth 60fps with the stock Safari. I'm wondering what's up with that? I've tried both with and without the relevant entries I could find in "Experimental features" (WebGL 2.0, WebGPU, and something I forgot)

Likely has to do with the increase in the number of pixels on your desktop display. Even worse if you're using a high resolution external display.

Mobile phones actually have a surprisingly high pixel count compared to desktops. FHD displays are common, and QHD and even near 4K displays are pretty common.

The link to the online playground is broken. It goes to https://stardust-vis.github.io/playground/, but since the project seems to be named stardustjs right now, I tried https://stardustjs.github.io/playground/ and that worked.

Yes, the link on the topbar works, not the one in the readme.

I'm reading through the paper, and this library almost seems custom-built for my use-case: a data browser for datasets with arrays of hundreds of thousands of datapoints.

One of the reasons I didn't go with D3 or any of the existing charting frameworks is that they simply do not seem built with that quantity of data in mind. Not only the rendering, the whole model: data is all packed in trees of JS-objects (at least the examples I see).

I have simple data that can be represented as dense typed arrays[0], with all the performance benefits of such, and no library seems to make good use of that. If I read the documentation for Stardust correctly, it's an exception to that rule (which makes sense, because typed arrays were mainly introduced for the sake of WebGL).

Similarly they also seem to do smart things with avoiding having to re-upload data to the GPU. I really hope this lives up to my expectations, it would save me a whole lot of work!

[0] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Type...

Have you considered using dygraphs (http://dygraphs.com/)? It is designed to endure large ammounts of data.

Thanks for the suggestion, but it doesn't seem to support anything other than line charts, which is too limited for our needs.

That force-graph demo is awesome... those things usually take down my browser if they're non-trivial.

Totally agree. And it turns out not caused by the layout algorithm but exactly the rendering.

We've been drawing on canvas without proper acceleration for too long.

For more scalable on both, generally happy to share Graphistry API keys -- ping pygraphistry@graphistry.com . At the 100K+ scale, you need to optimize both :)

I'm excited to see both Stardust and the related DeckGL project. We've internally built several related framework layers (ex: streaming for cloud GPU offloading), and picking accessible abstractions for different developer personas is hard. Stardust & DeckGL are both (potentially) enabling dedicated visualization engineers to work more closely with high performance computing engineers. That's impressive: we deferred that problem to focus energy on enabling embedding for regular web developers (which goes above stardust) and scaling to the next 1000X (which goes below/adjacent)... but at the expense of making it harder for non-webgl visualization engineers to work on parts of our stack.

Long-term, I think their mindset is right, so, I've definitely been enjoying these projects!

actually that exact demo took down my browser (chrome 59 on win 10)

What makes this library fast?

I thought there were already many GPU accelerated paths for graphics in the browser, even including basic CSS functionality which leverages hardware acceleration in a significant number of scenarios.

It's great to see this library, just thought that since a lot of visualizations don't require advanced graphics techniques that a lot of GPU benefit was already being realized.

> I thought there were already many GPU accelerated paths for graphics in the browser, even including basic CSS functionality which leverages hardware acceleration in a significant number of scenarios.

That is true, and therein lies the catch. Those paths must support a significant number of scenarios, which means they must be very generic. This incurs (comparatively) huge performance penalties.

Using webgl trades genericity for performance. To put a bunch of 2d coordinate points on a canvas like this library does, only requires the equivalent of 1-2% of CSS functionality that browsers have to go through for every DOM element. It also means that whatever you're drawing has no impact on the rest of the page layout so there's another ton of overheads the browser can skip.

I plan to use this to modify my approach in unsupervised visualizations https://hackernoon.com/unsupervised-machine-learning-for-fun...

Impressive, works like a charm on Firefox/Linux.

This is awesome

D3 can use GPU shaders and is far more flexible. This is ok for canned vis, not for custom data vis. Like Vega vs D3. D3 wins every time.

What makes D3 more flexible?

Vega is built on D3. It's just operating at a higher level. It's like C++ vs Python. You can do anything in C++ you can do in Python, but it may take a lot more work.

This may be a bit confusing but Vega only uses some D3 helpers, it does not generate D3 code at any point. See https://vega.github.io/vega/about/vega-and-d3/ for details.

But you high-level point is accurate, D3 is more expressive since you can use all of JavaScript.

All I can say is finally. Death to d3.

I see d3 imported alongside stardust in this example: https://stardustjs.github.io/examples/daily-activities/

Edit: from their paper: "We see Stardust as a complement to D3 instead of a replacement. Stardust is good at rendering a large number of marks and animate them with parameters, while D3 has better support for fine-grained control and styling on a small number of items. For example, to create a scatterplot with a large number of items, we can use D3 to render its axes and handle interactions such as range selections, and use Stardust to render and animate the points"

What's wrong with d3?

It's not the easiest tool to learn

Doesn't that make it more fun/rewarding?

nope - d3 code is extremely imperative...

so after a couple of lines, they're somewhat unmaintainable to me...

but couldn't find anything that covers most of d3's features...

d3 is extremely functional. You can write it imperative, but it becomes unmaintainable pretty quickly.

It's very deliberately a lower level framework, for maximum flexibility. If you just want a bar chart in a line or five of code, there are many libraries which build on top of it.

Long live d3.js!

Reply on news.ycombinator.com