texodus

I work on perspective, glad you dig it! We just released 1.0 this week after 4+ years of open-source-first development.

Perspective was, in a previous life, a proprietary internal engine for Python/desktop applications at JPMC, that was ported and open-sourced to FINOS as a web assembly based browser component. Open sourcing perspective had a huge impact on the project, not just from the awesome community contributions and engagement and support from FINOS itself, but (as a huge surprise to me) the perspective project's visibility within our company.

Yew has an `html!` macro and a very React/Elm like architecture https://github.com/yewstack/yew

Transport Tycoon (Jun 1994) was published over a year before the first version of DirectX (Sep 1995).

https://texodus.github.io/mtg-perspective/?seasons-in-the-ab... https://github.com/texodus/mtg-perspective

A Magic TG (the popular card game) deck builder and card screener analytics app. I actually remember virtually nothing about Magic, I just built it to test-drive the analytics web UI! Built on Perspective, Apache Arrow to encode the entire Magic card database in a single 20mb file so I can embed it in a browser, and some great enthusiast web services for decks, card meta data, symbology and images.

Perspective is developed at JPMorgan https://github.com/finos/perspective/

WebAssembly cannot utilize UDP, or any other APIs not exposed by the Browser - just HTTP and Websocket in this case. However, Perspective can read Apache Arrow using the C++ client library without decoding the data structure to an intermediate JS representation first, which in our benchmarks has a substantial impact on overall update throughput.

Apologies, this was fixed a few days ago but hasn't made it into a release yet. https://github.com/finos/perspective/pull/1065

My $0.02, this was mostly due to historical lagging a more out-of-the-box 'complete' solution like JSX/React, specifically in:

(1) Poor native browser support until recently, Firefox originally refused to support Web Components v0 at all (now called Custom Elements in v1, they are supported in all evergreen browsers mostly).

(1a) Available polyfills were initially buggy and had severe performance issues on older browsers.

(2) Lack of proper templating, though I personally quite adore template literals and approaches like e.g. `lit-html`.

(3) Lack of robust styling API - custom styles felt coarse and tacked on in v0 especially, and were poorly supported by pre-processors like LESS/SASS.

(4) Lots of impedance mismatch issues connecting standard JS/HTML/CSS libraries with the Shadow DOM.

Here, I made a quick dump of this data pivoted by Senator/Date/Side into Perspective, a wasm pivot-table project I work on:

https://bl.ocks.org/texodus/8fe1596e298720dff9e1051668e240ce

https://perspective.finos.org/ supports simple SQL-like operations, group by, filters and sort e.g.

Also from JPMorgan https://perspective.finos.org/

Try https://perspective.finos.org/ (example https://bl.ocks.org/texodus/77fb8ef87155b3a7fe341932bfc33382 )

WebAssembly engine can handle 1mm+ rows, and can be run virtually in node.js, Python or C++. Does filtering, pivoting on multiple axes, sorting, linking, click events, and more.

Good rec! Perspective was purpose built for this precise use case at JPMC, and has some unique features such as streaming Apache Arrow support, which can pretty easily handle datasets in the low millions of rows, at thousands of updates/second.

Source - I work on perspective.

You may want to take a look at [perspective](https://perspective.finos.org/), a WASM-based serverless dynamic visualization library with [D3FC](https://d3fc.io) charting.

Might I suggest giving Perspective.js a look? Supports many of the same visualizations as Muze (and some it doesn't, specifically datagrids), is user-configurable, written in WebAssembly (C++) for extreme performance, and can run trivially on the server via node.js - there is even a CLI version:

https://github.com/jpmorganchase/perspective https://github.com/jpmorganchase/perspective/tree/master/exa...

It is virtual in the sense that it does not realize the entire dataset in the JS side of the WebAssembly bridge - this is a performance optimization but entirely in-memory. While you can run the engine in node.js and use it to efficiently stream updates to a symmetric engine in the browser, we do not currently implement server-virtualized views.

Though, we have quite alot of experience doing this in the past - and the design of Perspective is very much a reaction to what we learned, at lesat in regards to the typical financial dataset which is much smaller than 1TB.

Yes, this is the library we use.

We have met before actually, you did a demo at JPM in midtown several years back. Graphistry has come a very long way since then - impressive work!

There is an early arrow example in the examples package - superstore-arrow.html. The idea is that, instead of converting your data to internet->text->JSON->ArrayBuffer, you just keep the data in binary and write it directly into the C++ heap ArrayBuffer as-is. We currently do not read this in C++ directly for various reasons related to how emscripten allocates memory, but the general idea is the same.

The former. In practice, it seems some runtimes are still a bit buggy - so we also revert to ASM.js if e.g. you are on an iPhone

Not through the UI yet, only in the engine API - but the pivoting itself is quite fast, so should still be suitable for drilling down "on the fly" so to stpeak

The boundary-crossing is definitely the bottleneck right now. We are currently putting alot of work into the Apache Arrow support specifically to avoid this crossover, which will allow us to send data from the server in binary and avoid parsing in the browser.

We use Web Workers principally to separate data CPU load from rendering load, as the datasets we deal with update very frequently and are quite large, and e.g. Highcharts can take on teh order of 100ms+ on detailed charts.

We have some light benchmarks we use for regression testing, but definitely need more work in this area

[benchmarks](https://jpmorganchase.github.io/perspective/examples/benchma...)

Plain web components. The UI is quite light in terms of functionality so far.

We chose to go with a Web Components based interface for compatibility across frameworks, but alot of where we go in the future will be determined by the expertise of the developers we hire to work on it, and the community if there is interest there.

We do some benchmarking just for catching performance regressions and validating assumptions against the non-WebAssembly version, but definitely need more in this regard. [benchmark for wasm build](https://jpmorganchase.github.io/perspective/examples/benchma...)

It supports virtualization, and it is utilized by both plugins (Hypergrid and Highcharts).

N levels of pivoting on both axes.

Should work on all browsers - if not, please open an issue!

Ah right - texodusmedia at gmail dot com should work for now.

I don't know the Cakeshop developers personally, but it looks slick!

We very much plan to continue developing this entirely in the open! It is reputationally important we do not just dump our dead projects on the internet as Open Source, which is one of the reason why we chose Perspective for this project in the first place.

Right now, you can call a load() or update() method on an engine or widget to pass JSON, CSV or Apache Arrow data (as an ArrayBuffer) to the engine.

Apologize for the documentation clarity - this is our first release and very much under active construction!

The frontend is actually pure JS/HTML - the WebAssembly part is just the data engine, and runs in a WebWorker for CPU isolation.

Hypergrid is a canvas-based renderer, yes.