Indeed, OLAP cubes keep finding new life. The critical difference here is that there isn't a heavy query engine (just a Parquet reader) and a few http requests to pull from the cube. So it's light on the backend and light on the frontend.
384 karma · joined June 19, 2009
I provide caveats for when this would work vs. when it doesn't in the post. For a lot of customer-facing dashboards, I think it's probably pretty good.
This is one of the reasons I'm excited about DuckDB's upcoming [2]PEG parser. If they can pull it off, we could have alternative dialects that run on DuckDB.
[1] https://www.malloydata.dev/ [2] https://duckdb.org/2024/11/22/runtime-extensible-parsers.htm...
Observable's column summary feature is very nice! But I do think there's a very common lineage around these kinds of diagnostics which motivated both Observables and ours. See Jeff Heer's profiler paper[1] for more.
I'm very passionate about this area because I think "first mile problems" are underserved by most tools, but they take the longest to work out.
We had to do some gnarly things[2] to make this feature work well; and there's a lot of room to make it scale nicely and cover all DuckDB data types.
[1] http://vis.stanford.edu/papers/profiler [2] https://motherduck.com/blog/introducing-column-explorer/
https://hacks.mozilla.org/2019/04/pyodide-bringing-the-scien...
I've been using it internally at my current job to great effect since March, after I convinced my manager to let me try it on a one-off project. That small project has led to us using it for other bigger projects, again pretty successfully.
tl;dr – I probably wouldn't choose to go back to other frameworks after my time in Svelte-land unless I'm in a situation where I have to use React. I don't MIND React, but it's clear that all things equal it simply takes longer to make things with it and the thing I make tends to be harder to work on over time. Svelte is just too good and has probably saved my company some money in dev time & dev happiness. For reference, I work across the data stack – data engineering to stats / ml modeling to frontends / products that use data. It shines in all areas, but it REALLY shines in data viz.
pros:
- the time to first meaningful render in svelte is substantially, substantially lower than it is in React, and you face none of the drawbacks of using the React starter stuff (ie ejecting isn't a thing). This may seem like a weird metric, but I think it's pretty vital. I can easily start a side project w/ svelte and not have to think deeply about getting all the dependencies, all the webpack stuff fiddled with, all that. The ramp to productivity is pretty flat.
- you don't need the Friends part of React & Friends nearly as much, but you can definitely use the ones that are framework-agnostic (like immer, or redux, or whatever). W/ the bigger React projects I work on, I have really struggled with the development speed of 3rd party libraries forcing me to update my components and wasting time. Having fewer dependencies has made my life substantially better. Most projects I just use immer + d3 modules.
- I was pleased to discover that in practice, the boilerplate doesn't ever sneak in, even in complex components. This means that I can either (1) deliver the thing faster, and / or (2) in the same amount of time, make it that much better. It ends up being that both of those things happen.
- svelte stores are incredible. We tend to use them with immer to make redux-like app state work nicely, but we use stores for all sorts of things – contexts for complex components like data graphics (ie we need to share reactive changes arbitrarily to children), tweened and spring. One way that I enjoy using them – I take params from my app store (the one I make w/ immer) and create a derived store that encapsulates a cached dataset request. There are SO MANY cool patterns you can implement with stores that makes your app easy to reason about. Example of a svelte immer app here: https://github.com/hamilton/svelte-immer-example
- having a great story for both style & animation really makes you think that most other frameworks are feature-incomplete. The component authoring experience really makes html and css feel like first-class citizens again. I currently have a designer helping me tweak the css of a bunch of components I've written, and it's wonderful – he doesn't have to learn all the css-in-js stuff I've encountered in other frameworks.
- Svelte truly, truly shines with data visualization, where the reactivity has reduced the cost of authoring data graphics for me. I say this as someone who used to use Protovis and then spent a lot of time in D3-land. Hoping svelte gl lands publicly sometime soon so I can get off of the regl train.
- Svelte creates joy the way jQuery did in the past. I noticed a talk by Rich where he talks about jQuery. This has been how I've described svelte to folks @ my work, and they get it – it DID feel magical then.
cons:
- the testing story isn't well-documented yet, and it probably just doesn't exist yet. I tend to just move code that definitely needs to be tested into separate modules and test them there.
- typescript support would be really nice.
- the community is small, so you have to find answers yourself or make them. This may not appeal to a lot of corporate frontend devs, but it works @ Mozilla where I work, since most folks are pretty web-friendly. I personally prefer this since things that are hard in other frameworks are trivial in Svelte. That is, you just don't need that big a community to be similarly productive.
- your dev culture has to be right for this, and the risk of moving has to be understood. It takes time to figure out how new frameworks pay off.
Another aspect I have really appreciated is that I have substantially fewer dependencies, since there is more included in Svelte than in React. I tend to only use d3 utility functions (format, scales, etc.) & immer (which I pair with svelte stores to get something kind of Redux-y, which is great for other devs jumping into Svelte from React-land). Fewer dependencies means I'm less likely to encounter awful dependency hell.
Generally, I feel like every time I see a Svelte thread here, there's a lot of random noise, but it's good to listen to the folks who actually use it. There are situations where it makes sense, and situations where it probably doesn't. The main reason why I might not use it is ecosystem friction – a lot of devs I know would prefer to just use what they know, even if it is painful relative to Svelte & friends.
In the meantime, I would recommend walking through the tutorials & examples. They're very well written and I go back to them regularly to try out different parts. Try just making a simple line chart! Now make six on one page! Now try to link the rollovers for all of them! You'll probably learn a ton doing just that.
A few unvarnished takeaways from a data science / viz point of view that I was thinking of writing up, but will use HN as a sketchpad:
1. I don't really miss the React 3rd party ecosystem as much as I thought I would. Since most of what I do is data viz / data presentations, I really just need a few d3 helper functions, and most other things I can build myself w/ svelte's affordances. The biggest downside, however, is that there aren't any good accessible UX component libraries for Svelte, so I have built my own. This all said, I'm productively code-homesteading and I love it. Animating component lifecycles, component style scoping, and state management are 1st-class citizens, so these pieces are pretty high-quality, usable, and thoughtfully built.
2. my "time to first meaningful render" metric was at an all-time low with Svelte. Getting started on a new idea or project is far, far easier than w/ any React project I've ever done. Substantially less boilerplate means I can get much, much more done in the same amount of time. I feel this every time I jump back into the React projects I work on & maintain.
3. as my apps have grown in complexity, I've found that Svelte really holds up beautifully. The style/layout/behavior colocation strategy keeps the cognitive overhead of working on all the parts of multiple components to a minimum. The performance gains of Svelte, even with complex components and interactions, feels magical. Bundle sizes are bizarrely small.
4. as someone who feels CSS will absolutely outlive all of these frameworks, I think the design of Svelte really feels like the best of both worlds. And that's how it should be – css + markup should be first-class citizens.
5. the testing story isn't fully there yet, so I would say to that end, proceed with caution. I typically try to compartmentalize JS logic from presentation as much as possible, but perhaps you don't.
6. the reactivity parts shine everywhere and are fairly easy to reason about in all cases, but they REALLY shine when you're building complex visualizations.
7. this is an important one to me – building something in Svelte really tickles the same part of the pleasure centers of the brain that JQuery did for a previous generation (and d3 did for early data scientists). You just SEE the thing work, effortlessly, at a low cognitive cost, and everything fits together so nicely. I often start svelte-powered experiments and end with a great, reusable component. I think there is a large group of engineers who remember the JQuery days and sat out the React era because it just seemed like way too much to do simple things. I think Svelte is especially for them.
In sum, really highly recommended to at least try it. I wanted to find evidence to discount Svelte, but haven't found any at this point, despite my best efforts. There is probably a certain class of dev that should be trying it – one without a ton of organizational constraints, probably. But if you do data science and want to level up your data presentation skills, you should be using Svelte imo.