mbostock

I don’t think the `npm install` will scare people away (Evidence uses that, too), and we’ve definitely tried to make the onboarding process as guided as possible (shout-out to clack.cc for a great CLI prompt library):

https://observablehq.com/framework/getting-started

And plus you can import libraries directly from a CDN rather than needing to use npm or yarn to manage dependencies. (Though we plan on supporting the latter in the future, too.)

https://observablehq.com/framework/javascript/imports

See this example for getting started with Python:

https://github.com/observablehq/framework/tree/main/examples...

But of course we’d love to add more affordances and documentation for other languages. We’re naturally biased towards JavaScript as our focus has historically been on visualization, but I like to think we’re making progress on the polyglot dream.

Haha, my compliments.

The Observable Framework CLI supports a `convert` command for downloading an Observable notebook and converting it to Markdown. E.g., `observable convert @d3/bar-chart` will download the notebook and save a bar-chart.md and alphabet.csv to your current working directory. (We did it from the command line so it’s easy for you to automate or batch-convert or refresh and keep things in sync.) You may have to make some tweaks to the code due to Framework’s vanilla JavaScript syntax, but we’ll work on making that more seamless over time.

And you can `observable deploy` to deploy your app to Observable for sharing — though most often you’ll want to setup continuous deployment to keep your app up-to-date automatically.

More developer-focused, and yes, you can use git for version control and develop locally, setup continuous deployment, and self-host apps anywhere.

1. We don’t have immediate plans to bring other languages to the front-end — maybe TypeScript, but that’s just stripping annotations; maybe some WebAssembly. Our idea is to have a clear serializable “membrane” separating your back-end (in any language, running on build on your servers) from your front-end (in JavaScript, running on load in the client). Data loaders produce data during build, which gets handed-off to the client to render. Trying to do data processing on the client is often a frustrating and poor user experience. Likewise trying to render great interactive charts without web technologies is quite limiting!

2. I can’t speak to Quarto’s plans. Observable Framework is open-source so they might pick up some of this stuff. I look at Framework more as an alternative to Quarto than a complement.

3. As the creator of Observable Plot (and D3 before that), I’m a huge fan of visualization grammars! Apache Echarts is a chart typology, and while it’s got a lot of chart types in it, it has no overarching conceptual model of how to represent a visualization. And so it’s not very interesting. But “the proof of the pudding is in the eating” as I say in the post, so I encourage you to look at Observable Plot and decide for yourself if you like both the syntax and the resulting plots. I certainly do!

Leland Wilkinson said it best: “If we endeavor to develop a charting instead of a graphing program, we will accomplish two things. First, we inevitably will offer fewer charts than people want. Second, our package will have no deep structure. Our computer program will be unnecessarily complex, because we will fail to reuse objects or routines that function similarly in different charts. And we will have no way to add new charts to our system without generating complex new code. Elegant design requires us to think about a theory of graphics, not charts.”

Hosting & compute — operationalizing/productionizing data apps. Observable Framework is open-source, but our hope is that we offer a compelling complementary paid service for you to host your (typically private) data apps on Observable. We make it easy for you to share data apps securely with your team or customers or clients or whoever, and manage the complexities of keeping your app & data up-to-date with continuous deployment, scheduled builds, access control, collaboration, monitoring, analytics, etc.

We’re working on zooming and panning for Observable Plot (https://github.com/observablehq/plot/pull/1738) and other interactions such as brushing (https://github.com/observablehq/plot/pull/721) — all of this is already possible, we just haven’t packaged it up in a convenient way yet (https://github.com/observablehq/plot/pull/1871). And as skybrian pointed out, you can also get interactivity “for free” with Observable’s reactivity and re-rendering.

We’ve been focused primarily on the static display of visualizations because that’s what viewers see first, and often that’s often the only thing they see. Relying too heavily on interaction places an onus on the user to find the insights; a good display of data should be opinionated about what it shows and guide the user to what is interesting.

We’re not trying to convince you to switch to JavaScript here — a main value prop of Observable Framework is that you can write data loaders in any language (Python, R, Go, Julia, etc.). So do all your data preparation and analysis in whatever language you like, and then do your front-end in JavaScript to leverage the graphics and interactive compute capabilities of modern browsers. It’s pipes and child_process.spawn under the hood. And you still get instant reactivity when you save changes to your data loaders (when you edit Python) because Framework watches files and pushes new data to the client with reactive hot data & module replacement.

And you can compress (aggregate or filter) the data as much as you like, so it’s up to you how much data you send to the client. For example your data loader could be a minimal CSV file that’s just the numbers you need for a bar chart. Or it could be a Parquet file and you use DuckDB (https://observablehq.com/framework/lib/duckdb) on the client to generate dynamic visualizations.

Another tidbit buried in this announcement is that Observable Framework is 100% vanilla JavaScript syntax — so you get Observable’s reactive runtime without the quirky Observable JavaScript syntax (as in Observable notebooks). And you can use static ES imports from npm or local modules, declare multiple top-level variables in a code block (not just a single named variable per cell), call the built-in display(…) function to put things on the page, etc. It’s a huge relief to have vanilla syntax and greatly improves interoperability. And we’re figuring out how to port these improvements back to Observable notebooks in the near future.

Yes! It first felt counterintuitive and constraining to prepare data ahead of time, rather than just loading whatever you want on the fly. But we’ve found this to be a great discipline in practice because it forces you to think about what data you actually need to show, and to shift as much of the compute to build time to reduce file sizes (through aggregation and filtering) and accelerate page load. And once you become accustomed to instant dashboards it becomes unthinkable to go back to slow queries.

Yep, Evidence is doing good work. We were most directly inspired by VitePress; we spent months rewriting both D3’s docs (https://d3js.org) and Observable Plot’s docs (https://observablehq.com/plot) in VitePress, and absolutely loved the experience. But we wanted a tool focused on data apps, dashboards, reports — observability and business intelligence use cases rather than documentation. Compared to Evidence, I’d say we’re trying to target data app developers more than data analysts; we offer a lot of power and expressiveness, and emphasize custom visualizations and interaction (leaning on Observable Plot or D3), as well as polyglot programming with data loaders written in any language (Python, R, not just SQL).

Hey, HN. We’re thrilled to release Observable Framework today — a new open-source tool for developing data apps. I highly recommend viewing this example report adapted from our internal dashboard analyzing web logs:

https://observablehq.com/framework/examples/api/

This technique of “just plot everything” (7.6M requests as a scatterplot) has revealed surprising insights we’ve used to optimize our servers and better control traffic. We’re also sharing a more traditional dashboard that visualizes the adoption of our open-source visualization library (and in some ways the successor to D3), Observable Plot:

https://observablehq.com/framework/examples/plot/

In addition to releasing Observable Framework, we’ve also made Observable free again for individuals (including private notebooks and databases connectors). Let me know if you have any questions!

What do you mean by “so little of the internet uses d3 or similar interactivity”?

Jason also made a spherical Voronoi map with a dataset of world airports: https://www.jasondavies.com/maps/voronoi/airports/

There’s an Observable version here if you want to see an implementation using d3-geo-voronoi: https://observablehq.com/@d3/world-airports-voronoi

Yes! We’re trying to make it easy (convenient/fast) to make readable visualizations. If it’s tedious to add a tip, then you won’t do it as often, and you might miss an interesting observation.

Thanks, Simon! We’ve been working our way through the most asked-for features. This was #2 after TypeScript declarations which we shipped in 0.6.5.

Support for interaction was the biggest conceptual piece left to declare “1.0”, so I’m thrilled by the progress we’ve made. This is the culmination of a couple years of thinking and experimentation! Fil got brushing working a few minutes after the release, so I suspect we’ll see brushing and other selection modes in the near future. (Sneak peek: https://observablehq.com/@observablehq/plot-brush-interactio...) We also want to allow panning & zooming and declarative animation, but I expect those to take longer.

I’ve updated the Getting started guide to give examples of both client- and server-side rendering, and there are links to CodeSandbox for code you can copy.

React: https://observablehq.com/plot/getting-started#plot-in-react

Vue: https://observablehq.com/plot/getting-started#plot-in-vue

In what sense? All the examples on the Plot documentation site are server-side rendered?

Check this out: https://juba.github.io/pyobsplot/

It’s a vanilla JavaScript library and you can use it with any framework or site generation tool that you wish. You’re welcome to copy the Vue component we wrote for our VitePress site, or to adapt the same technique for other frameworks. You can supply whatever DOM implementation you like using the document plot option as described here: https://observablehq.com/plot/features/plots

As I mentioned in my other comment, this documentation site is built with VitePress, which is SSG, and we use a Vue component to render the plots statically so that you don’t get any reflow on the client.

We built Observable’s chart cell on top of Plot (the auto mark).

You can read more about our chart cell here: https://observablehq.com/@observablehq/chart-cell

And Plot.auto here: https://observablehq.com/plot/marks/auto

I expect we’ll release tooltips in a few weeks; I linked that newer PR in another comment. We’ve been exploring interaction while working on many other things (https://github.com/observablehq/plot/blob/main/CHANGELOG.md). Now that we’re feeling good about the foundation for static charts we are starting to address interaction.

This might help: https://observablehq.com/@observablehq/plot-vega-lite

More generally, our motivations are here: https://observablehq.com/plot/why-plot

If you’re primarily interested in static charts (or your interactivity is driven by external controls, such as drop-downs and sliders, where you can regenerate the chart as needed for interaction), then Plot is ready for use. On the other hand if you need direct manipulation interaction, such as brushing or painting dots in a scatterplot, then you might stick with Vega-Lite for now.

We’re working on tooltips now, and interaction next. There’s a PR here with an early sketch and demo video: https://github.com/observablehq/plot/pull/721

Slightly off-topic, but I wanted to share what a joy it was to use VitePress to build this new documentation site. We already use Vite for Plot development: instant hot module reloading is fantastic for iterating on tests as you can instantly see the visual effect of your code changes. VitePress brings the same experience to docs, and the default theme (which we use with minor adjustments) is beautiful out of the box.

For this site, we wrote a little Vue component to support server-side rendering for Plot (as part of static site generation at build time). There’s just a little adapter to convert Vue’s hyperscript to the standard DOM API here:

https://github.com/observablehq/plot/blob/main/docs/componen...

The end result is that all the Plot examples are authored simply as fenced js code blocks, but we get server-side rendering so the plots are already rendered when the page loads with no reflow! You can see an example here:

https://github.com/observablehq/plot/blob/33c5560e24aee08cdb...

(For plots that would be too expensive to serialize to inline SVG, we render them client-side-only, and use requestIdleCallback or an IntersectionObserver to defer rendering to avoid jank.)

Add to that, while we were writing the docs, the VitePress team incorporated minisearch for instant search with no dependencies on external services. Huge kudos to Evan You and the VitePress team for building such a great product! It’s a huge boon to the open-source community to make it easier to write beautiful and usable docs.

Oh. And be sure to try dark mode!

We haven’t implemented it yet, but that’s what we’re working on right now! There’s a PR up here (we just took a small break to focus on TypeScript declarations and then the new documentation website):

https://github.com/observablehq/plot/pull/1304

In the meantime, you can do lots of interactive stuff with the SVG that Plot generates. There’s a nice tooltip plugin by Mike Freeman here:

https://observablehq.com/@mkfreeman/plot-tooltip

Here’s an example using D3 to add interactive tooltips to Plot:

https://observablehq.com/@mkfreeman/plot-tooltip

Here’s one that lets you click on a region in a line chart to select it:

https://observablehq.com/@skybrian/select-from-predefined-re...

Here’s using rough.js to stylize a plot:

https://observablehq.com/@fil/rough-plot

Thanks, and great questions!

I like having multiple tools that make different design trade-offs. There’s no “one ring to rule them all.” For analysis I like to start with a high-level tool so that I can explore quickly and see if the data is interesting. I can then “eject” to a lower-level abstraction if I reach the limits of expressiveness in the higher-level tool and determine it’s worth the additional effort to drop down.

Observable Plot is optimized for fast visual exploration: it’s concise, with good defaults, and lots of built-in conveniences for transforming data. In this regard Plot is similar to Vega-Lite and other high-level tools based on the Grammar of Graphics. Whereas D3 is optimized for bespoke presentation, like the sort of work we were doing at The New York Times; D3 prioritizes expressiveness.

I recommend Plot for most people over D3 because it’s more efficient: more of your time can be spent deriving insight from your data rather than programming. But there are still plenty of things that Plot can’t do yet (without plugins), like brushing a scatterplot, panning and zooming, and animation. We’re working on those!

And yep, Plot is built on D3. It’s nice to not have to re-implement the low-level visualization primitives like scales and layouts. Plot generates SVG output, so you can readily manipulate what Plot generates using D3 selections. Also, Plot is highly extensible: you can implement custom mark types, custom symbols, custom transforms, etc. Here are two examples, the first to apply some custom clipping, and the second to design a custom symbol for a dot map:

https://observablehq.com/@fil/plot-difference-chart https://observablehq.com/@observablehq/plot-spike-map

We haven’t fully documented how to write custom marks in Plot yet, but we plan to do that (along with animation and interaction) prior to the 1.0 release. There’s a ton to do but it’s exciting and we love hearing feature requests!

I love Graphviz! It’s in the Observable standard library as the dot tagged template literal, powered by Viz.js.

https://observablehq.com/@observablehq/dot

Here’s a recent example where I used it to figure out why certain dependencies are in my node_modules:

https://twitter.com/mbostock/status/1479693504511053824

Plot generates SVG. There’s a variety of tools you can use to convert SVG to PDF. Plot is intended to be used on the web so I don’t think we’ll add direct to PDF rendering. (Though it’s possible that Observable might support it — Observable already supports download options that render SVG to PNG for example.)