wattenberger

oh interesting! I'm using a basic Netlify setup so I never looked into this. I'll check it out when I get a moment, thanks for flagging!

I think I should have written more about this in the blog post. Creating data viz with d3.js is basically as low level as it's going to get for drawing charts in the browser. There are many higher level libs (most of which utilize d3.js functions) that make it easier, at the expense of customizability. For anyone who just wants a standard chart, there are tons of awesome libraries out there.

D3.js hasn't changed radically in the past ~8 years, mostly because it doesn't need to. The SVG API has been fairly stable (at least as far as what has been implemented across browsers), and the basics of visualizing data are static (eg. how to convert data values to physical values using scales.

so great to hear it, thanks for the kind words!

k is right that d3 has some helper functions for drawing on canvas, so most of the methods here will work:

https://wattenberger.com/blog/d3#drawing-svg-shapes

Canvas is way more performant than SVG for a lot of shapes, since they don't have the overhead of a DOM node.

But in general, you want to work outside of the browser if you have that much data. What I'll usually do is either:

- write a python script to process my data and use d3 for the final visualization, or

- have an API that returns a sliced portion of the data and create an interface that will request different sliced datasets when interacting with filters, controls, etc.

This election dashboard is a good example:

https://currents.parsely.com/election

it hits two endpoints, which return basically the maximum amount of data you can expect a browser to handle (the massive dataset is processed elsewhere). You could create an endpoint to return different views when you click on, eg, a different date, although then the user would have to wait for the data to load.

ah! thanks for flagging that, just fixed it.

I totally agree - especially when I consider how long ago it was created (early 2011, more than 8 years ago) and how quickly most things in the Frontend web world age.

Exactly! I should probably have a section (or at least a link to a resource) that talks about _why_ you would want to learn this.

I have an image in the book detailing the spectrum of data viz tools on the web. At one end are the quick-pickup tools that aren't customizeable, and at the other end you have.. d3. Which isn't really a framework so much as a collection of tools that can help with making web charts.

What I probably failed to communicate in this post is that there are tons of opportunities to help communicate data with the web. Tooltips are just the most common interaction, but you could easily envision using scroll as a trigger, or adding a way to "zoom in" on parts of a chart.

The learning curve is definitely steeper, but once you have the skills, the world is really your oyster for visualizing data online. The pudding (https://pudding.cool/) is a great showcase of possibilities, or even uncommon chart types like at https://wattenberger.com/fishing

hey thanks! I really appreciate your kind words

I'm really interested in using this amazing tech most of us have at our fingertips to try new ways to learn things. There have to be so many avenues we haven't yet explored, and I find that so exciting.

thanks! I really appreciate it

Interesting! I'm actually a big fan of the font. Fonts are a weirdly personal choice - I'll think on adding a font toggle, since the whole point of the article is to be read.

If that's the worst thing you found, I'm stoked!

I was hoping that only happened at few enough window sizes that not many people noticed. This is an unfortunate side-effect of using iframes to show inline examples. There are a few solutions - I'm hoping to have time to work through one of them this weekend.

Thanks for flagging! This post was a bit of a WIP and I had just moved the examples to their own repo last night :) Fixed and updated

Hey thanks! The code is all up at https://github.com/Wattenberger/blog/ for anyone who's curious.

There are some interesting bits in there - the basic structure is:

- the post itself is at src/Interactions

- the code changes using a ScrollEvent component (src/_ui/Code)

- the code examples (inline and fixed on the right) are using the Code component (src/_ui/Code)

- the inline demos are using the LocalExample component (src/_ui/LocalExample). The LocalExample component basically builds an iframe and displays it

Both of these components support:

- a list of lines that are removed

- a list of lines that are added

I've been liking that this setup lets me have complete examples that work as standalone web pages, but I can also edit them on the fly and work through changes on-the-fly. Which lets me talk about interim states [like this one](https://github.com/Wattenberger/blog/blob/master/src/Interac...).

I'm happy to answer any questions about it, the code is a bit messy :)

I totally agree that some of the data is suspicious. I did spend some time cleaning the data, and removed any data points that looked too unrealistic. That particular one actually was discussed in the original thread (https://news.ycombinator.com/item?id=8573423), and I chose to leave it in because it's definitely feasible.

There are lots of unavoidable issues with self-reported data sets, which is partially why I added the list on the right to increase transparency and let people make their own decisions about confidence in the data.

Umbel — Austin, TX

--

Umbel is building the infrastructure by which companies own, control and access their consumer-centric data. Our platform enables customers to leverage their data rights and create tangible business value. We take employee satisfaction and growth seriously. We also expect a lot in return. We are currently seeking top-notch talent to join our fast-growing and dynamic engineering organization at several positions:

* Python Developer

* Operations Engineer

* Front End Developer

* Go Software Engineer

--

More information about Umbel and the opportunities we have available: https://www.umbel.com/engineering/?source=Hacker+News