Comment by monfera | Hacker News Reader

Comment by monfera | Hacker News Reader

Parent Full thread

Fantastic work! D3 has a large API surface area, due to the functionality it covers. So even a relatively conservative major upgrade is a TON of work, and some of it might not have been pure fun only.

1) What's your opinion on the apparent case that the bulk of changes come from you, as society promotes teamwork so much? Is it a case of 'small team efficency'? (Btw. I know of others' contributions too, and reliance on Rollup, ColorBrewer, matplotlib/Viridis etc.)

2) What kept you going? It's probably at least a person-year of work just on your side that's not paid for by companies, not to mention some kind of startup bet. Have you alternatively toyed with the idea of starting a value-added layer, since so many for-profit organizations benefit from your work?

3) While it changes the API, and most everything is rewritten, it's IMO a conservative upgrade in that it doesn't steer away from familiar concepts and structure. It still does DOM binding, transitions, layout etc. fairly similarly. Have you considered much more disruptive departures? As an example, React's DOM diffing is more radically different from D3 selections than the change from D3 3.* to 4.* or the Grammar of Graphics as an API concept is more radically different from D3 than the 3.* -> 4.0 API changes. Or something like moving away from the 'functional objects' concept in favor of curried, fixed-argument pure functions, perhaps prefaced with the familiar chaining API would have been a large change (these are mostly examples rather than wishlist). What were YOUR ideas, had you wanted more disruptive changes, or 'D4'?

1. That’s a difficult question. There is an efficiency that comes from working with people you already know--maintaining a smaller set of relationships--rather than trying to manage a lot of small contributions from many. Jason Davies made huge contributions to D3 3.0 (in particular d3-geo), but alas I haven’t heard from him lately.

Most external contributions to D3 tend to be either bug fixes or new features. Bug fixes are often merged quickly, but new features require careful assessment. Each addition to the API has a complexity cost: it makes the library bigger; it has to be documented; it may instigate support questions and issues; it may have ripple effects on the development of other features. I’ve long taken Josh Bloch’s advice to heart and try to maximize the “power-to-weight” ratio of my APIs, and to try to keep them as small as possible while still expressive. Part of the goal of 4.0’s modularity is to make it easier for others to release their new features as standalone libraries that they can use with D3, rather than it needing to be part of “core”. There isn’t a “core” D3 anymore; there’s just a default bundle. (And if we ship a web-based tool for rolling your own bundle, the barrier will be even lower for plugins.)

2. Yes, I’ve been working on D3 full-time for the last year, since leaving The New York Times. Partly I am playing catchup. D3 has been more popular than I could have imagined, and I want to make sure that it’s as good as I can make it. I loved working for the NYT (and I miss my colleagues dearly), but it was hard to think deeply about the design of D3 when you’re on deadline. I am interested in finding a way to make D3 development self-sustaining financially, as I cannot do this for free indefinitely. Prior to launch today, I have been focused on shipping 4.0; now that’s it’s released, I am thinking more about what’s next.

3. Yes, it’s not a huge departure despite the many changes and the new implementation. I hope that’s a testament to the strength of the original design… or perhaps it’s just my confidence in it. (There is more different between Protovis and D3 than between D3 3.x and 4.0.) There is not only one right tool for visualization; the right tool depends on the application, such as whether you’re designing bespoke custom news graphics, exploring a dataset of the first time, or doing realtime monitoring of your network. D3 is intended to be the lowest layer of visualization tools: the visualization “kernel”, or “standard library”. You can build higher-level abstractions on top of D3 that is more tailored to your application. One of the areas I am considering exploring is higher-level abstraction, perhaps more along the lines of Grammar of Graphics.

IIRC a key point of d3 was to not be declarative, like the gog. That was more an influence in protoviz, its precursor.

You might be interested in Vega and Vega lite if you want to go declarative.

I'm contributing to plotly.js right now, which is also declarative, and I see the pros and cons of levels of denotative semantics (mostly depends on goals and library user requirements) but it was just an example for what I mean by more radical changes. Good you mentioned it anyway, protovis -> D3 also represented a rather disruptive change. But perhaps I was late with these questions for this specific AMA.

Reply on news.ycombinator.com