wattenberger

I hadn't seen it until today - looks like a useful product!

Spot on! It's a space I've been wanting to explore, and so have been tinkering with it this last week while we're in between larger projects. I definitely don't see it being a product in it's current gotten, but wanted to share the exploration! Although I will say that the more I've been visualizing different repos, the more convinced I am that it's more useful than is easy to convey.

oh no! My best guess would be that there are many element-heavy SVGs on the page. I was hoping for better performance, since the files will almost definitely be smaller than large raster images, but there are likely older browsers that have trouble with that. I'll look into it, thanks for the report!

There are definitely many trade-offs with using a circle pack layout - I snuck a bit of the reasoning into the collapsed section halfway through the write-up! Overall, this layout worked best for me, with the nesting feeling very natural and the circles feeling very "atomic".

But big picture, this write-up isn't tied to the current visualization! It's more focused on exploring _how_ a visual representation could help our understanding of codebases. There are tons of jumping off points, including different vizes!

Hmm, it should be loading! What OS/browser are you on?

We always look at our code in a file/folder list - I explored an alternative way to view codebases, showing a bird's-eye-view of its structure. This write-up walks through the motivations, ways to use the visualization, and potential future directions (there are many!).

There's also an interactive tool to check out your own repos and a GitHub Action if you want to integrate a diagram into a README.

I also added it after your first comment so might have snuck it by you ;) I'm curious what it is that you're seeing, if you have the chance to leave an image!

interesting! I'm seeing them grouped in the middle. It's a tricky layout, using d3.js's circle packing algorithm, then recursively using a force layout to relax each folder's contents.

ah shoot, thanks for the updated link!

I probably need to add more context to the page, but I want to specify that my intention with this tool wasn't that long sentence == bad. I was inspired by this image:

https://twitter.com/misscrisp/status/1202792895448662016

It's really interesting how different authors play with the rhythm of their sentences - one of the best ones on here is The Raven, with very short sentences interspersed with longer ones.

Sounds similar to this guide I put together - understanding how the different modules work together and who is responsible for what is really helpful.

https://wattenberger.com/blog/d3

Repeating a comment I made above, this semi-finished blog post I'm working on might be helpful:

https://wattenberger.com/blog/react-and-d3

I generally like to stick with React for DOM manipulation, and use D3 like a utility library.

one thing that might help is this blog post I'm (slowly) putting together:

https://wattenberger.com/blog/react-and-d3

I've done a lot of work that uses d3 within a JS framework (React, Vue, Svelte, & Angular), and the key is to understand 1. how to create shapes in plain SVG, and 2. how to use the parts of d3 that don't do DOM manipulation. I've found the least-hacky way is to handle DOM manipulation with the framework, and use d3 for things like data manipulation, creating SVG path strings, creating scales to map from the data domain to the visual properties, etc.

There are tons of different d3 modules (40+). As someone who uses d3 extensively, but rarely uses its selection and data binding functionality, I put together a birds-eye view of the different modules. There is tons of great functionality that is usually skipped over in favor of the DOM-manipulation methods, like managing colors, dates, data munging, etc.

https://wattenberger.com/blog/d3

I really love creating things with SVG + React! I usually will use that until things start to jank up, and then switch to Canvas, which is more annoying for interactions and debugging.

You can get pretty far before that happens - for example, here's a post representing every country in SVG. https://wattenberger.com/blog/d3#maps-and-globes

ah, the joys of frontend web development!

I added a 30ms debounce, just to help out a bit. I'm seeing some serious jank in Firefox, which smells like the beginning of a rabbit hole. Hopefully I'll find some time to dig in there.

I totally agree! I'm planning a video course on SVG, since there really aren't many good resources on it and I find it really powerful, and also fun!

Good question! I'm trying to find a good way to explain this bit in the guide.

This is due to the preserveAspectRatio property on the svg element. Its default value is `meet`, which scales the svg with three rules:

- aspect ratio is preserved - the entire viewBox is visible within the viewport - the viewBox is scaled up as much as possible, while still meeting the other criteria

https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/p...

If you fiddle with the attribute value in the Dev Tools, try setting it to `xMinYMid` or `none` to see if that gives you a good sense of how it works. I'm worried that it might just confuse if not explained well :)

I share your opinion! And that's actually partially why I wrote this post - it's really great to know the different utility methods of d3.js to help render a data viz (using React, etc).

it really depends on your use case! d3 is basically as close to the metal as you're going to get with visualizing data in a browser. It lets you build custom, interactive visualizations, and really is just a rocket booster. If you just want a simple chart and don't need much customization, there are tons of great higher level libraries that are easier to use.

previous experience with javascript is helpful, but not at all required. I spent a lot of time working in resources for people new to web development so that everything would be accessible and to keep readers on track.

Thanks for the feedback! There were so many rabbit holes that I had to prevent myself going down - I scope creeped this article enough as it is!

I also specifically didn't want to go into the data join patterns for two main reasons:

1. there are lots of other articles that talk about it in depth, and 2. these days, I largely don't use it at all.

This is partially why I wanted to write this up, because the DOM manipulation parts can seem like an integral part of D3, but there are really only 2 modules devoted to it. I mostly use D3 for its utility methods, and a JS framework like React to handle all of the rendering, and I really want other people to be able to discover those functions without first "learning d3".

Good question! This is meant more as an overview, whereas the book dives right into practical examples, while gradually weaving the basic concepts in. The book is a lot more extensive and detailed, and gets into data visualization fundamentals and more advanced concepts.

If you're curious, the first chapter is free to download and gives a good sense of the flow:

https://www.newline.co/fullstack-d3

so true. I think this is one of the downsides of d3 being so prolific in the past.

all examples on Observable are up-to-date, so searching on here could be useful:

https://observablehq.com

and if you find a bl.ock that you really want to port, this changelog could be helpful:

https://github.com/d3/d3/blob/master/CHANGES.md#changes-in-d...

most of the changes are simple name changes that put the methods directly on `d3`: eg. from `d3.scale.linear()` to `d3.scaleLinear()`

Most d3 examples will have a "draw" function that you call every time the data is updated. You basically bind your dataset to SVG elements, then D3.js will diff the new data with that bound data and decide what to update. A good intro to the concept:

https://bost.ocks.org/mike/join/

I personally usually am working in a React app, so I would use that to render (and update) the chart.

However, that's a pretty complicated example to start with! You might be better off using one of the higher-level libraries, like this one:

https://apexcharts.com/javascript-chart-demos/candlestick-ch...

There are a ton to choose from (just google `js candlestick chart`), and they can be pretty customizeable.

that makes total sense, and I love the concept. (even though I also want the recording of the presentation!)

this is really great, thanks for sharing! I like the linked Papers We Love slides - wish I could have seen the presentation!

it is! and thank you for submitting this to hn, I really appreciate it

To give you a bit of context, the _large_ majority of people buy the advanced package (which really surprised me). The advanced package really just tries to bring all of your new knowledge together, going over d3 + React and d3 + Angular. It also has detailed walkthroughs of 3 complex data vizes, which could be really helpful. But both versions give you a great foundation to build off of.

thanks! I'm always open to requests for new posts as well, if there's anything you want to read about.

status_quo69's answer is great, just here to add a few thoughts.

This is largely why I wanted to write a post like this - often "using d3.js" is what we think we need to do, but I wanted to show that most of the API is just utility functions. Here are the two modules devoted to manipulating the DOM:

https://wattenberger.com/blog/d3#manipulating-the-dom

I recommend (and follow) that you render elements like you normally would, and use things like d3 scales or d3.line() to convert your data into the physical aspects that are needed. I find it really hacky to do it any other way. I want to explore more, but I did a quick test to see which is more performant (letting React render a scatter plot, or letting d3 render a scatter plot), and pure React was 4x faster:

https://twitter.com/Wattenberger/status/1123413424678027265

Of course, this was just one example, but it agrees with my hypothesis.

I do have a whole chapter devoted to d3 + React (and one on d3 + Angular) that might be worth checking out. I think it largely comes down to understanding your tools and being able to pick the parts of d3 that will be most useful to you.