My Problem with the Four-Document Model
hillelwayne.com
hillelwayne.com
> In short, the 4doc model is not universal. It was designed for documenting tools and doesn’t fully address the needs of frameworks and programming languages.
I think that the author is onto something here, but I'm not sold on the exact example that the author provides (programming languages). Honest question: are the Python docs deeply loved by the Python community? In other words, should we be holding up the Python docs as an example of great docs and therefore an example that the 4doc model is inadequate? I've done a fair bit of Python programming myself. In my experience the Python docs are decent: not great, not terrible. If we look at the Rust docs, which seem to get much more praise than the Python docs, then the author's argument seems to break down for me a bit. Because the Rust docs seem to be more aligned with the 4doc model than the Python docs, and the Rust docs are more praised than the Python docs.
One domain where I recently saw the limitations of the 4doc model is OS bringup. I was writing docs explaining how to get an OS running on new hardware. Customer feedback suggested that the ideal doc would start off as an overview, and then morph into a guide, but the guide section would also need constant small explanations of conceptual stuff.
I also agree with the "it's not comprehensive" premise but again the example fell flat for me. Conceptual overviews totally overlap with explanations. I can elaborate if needed but I honestly think the author is imposing a narrow definition of explanations that does not exist in the 4doc model.
The main instances where the 4doc model is glaringly incomplete for me are homepages and release notes. You can kinda bucket homepages under explanations or references. And you can kinda bucket release notes under references. But those classifications are a stretch. The fact that practically every technical project needs a homepage and most need release notes really suggests to me that they are their own categories and deserve unique guidelines.
The key here would be to go back to goal-focused thinking. My guess for the #1 reason for 4doc's success is how it aligns each doc type with a goal:
* Tutorials - Learning
* How-To Guides - Tasks
* Explanations - Understanding
* References - Information
If you're going to propose a new universal doc type, you should be able to propose a new universal goal type as well. My argument about release notes begins to fall short here: release notes are totally about information so they should probably just get bucketed cleanly under references. But I think my argument about homepages holds its ground better:
* Homepages - Convincing
MAYBE (but probably not) we can save release notes like this:
* Release Notes - Diffing
The Pigweed docs might be an interesting example of where the 4doc model falls short. Here in SEED-0102 (our docs plan) you can see how the 4doc model is the foundation but we felt the need to define further categories of docs. In terms of domains (recall how the author suggested the 4doc is good for tools but not programming languages) Pigweed is a collection of a bunch of stuff for embedded development (libraries, tools, etc.) so that could explain why we needed to define more doc types. https://pigweed.dev/seed/0102-module-docs.html
The topic of examples came up recently on the Write The Docs Slack. Daniele Procida had a very thoughtful response. Original comment from Rob H:
> I tend to like to categorize some of my items by whether or not they have "examples," which seems to be an attribute that could apply to multiple diataxis types, but not a standalone type by itself. I could use a tag on these items with the word "examples," but I feel almost like it needs a category by itself, given that it seems to be my learning style. But then there's another twist: in some cases, an "example" is more of an "inspiration for a way to use something," and in others, an "example" is more of a "clarifier that completes the learning process."
Response from Procida:
> You can and probably should use examples everywhere. I think that a well-crafted example can often illuminate something much better than a painstaking attempt to describe it fully and completely. In every aspect of life in fact. I bet that you can think of someone who was an example to you, of say moral righteousness, that taught and inspired you more than any amount of moral theorising and argument (I certainly can). Examples are very powerful, and we are intelligent, so we use them very well when we encounter them. Examples illustrate extremely effectively, whether they are illustrating how-to guides, explanation, or whatever. But examples on their own seem a bit deracinated. What are they examples of? Don’t they need context?
I think the main benefit of 4-doc model is that it makes you think about use cases for documentation. I think what you're alluding to is that there are more use cases.
The "release notes" serve the "I want to get overview of what changed" use case. The "homepage" serves the "I want to get a quick overview of the thing and see if I can use it" use case.
It's not clear though how the model could be extended to accomodate the above.
Tutorials, how-to guides, reference, explanation are modes of documentation.
They are not an exhaustive list of every kind of content that should appear in your documentation.
Diátaxis is not a list of four boxes into which all content should be mercilessly shoved whether it fits or not.
Consider: a homepage, an introduction, a foreword, a contents page, a landing page for a section, an index, credits, a list of contributors (I am sure you can think of more).
These can all be important. Documentation without a homepage would be positively stupid. I don't think we should have sections that are not introduced by landing pages.
Diátaxis doesn't prescribe anything for them, not because they are not important, but because they are not themselves modes of documentation. They are part of its furniture, if you like - just as an introduction, translator's note etc might be an important part of a book, but nothing whatsoever to do with the story it contains.
Homepages and landing pages are explicitly mentioned at https://diataxis.fr/complex-hierarchies/, by the way.
(Release notes could equally well be expressed as reference or in a section on their own. It really doesn't matter. It doesn't seem like something Diátaxis needs to worry about.)
4doc's contribution here is to ask you who the audience is for your release notes. Is it task-oriented, e.g. release notes for a database to help DBAs upgrade to that new version? Then write your release notes in a how-to style. Is it understanding-oriented, e.g. release notes for new B2C product features, to help users understand what it is you built? Then write your release notes in an explanation style. Different products have different requirements from their release notes, but 4doc helps you conceptualize who is supposed to benefit from it and therefore what style (and organization, and delivery) will be most beneficial when writing them.
The worst release notes are auto-generated from Git commits, in projects where Git commit messages are basically jotted-down handwritten chicken-scratch that are not really intended nor edited for external consumption. 4doc points out to you that autogenerating the release notes from commit messages like that is basically beneficial to nobody.
I find this table helpful to understand the four types model: https://diataxis.fr/compass/ This makes two distinctions: step-by-step vs. knowledge, and studying vs. working. The names given to each of the four categories are not the main point in my opinion.
If we accept that categorization, then the offical Python tutorial is in the knowledge/studying quadrant IMO, so actually not a "tutorial", but an "explanation"! Which makes complete sense to me, you need an explanation to work with a programming language, and actually defangs most of the criticism in the article. See for yourself, it's not step-by-step at all, instead, you get very small code examples among a sea of explanatory text: https://docs.python.org/3/tutorial/
So, yeah, words are used differently by different people.
I get your categorization and agree with it but honestly... the world would be a much better place if we could collectively be more consistent with the labeling of our docs:
* Users would find the docs they need faster because they know that the content of the "tutorial" really does line up with the usual definition of tutorial.
* We would make more progress on documentation innovation because we would spend less time clarifying miscommunications like this.
It's not going to happen, of course, but a man can dream
The most common reasons I check the docs are because I want to know all the functions available for an object, or what type of parameters a function expects, or what a function returns. None of these are straightforward in the Python docs, and you often have to read several paragraphs of text that may or may not have the answer.
Here's one example: https://docs.python.org/3/library/urllib.request.html#module...
What type is cafile supposed to be? You have to ctrl-f or read a bunch to find out that it doesn't even say. I guess you're supposed to assume it's a string from the context (or is it some kind of File object?). If you work primarily in Python and you're familiar with the conventions this might be obvious, but if you're not a frequent Python user and don't immediately remember if there's a special Path type or not, this is annoying.
What does that function return? There are three paragraphs describing the potential return types in the middle of a very long docstring, they aren't highlighted in any way even though this is pretty fundamental information, and they aren't comprehensive. I might need to click a couple links to figure out what exactly this function is giving me.
Here's another example: https://docs.python.org/3/library/datetime.html#datetime.dat...
Is timestamp a string or a number? Again you're just supposed to already know the Python conventions, or go do trial-and-error in the REPL.
These issues are pervasive for me on docs.python.org - it always feels way too verbose while lacking basic details. Every time I visit a page I wish there were a simple quick reference table at the top listing all the functions and their types before it dives into the minutiae.
I'm (maybe this goes without saying) a big fan of the Python docs. They make simple things easy by covering the 80% use cases really well, and for the rest they link source code up at the top. Django does this too which I really appreciate. I think you generally learn to use the affordances of the internet and browsers (cafile site:docs.python.org, ctrl+f) with all websites like, I mostly don't want websites reimplementing that stuff--usually worse--for me.
I would contrast it with the Python gRPC docs [0] which are a lot more type-savvy but don't really very much context about what the methods and such do. Like I've read [1] about 4 times and I don't understand it any better than after the 1st read.
IDK all I'm saying is there's more to it than just type information. Communicating technical information to people is really involved.
There aren't only 4 general document types, there are others, and you don't always need the 4 ones either.
Anyway, I'd say release notes are for upgrading.
My intuition says that the critiques of "it isn't universal" and "it isn't comprehensive" have merit but without stronger examples it's hard to tell.
But maybe the whole topic is moot because, as the post says, it was only Divio that claimed that this was the grand unifying theory of documentation. If Diataxis has distanced itself from this claim then there is no longer a need for it to be universal or comprehensive.
> There aren't only 4 general document types, there are others
The Diataxis author replied elsewhere in this thread reminding me that he thinks of them as modes of documentation. Which is different than doc types. The main idea in my mind is that there are other important modes of documentation that are not covered by learning, tasks, understanding, or information. That's what I was trying to work out in my comment about homepages and release notes. They seem to represent new modes.
* There's a bit of a "have your cake and eat it too" problem with Diataxis. The key underlying idea of Diataxis is that it represents modes of documentation, not concrete doc types. In that regard it's super counterproductive for us to discuss Diataxis as "the 4doc model". But on the other hand Diataxis does map each mode of documentation to a concrete doc type (Learning => Tutorial, Task => How-To Guide, Information => Reference, Understanding => Explanation) so I think it's understandable for people to conflate the two. Also, to make Diataxis useful, you have to be able to translate the ideas to the real world quickly. My impression at the moment is that Diataxis might be a little too abstract and it's causing us to have discussions where we just keep circling around the same ideas and arguing about definitions rather than actually having conversations that lead to documentation innovation.
* The article brings up a great point that Divio was initially billed as the grand unified theory of documentation, but Diataxis has moved away from that phrasing. Based on how Diataxis deals with modes of documentation I do think that it's essentially still attempting to be an attempt at a grand unified theory. And that's a good thing! Most of us agree that it's a useful foundation. The main questions in my mind are these: 1) are there any other modes out there? 2) is learning/tasks/understanding/information the most useful way to think about the fundamental modes? The main research approach for figuring out these questions is to look at real docs in the wild and then contemplate how they relate to modes.
I can't speak to the Python docs. Sadly, it seems that Mozilla has lost the recording of me in 2013 laying out my vision for Rust's docs, which is what became reality.
I had never heard of the four-document model, but instead presented a "three document model":
* Tutorials: long form documentation that explained concepts. This became The Book, but pre that, there were a number of ~5 page tutorials on various things.
* Language Reference: specific documentation that's closer to a specification
* API Documentation: documentation per module, function, whatever specific unit, unconnected to other stuff.
I had come to understand documentation in this way based on my time with Ruby and Rails. Tutorials are Rails Guides. Reference is a language spec (of which Ruby doesn't really have one but I thought that was pretty sad). API documentation is rubydoc.
Eventually, "tutorials" kinda morphed into "the bookshelf," thanks to mdbook.
You have to be pragmatic. If your docs could be improved by adopting a 4doc approach then you should, because they'll be better docs for it, even if the system is technically flawed. Similarly, if you have a doc that doesn't fit anywhere in the 4doc system but it's a useful doc, then it's still worth adding. 4doc is a process. It is not perfect, but it's still useful, and you might need to step outside of it occasionally. That's fine.
> I’m glad that people use it. I’m also a little frustrated that people use it even when its inappropriate.
> I’ve read plenty of good 4doc pages. Even so, it still leads to the same kind of “this is the only way” thinking. People use the model even when it’s not the right choice and they don’t vary up the format even when they should.
and you say
> 4doc is a process. It is not perfect, but it's still useful, and you might need to step outside of it occasionally.
---
The specific issue here is that it seems like the author has come across many people who use the 4doc model dogmatically. The article is geared specifically towards them, and is not an attempt to tear down the 4doc model as a whole.
But, short of that hire happening, 4doc helps point out where our docs are weak, by giving us a framework to discuss what any particular document 'is'. I usually find we've not got 4 kinds of documentation, but 2: a whole bunch of reference documentation, then a much smaller amount of tutorial, but little if any 'how-to' - which on the ops side, I find is what people are looking for. (the fourth kind, 'Explanation' is vanishingly rare, and usually only exists as tech talks)
I feel devs are uncomfortable with writing good how-to's because they want it DRY: either it ends up as a bunch of links out to the reference doc ("here's the bits you need, some assembly required") or they make a doc full of "if A then B", to avoid repeating parts of their answer in multiple pages. This reads terribly. A little flexibility is ok, but choose-your-own-adventure is not; just write multiple pages answering specific questions, and don't worry about the repetition.
In my current workplace, they've got stack overflow for teams filling this how-to role; I've seen many attempts at that fail due to lack of engagement but here a few people have pushed hard to make it useful and it's got some traction. We've even started using bookmarklets to pull conversations from slack and reformat them for SO.
Now I'm wondering if you could write the how-to in Twine so that it automatically only shows the reader the parts relevant to them based on what they've chosen as they go through it :-)
edit: and having read the article, I see Julia Evans had and executed that idea already -- wonderful!
To grossly oversimplify, the EPPO page structure is almost like a Wikipedia article. Imagine you end up on the page for, idk, anthrax. If you already know a bit about yucky diseases and want to learn more, you'll read on. If you get a couple paragraphs in and you're like "wait, what exactly is a bacterium?", you can click the hyperlink to the relevant article about bacteria. And if you actually wanted to read about the metal band, there's an escape hatch to redirect you to the correct location.
https://www.swyx.io/documentation-levels is my proposal. match documentation features to project maturity.
Level 0: Basic proof of concept
---
Example audience: you/colleagues/hobbyists One sentence description for github headline README with API docs - goal is to save yourself from looking at source code
Level 1: v0.x
---
Example audience: greenfield early adopters. Ok with missing documentation, they are here for the idea. can contribute code/docs
One paragraph description with more context - could be a sales pitch but also give an idea of when to use you
Feature list/Project Goals Install + Config Instructions
Level 2: v1
---
Example audience: brownfield early adopters. Ok with bugs, they have some problem that isnt well addressed. Needs convincing.
Comparison vs comparable libraries
Problem Statement - what you solve, how the developer/end user is better off with your thing than handwritten. aka Why You Exist
Basic Examples
Live Demos
Level 3: vX
---
Example audience: early majority user. Wants ease of use, quick wins. Need to be very reliable. Needs content to sell solution to team
Project Status badges
Tutorial
Third Party Plugins/Libraries
How-To Guide/Cookbook/Recipes
User/Maintainer Content
Official Blog/Project and Meeting Notes
Talks
3rd Party Blogs
Video Tutorials
Podcasts
Comprehensive Examples
and so on (up to level 6 in the doc)
I've seen several companies try to follow it pedantically as a structure for their docs (like literally having a sidebar with the four categories) and I've never seen that work out well, for the reasons explained in the article and the other comments here.
Tech writer for too many years here:
I agree with the 4 doc model. It boosts a proven structure for individual pieces of a doc set.
DITA formalizes this with its topic, concept, and task, and reference topics. You see another form of it in man pages.
And I agree with the FTA, it's often overkill or just plain unsuitable.
If you're stuck with writing anything from a readme to an aircraft maintenance library, then you can make your content engaging and useful by just making sure that it's correct, clear, concise, consistent, and cohesive. Above all it should be compassionate[1].
Check off those 7 C's and your doc will be appreciated. The 4 doc model is just one option to get you there.
That kind of information is absolute gold for solution design. Far too often I've found myself trying to discern the true envelope of some subsystem's capabilities by the negative space implied in the product docs, a task sometimes comparable to reading tea leaves.
Alas, many product managers regard documentation as propaganda, thus in the interests of fragile egos such topics are commonly omitted.
Where performance, security, and similar qualities were recommended by programmers but rejected by management, I especially want to record those for later reference in case of problems. Maybe what I write doesn't make it into end-user facing product documentation, but it's on the record. To give an example, at one project I consulted on, all of the user interface text was written in english, with US measurements with no i18n or l10n system. The company I was working with has customers from, if not worldwide, at least all of the western hemisphere. About six months later project management finally came to the realization that they were going to need at the very least multiple languages, and a lot of work was ahead to retrofit.
> You can and probably should use examples everywhere. I think that a well-crafted example can often illuminate something much better than a painstaking attempt to describe it fully and completely... Examples illustrate extremely effectively, whether they are illustrating how-to guides, explanation, or whatever. But examples on their own seem a bit deracinated. What are they examples of? Don’t they need context?
Best practices and anti-patterns usually work best in the context of how-to guides or references IMO
I think it’s safe to say that all four-plus kinds of documents work best as a coherent whole, each revealing an aspect of the system in relation to the others
Also, how to guide are not more and more covered by chatgpt.
But I don't think it should be a good reason to stop writing them.
For example, I wrote an article on "Parameters, options and flags for Python scripts":
https://www.bitecode.dev/p/parameters-options-and-flags-for
This is typically the type of things you get great ROI with chatgpt.
Yet:
- it's still faster to copy the summary of the article
- you need to understand the output of chatgpt so the article is useful to beginners
- you need to be able to use the right words to ask chatgpt what you want, which again, implies you understand the problem
- good practices and conventions are contextual, so the doc still helps
On the other hand, it's hard to find a lone article on the net, most people will find it faster with chatgpt.
But for your particular documentation of your particular product, that's still important to produce it. Because your best users will get familiar about your docs.
The biggest problem I find with software documentation is that the only rigorous part is a "Reference" which is organised as a list of functions or configuration options or commands or whatever, and while each individual item looks OK, they don't add up to a full description.
One clue that you've got this problem is having reference items which use technical terms which aren't defined, or are only defined in the informal part of the documentation.
Another form of this problem is a reference that's heavy on how options modify behaviour but omits any promises about the parts of the behaviour which are unconfigurable.
The four-document model's division of "Explanation" and "Reference" doesn't help avoid this problem. The "Explanation" is for teaching and is apparently not intended to be rigorous, and it doesn't tell you to make the "Reference" more than lists of individual items.
Reference level would be any API references, function explanations, data types, protocol specifications etc.. Guide level includes tutorials, conceptual explanations, how to guides, examples.
I find this to be a useful way of talking to people who need to produce some reasonable amount of documentation for something they're developing (or maybe some bare useful minimum). Often they will just write an API reference and think that they're done. By talking about "reference" and "guide" you can easily make it clear that they've got reference (kind of) covered, but they're missing and guides for how to get the SW built and then examples of how to use it.
We talk about the Diátaxis model a lot at Scribe (scribehow.com) where we feel like the how-to guide gets very little love from most of the traditional documentation platforms, which feel like they were all designed for reference material and not much else.
We wrote a review of bioinformatics software documentation, which includes a taxonomy of research software documentation (Table 1) and examples of which popular bioinformatics software packages use each two (Table 2). It is clear that not every kind is needed for everything, and it's more of a menu than a prescriptive set of documentation types needed for everything:
> the original Divio page doesn’t have this disclaimer and instead has big quote calling it the “The Grand Unified Theory of Documentation”. Procida doesn’t seem to be believe this and I wish divio would stop pitching 4doc as the one-and-only documentation system
This is a key point. I still equate Diataxis with this notion as the grand unified theory of documentation, because that's how it was originally billed. If Diataxis is no longer claiming to be anything close to a grand unified theory, then there's no point in debating whether it's universal or comprehensive. But the way that Diataxis talks about modes of documentation suggests to me that it still might be aiming towards universality and comprehensiveness. Put another way, are there other modes of docs that Diataxis doesn't mention? If these are the only 4 conceivable modes then doesn't that suggest that Diataxis is still aiming to be universal and comprehensive?
I personally think that it's worthwhile to formulate it as an attempt at a grand unified theory because for one, most people agree that it's a useful foundation, and for two, the "modes of documentation" approach lends itself well to further refinement. It basically boils down to researching/discovering whether there are more modes of documentation out there and debating whether the 4 existing modes are truly the most fundamental modes.
[1] The author of Diataxis who replied elsewhere in this thread
These already exist within the 4 models described.
> Conceptual Overviews
That's an explanation. It's covering the understanding. That's the purpose of that document.
> (“Aren’t these just a kind of explanations?” Explanations are intended for people who already know, or are in the process of learning, your subject. It comes after the tutorial. Conceptual overviews come before the tutorial. They are for people who are getting ready to learn the subject, or for outsiders evaluating if they want to learn it.)
This is wrong. When you start reading the documentation, you have started the process of learning. An explanation before is fine. The overview is literally a part of learning about the topic. Where it sits is immaterial. The purpose and the intent is important.
> Snippets and Examples
That's How-Tos.
> Imagine I present you an example of a code, and then an example of the same code, optimized. I’m not giving you a rote how-to on optimizing code, I’m demonstrating a way of thinking about optimizing code. It all comes back to the mental model.
You are demonstrating "how to think about optimizing code."
I don't think this is the fault of the 4doc model, but I do think it's evidence that it's not all you need.
As a case in point, all 4doc sites I've interacted with, as far as I can remember, have had documentation for how to use the documentation. That seems weird to me.
I agree with the author, a 'concepts' section is the big thing I find missing from the otherwise very useful 4 doc framework. This is also the most common addition that I see in open source projects, e.g. Istio and Kubernetes
Just today I failed to find an example of mounting multiple drives as one Btrfs filesystem. Plenty of how to create Btrfs filesystem examples. But how to mount them? Crickets. I eventually figured it out, but at a large cost in time.
For the record: to mount a multiple device btrfs filesystem, use an ordinary mount command with any one of the block devices that are part of the file system. Every device contains information about what other devices are supposed to be part of that filesystem.
I'm not sure if there are any techniques documentation writers can use to avoid this "hiding in plain sight" kind of problem.
If something works "automagically" like this then I jump into thinking about failure modes. Will it mount if one of the other devices are missing or failing? Will it just warn that the raid is degraded and I should add a new device? Will it mount read-only?
> I'm not sure if there are any techniques documentation writers can use to avoid this "hiding in plain sight" kind of problem.
The problem is probably lack of emphasis. It's guess it's hard to achieve in writing, if a solution is simple than you don't need many words to explain it, then it becomes easy to skip over.
That's all well-documented, but it can also be summarized by saying that btrfs does the right thing. It won't mount normally if it's missing any devices, because running in degraded mode shouldn't be a silent error (especially at mount time).
I am curious to learn about what other common best practices are around?
Cheatsheet, with variety of not overly simple examples (where i can figure out that if I don't want an option I can leave it out, but now I know the option
(TL;DR the other 4 are Tutorial, How-to Guides, Explanation, Reference)