Hacker News Reader

Comment by nostrademons | Hacker News Reader

Because software changes so rapidly.

Literate programming is based on the idea that you should explain what software does and how it does it in detail so that a reader can follow along and learn. That works great for TeX, which hasn't changed significantly since 1982. It works less great for say, Google/Alphabet, which wasn't even the same company last week.

The general problem with documentation is that it gets out of date as the software evolves to fit new requirements. Most real-world products face new requirements on a weekly, sometimes hourly basis; as a result, most fast-growing startups have oral cultures where the way to learn about the software is to ask the last person who worked on it.

View on HN

"Most real-world products face new requirements on a weekly, sometimes hourly basis; as a result, most fast-growing startups have oral cultures where the way to learn about the software is to ask the last person who worked on it."

The weakness of this approach is that the last person who worked on some piece of code may be that guy who quit last week.

Also, once you have a big code base, there may be parts of the code that are critical (e.g., low-level APIs), but that nobody has worked on in a year.

Also, while startups may have smaller code bases and thus have less problems with these issues, successful startups eventually do become larger, established businesses with huge legacy code bases (think Google or Facebook). At some point, lack of accessible knowledge about how the code works can grow into a massive technical debt. (Been there, done that, regretted it.)

All of those are problems in practice. For some, there are mitigating effects, eg. many companies use code reviews or pair programming to ensure that there are multiple eyes on each section of code, or they rotate out tasks between programmers so that multiple people need to get familiar. For some (eg. the large codebase that got big but has no documentation, and all the original authors have cashed out their options and are sitting on a beach somewhere), it just sucks.

The thing is - being eclipsed by a faster, nimbler competitor who steals your market is a bigger problem. Bad code will make your programmers groan and occasionally threaten to quit, but it usually won't threaten the existence of an organization. A bad product will threaten the existence of an organization. So among surviving companies, code generally tends toward shitty, because the ones that spent a lot of time on code hygiene often went under during the growth & competition phase of the industry.

> Bad code will make your programmers groan and occasionally threaten to quit, but it usually won't threaten the existence of an organization.

You're joking, right?

I have seen companies be "eclipsed by a faster, nimbler competitor" because they have a crappy codebase.

The reason for the implosions were that, "during the growth & competition phase of the industry," they could not grow because new developers really could not ramp up on the crappy, woefully-undocumented codebases, whilst clients left for less-buggy products that were adding new features.

But isn't the point of literate exactly that? You keep your code and your docs (design, description etc) closely knit. So even if you need to change your software rapidly, you change your code and docs together.

PS: I'm trying to do literate for my full time programming, still trying to cross the hurdle.

> keep your code and your docs closely knit

It is hard. Most people I know try to replace comments with very descriptive variable and method names - that are easy to change in any IDE. On the other hand comments (and other docs) sooner or later turn into misinformation/lie, unless you have very disciplined developers, all of them.

The only thing worse than a big system with no documentation is a big system with documentation that is full of lies and untruths.

Short of hiring people full-time to write documentation, and to ensure that all changes to the program are done in a heavyweight-enough process to ensure that documentation changes match the programming changes, those are your two choices.

A not as big system decomposed into well-defined parts with clear and concise documentation is best of all. Great documentation hits many purposes including lessening learning curve for training and behavior qualtitative verification without burdening readability by mindlessly duplicating boilerplate that detracts.

> most fast-growing startups have oral cultures where the way to learn about the software is to ask the last person who worked on it

"Oral culture" - I like that euphemism.

It's not disorganized, confused, poorly-planned, and glaring technical debt, no, it's "culture."

> Most real-world products face new requirements on a weekly, sometimes hourly basis

If your requirements are changing on an hourly basis, it's time to have a serious talk with your project manager.

> Google/Alphabet, which wasn't even the same company last week

Let's not pretend that some corporate moniker shuffling had overnight effects upon their codebases.

> Because software changes so rapidly.

So you don't have time to change your tests or other dependencies?

It's arguably not really a fair use of the term "oral culture" either. Most actual oral cultures placed a high premium on the ability to accurately retain core information (e.g. the whakapapa in Maori) with prestige for being able to convey it well.

You must have a nice cushy well defined corporate job if you think that constantly changing requirements are something unusual.

Changing requirements are quite normal, but changing by the hour is just poor project management, no matter what the environment.

Because software changes so rapidly.

I imagine you’re right in practice, but one might also ask whether code that is changing faster than understanding is changing too fast. It’s not as if mechanically updating literate-style documentation to reflect a change should be disproportionately expensive compared to making the change itself, or to making corresponding changes to test suites, getting the code reviewed by other people, or getting the changes merged into source control.

Another thing to note, however, is that not only is the code changing too fast, but also that code doesn't always do what the programmer thinks it does.

That is a good point. Reason I was asking this question is because of my current workflow. The workflow looks has the following stages -

Stage 1. Read a bunch of papers and blogs for the problem being solved.

Stage 2. Assimilate relevant ideas and come up with some kind of design. This stage usually involves hand written notes, doodles etc.

Stage 3. Write code based on the design.

I currently document stages 1 and 2 in a wiki but since it lives far away from the code, those things dont get used as much as I would like. Being able to add written reference material in a file along with the code and then having the ability to see a code only view seems appealing from where I am at currently.

Since you are doing stage 1 and 2 on a computer system, you could go on, and write the code of stage 3 in that very same wiki too!

Then write a spider to scan this wiki, extract the code blocks, and assemble a compilable program, and you're done, wiki literate programming!

A wiki is a terrible instrument for writing software documentation. For example, there is no revision control to track changes as required by by changes in the code, there is much duplication. I could go on, and on, why a wiki is not optimal for any documentation associated to a software project.

Better to use a system like DITA or dockbook.

Which wiki engine do you have in mind? There certainly is revision control in MediaWiki, Confluence, and others. Granted, I wouldn't use those in place of a source code revision control system such as git. There are generally no "annotate"/"blame" or "pickaxe" features. I would agree with writing something code-friendly in the first place, examples of which would include DocBook and Restructured Text.

There's at least one wiki that's built on top of Git:

https://github.com/gollum/gollum

There's also Gitit, which supports multiple VCSs including git:

https://github.com/jgm/gitit

Maybe to get the wiki closer to the code, you could put links to relevant wiki pages in your code comments. Some editors even recognize URLs in the code and allow you to click on them for instant access. And if your code repository is HTTP-accessible, you could put links to your code on your wiki pages.

Reply on news.ycombinator.com