Novellas with Groff

I think it’s fascinating to see all the different ways that techfolk who write fiction set up their writing pipelines. Some people go with a bare-bones coding editor like Vim; others use “pro” tools Scrivener. And then there’s the typesetting pipeline. Using troff is pretty out there.

When I started writing fiction, here’s what I came up with. I write in Google Docs because it’s lightweight, works everywhere, and can easily export to Markdown format. From there, I manage the Markdown source in Git and Emacs. For typesetting—I cannot abide crappy word-processor output—I used to use Pandoc to generate LaTeX markup and run LuaLaTeX to produce PDFs. But that took several seconds. Now, I use Typst to go straight from Markdown to PDF files. It’s pretty much instant.

If you want to see what the output looks like, here’s the first chapter of my current project:

https://drive.google.com/file/d/1Wl4k_rdYGFZp-r5nq864r5CNzD1...

(Notice: This is a typesetting sample, not a writing sample. It’s my W40k-adjacent fanfic, so you have been warned.)

It’s set in 12pt Erewhon, which is an enhanced version of Andrey V. Panov’s Heuristica font family, which extends Robert Slimbach’s Utopia font family. Slimbach is a sublime type designer—he created Minion—and Utopia is one of his early works that is especially spirited.

I typeset some university work in troff in 1994. The professor I submitted it to was impressed but questioned my sanity in front of half of the class. Not wrong to do so.

Might still have the source code to that work somewhere. And it probably still parses.

Can't remember why I did it but on reflection it might simply have been that LaTeX was not going to fit on my extremely space-constrained Linux machine but troff was already there.

ETA: this sentence made me chuckle. "It's much easier to read without the inline comments:" Oh yes, much ;-)

O'Reilly used to typeset their books using troff/groff: http://www.lemis.com/grog/Documentation/PUS/tmac.Gbignuts.G (“O'Reilly & Associates book macros by Steve Talbott, Lenny Muellner”)

>Building a world with its own set of rules and history and what not turns out to be very fascinating. ... The length of the stories make proof-reading ridiculously hard.

Sometimes I wonder if writing more short stories would be like playing 9x9 Go games for new authors. It won't exercise the big structural challenges, and it comes with its own challenges, but it does practice a lot of this in a much more digestable size.

As far as world building goes, I think GRRM/Rothfuss/Sanderson style "world building through epic scales of structures and systems kept consistent" is overrated and maybe even Considered Harmful if you look at what it's done to my first two examples. By contrast, Borges does more world building with the ontological tricks in his <20 page short stories than it feels like Rothfuss ever did, in much the same way Tolkien did it with similar tricks. For example, treating himself as a mere translator and even doing some philologist tricks like telling you he's simplifying some names into English (Sam's real name? "Banazîr"), and withholding more information rather than dumping it on you, means that his stories feel like recovered translated works from a world just as real as ours and that's why people still devour The Silmarillion just to get more of a look into it.

Borges would do similar in his short stories, even embracing a little bit of inaccuracies in the (fictional) narrator's account of some events, then hitting you with a surprise endnote at the end of the story with scholia about the provenance of the story given that narrator's mistakes.

Wolfe was famous for it too. Embrace some of the challenge of getting the details right and if you frame it right, it makes it more human and more "real".

One potential weakness with using groff for professional publishing is that, according to its current mission statement (2014), its line-layout algorithm is limited and well behind the times:

Groff currently uses a greedy algorithm to format paragraphs one line at a time, but the Knuth-Plass algorithm, implemented in TeX, Heirloom troff and elsewhere, optimizes linebreaks throughout whole paragraphs and achieves a more uniform typographic grey. One of the most exciting challenges facing groff will be implementing an improved formatting algorithm, the most likely candidate being Knuth-Plass. It’s a big job, but with groff’s clean codebase and helpful community of experts, it is hoped additional developers will step forward and contribute their programming skills.

Equally important will be instituting native support for TrueType, Open Type, and other non-Type1 PostScript fonts, and improving Unicode support.

Source: https://www.gnu.org/software/groff/groff-mission-statement.h...

For those that don't know, groff's ancestor "roff" is very closely tied to UNIX's origin story. roff itself had it's own antecedent called RUNOFF (whence the name) on Multics. roff was basically UNIX's first "killer app" and was used to typeset patent applications by the division of Bell Labs that handled that sort of thing. It legitimized the further development of UNIX.

Groff seems to be somewhat supplanted by mandoc these days, at least on the BSDs.

Game of Thrones author George RR Martin writes all his novels on an old MS-DOS machine, not connected to the internet, and he uses WordStar 4.0. [1]

[1] https://www.bbc.com/news/technology-27407502

I was glad to see this page about groff, which is much underrated as a publishing system. The perl script for replacing some markdown markup with chosen groff macros is very interesting. I was however wondering about why the OA chose to write their own macro set rather than simply using the mom macros? Was it simply the need for German markup (and I take the point that languages other than English is an issue)?

Nils Holm (https://www.t3x.org/) also publishes ebooks using groff, but he does not (so far as I am aware) publish his markup or macros.

As to the writing: I'm still working on the sentences, so 45 pages is a distant goal.

Fwiw I wrote a bunch of my college papers on a version of Roff originally written by Col. G. L. Sicherman for CP/M, which I ported to Turbo C for the obsolete-when-I-got-it MSDOS laptop that I had back then. I ran that program under x64 Linux much more recently and it still worked after I ripped out some DOS stuff, despite some C anachronisms. The CP/M program is still around in old code archive sites.

I also remember writing a crude HTML to text renderer for an old Linux box because lynx -dump was too slow then. I wonder if the code I wrote is still around anywhere. Lynx on today's hardware is extremely fast.

Back in the day I was part of a 'commission' to produce a 5-yearbook for our student club. It being the digital age and we studying at the premier (and only...) agricultural university in the Netherlands I went and had a look at the possibilities to use university facilities to typeset the book. Those facilities ended up being a PDP-10 running TOPS-20 'cause the VAXen were too important for such work. After some experiments with TECO and RUNOFF I was convinced I'd never get the other members of the commission to work this way. Since my own C-64 was just that, my own machine and already in use for other things I ended up scrounging up an ancient tape-based word processing system consisting of a metal box with a tape drive connected through a thick cable to an IBM Selectric typewriter. While it wasn't the epitome of ergonomics it was more amenable to some of the more digitally impaired members of the commission. That the Selectric's motor burned out just when I was going to produce the master printouts which led me on a hunt for a motor (found one for ~$50) and a complicated repair (re-adjusting the whippletree linkage without documentation but with time pressure from the printers was not my favourite experience). Still, we made it, the book was published, all's well that ends well.

I think 9front (Plan 9 fork) dev, cartoonist and fiction writer Stanley Lieber also formats a good part of his stuff with troff, e.g: https://massivefictions.com/2024/06/24/0/

"This book was typeset (troff -ms|lp -dstdout|ps2pdf) in URW Garamond No. 8 by the author, using an MNT Reform laptop running the 9front operating system."

At my dayjob, our legacy customer communication generation (for letters and such) has a runoff syntax. It's z/os, so not groff, but something of that lineage. People are still working with those documents to this day, generating millions of them a year. They all look at me funny when I identify it as runoff, everybody else thinks it's something custom in-house.

The macro packages were a royal pain in the neck. As soon as you mastered one, somebody submitted a paper to a mill which demanded another. I alternated -ms and -me for ages because of this.

When we had a continuous paper wet process xerography printer, the cut-marks were a godsend. Once we stopped having that printer, they were a major annoyance.

For quite a long time TBL would spit out misformed tables. Mike Lesk told me this was an adustment added for the throwback function in the printer they used at bell labs and was needed to "shoogle" the table horizontals back into place for the next table. -Apparently printing a page-width worth of underscores in sequence caused an accumulating mis-alignment so they added some code to throw it back.

Whoah, wait, author uses groff without ms?! Even wrote his own macro package. Neat!

FWIW, to use ^H for bold print, set this:

    export GROFF_NO_SGR="Y"

or

    setenv GROFF_NO_SGR "Y"

depending upon your shell. Useful for old printers like I have :)

IIRC the Go book by Donovan and Kernighan was also written in groff.

Markdown if you can get away with it, typst if you need it.

I applaud all human-based writing tools. 10 days for 40k words is very impressive!

I suspect many people will prefer HTML for markup. That's why I built EPublish, which is related but for an HTML -> EPub workflow:

https://frequal.com/epublish/