Eric Raymond's 'The Cathedral and the Bazaar' Turns 19 | Hacker News Reader

Eric Raymond's 'The Cathedral and the Bazaar' Turns 19

opensource.com

87 pointsalxsanchez57 comments

There are lots of interesting observations and historical tidbits in ESR's writing, but it needs to be taken with a sizable grain of salt for it to not do more harm than good. He presents himself like a social scientist finding hidden truths by connecting dots that no one has thought to connect before, and I'm pretty sure that's what he thinks he's doing, but the core of what he does is come up with narratives, try to find facts to support those narratives, and selectively ignore facts that don't support those narratives.

He clearly prioritizes narratives over analysis, with heroes and villains and truth and lies and good and evil and grand forces of history shaping the world in ways only he can perceive. If he was really serious about being a software historian, he would spent a lot less time in "I'm just telling you like it is, I've got a bibliography with citations and everything" mode, and a lot more time actively looking for evidence that refutes his claims in the interest of evolving his views. He would also use phrases like "us" and "them" and "elders of our tribe" and "I'm your worst nightmare" and "do you even know who I am?!?" a lot less often (like, never).

Once I noticed that he was basically writing a story in which he was his own Mary Sue [1], I could never unsee it. :( That sucked because I really did enjoy and learn from reading his stuff during the peak-ESR years of the mid-to-late-'90s.

[1] https://en.wikipedia.org/wiki/Mary_Sue

It gets worse:

(...) Desperate for something to feed my jones, I snaffled my other sister's abandoned flute. And wow! I was a natural...immediately better with it than with the guitar I'd been hacking at for months. I was commuting to college at the time, and took it with me. I'd play as I walked between classes. Six weeks from a cold standing start I sat in with a professional jazz band for the first time...and they liked it. This was delightful but mystifying. All I'd had to do was learn to play a scale, and this amazing river of music poured forth with barely an effort on my part. It seemed almost as though my hands and lips had always known what to do, had been waiting for me to pick up the flute. (...) Until I realized, finally, belatedly, what had been happening to me. Until the Great God Pan reached out of my hindbrain and thundered "YOU!" And his gift is music and his chosen instruments the pipes and flutes. And his, too the power of joy; magic so strong that when it flowed out of me, even before I knew what I was doing, it amazed people into awe and incoherence and poetry. [1]

And worse: http://www.catb.org/esr/writings/sextips/bedplay.html

[1] Strong vibes of: https://www.youtube.com/watch?v=8CYF9Ge2JL0

What's wrong with bedplay.html?

The idea that ESR considers himself also capable of writing a sexual advice manual...

https://www.flickr.com/photos/pkdouyk/241484893

It's not like Richard Stallman is a paragon of mental health. That leaves Linus Torvalds. I am so whelmed I can't stand it.

Richard Stallman isn't an analyst or a historian. He's an activist, and from all available evidence a remarkably genuine and focused activist. I don't agree with a lot of what RMS believes (for instance: I'm just fine with DRM), but I respect him.

The same isn't true of Eric Raymond.

Right. And there's a big problem with "The Cathedral and the Bazaar" itself: it focuses on fetchmail, which is a relatively obscure (compared to e.g. Linux) piece of software that became remarkably cathedral-like in its development after ESR took over, to the detriment of its users.

(By "detriment" I'm referring to both design errors and numerous memory safety flaws resulting in remote code execution [1].)

[1]: http://pyropus.ca/software/getmail/faq.html#faq-about-why

ESR's writings taken as a whole do have their issues, as pointed out by others. CATB was a good bit of writing though, that pointed out the benefits of open source software to a lot of people.

It's effective writing, but is it "good"? Large chunks of it have been discredited. For instance, "many eyes make all bugs shallow" has been a running joke in the software security field for over a decade.

Yeah, I think it was pretty good. I still don't think anyone has a perfect explanation of open source economics and business yet, so for the time, especially, it was good.

Large chunks of the security field seem to love nothing better than to have something to sneer at, but there's a grain of truth in that phrase too. Remember when that Rails vulnerability came out a while back? It was like blood in the water for the sharks, and a lot of people started looking for more. Critically, yes, they were also the right people to be looking at it, but the influx of people looking brought to light other things, making the whole system more secure.

Raymond helpfully broke the essay up into numbered observations. Try the exercise of going through them and counting:

* Which are shoplifted directly from Brooks or from the early Unix designers (Kernighan and Pike in particular).

* Which are banal results of simply reporting the zeitgeist he was writing in, or, worse, blinkered by a lack of insight into how large-scale closed-source commercial development was already working at the time.

* Which have not been borne out at all; for instance, many eyes making all bugs shallow.

* Which are really sneaky attempts to associate his minor work on fetchmail --- that "abomination before God" --- with major work like Linux.

I don't think the essay holds up. It is effectively written. Raymond is a capable verbal communicator. He can write a coherent sentence. But that's an awfully low bar to clear. In its substance, I think the essay is in fact "not good".

Regarding "Linus's law", you have to do better than observe that random users will find bugs in software and have done so repeatedly for open source software. You have to account for two additional details that Raymond is in no hurry to talk about:

1. The major vulnerabilities in open source software that were hidden in plain sight for years and years and only discovered by commercial security labs.

2. The effectiveness with which the same group of securing security nerds has shaken bugs out of closed source software, most notably that of Microsoft, Raymond's bete noir.

> banal results of simply reporting the zeitgeist he was writing in

I think that's a bit too dismissive: he wrote effectively (as you say) and convincingly about free software in a way that no one else was doing at the time, and for an audience that was not just other hackers. That's pretty good in and of itself, even if he didn't have any brilliant insights.

I get why you say that, but my take is that nobody else was doing it at the time because "open source" had just entered the zeitgeist, and it wasn't Raymond who put it there; it was Linus Torvalds --- which is why Raymond keeps referencing him in ways that make it clear his audience is already familiar.

I am obviously dismissive of Raymond. I think Raymond has been a terrible figurehead for open source. Due to my bias, my reading of "Cathedral" is that Raymond saw Linux getting a huge amount of attention and wrote an essay that is essentially "Me too, I wrote fetchmail".

> I think Raymond has been a terrible figurehead for open source.

I agree with that, but still think CATB was a pretty good paper: I don't think they're two completely exclusive concepts. Along the same lines, I respect some things about RMS and what he's done, and disagree strongly with others. He is also not a very good advocate for open source (err.. free software): perhaps that's why ESR got some traction - he was not great, but better than the others. At least he wasn't opposed to people making money to the point of personally offending them.

I completely agree that Eric Raymond's writing is mostly spurious (and often offensive), but I want to step in for history as a discipline here. Narrative is a valid historiographic technique, and you don't have to perform social scientific tactics to be serious about being an historian. There are social scientific historians, but history is classified as a humanity for a reason.

What's problematic is presenting narrative history as if it were scientific, and also bad narrative history which diminishes rather than elevating the discourse around these events (which Eric Raymond's "great man history of computing [ps I'm the great man]" certainly does).

I was always turned off by this book's title. It really doesn't fit at all as a metaphor.

In the middle ages, cathedrals were collaborative, public works projects. A whole city or town came together to build one. People from all social strata donated money or work to build the most splendid building they could. The final result was a public space open to all citizens, with no commercial purpose.

On the other hand, a bazaar is literally a market, where people go to buy and sell their wares. It's all about the profit, not the common good, and it's all about competition, not collaboration.

Even if you want to ignore the entire free software aspect and focus entirely on the pure technical benefits of open source, the cathedral is the far superior artifact from an engineering point of view. And where is the openness in the bazaar? As if the merchant is going to tell you how much he paid for his wares, or how they were made.

A cathedral was a projection of power of the Catholic Church (later other Churches) over the local population. A bazaar is traditionally open to anyone to come and do their thing.

That is where the analogy comes from.

> A cathedral was a projection of power of the Catholic Church (later other Churches) over the local population.

Yeah, like the NFL projecting power over the cities by placing their teams there.

I like that angle, it's a new way to think about cathedrals for me.

With respect to free software though, the pertinent aspect of these architectures is the forkability. Once the cathedral is designed, there is no room for anything off script. You can't have your own services in the cathedral on Tuesday nights.

But if you don't like the way your meat lady treats her chickens, you can just open up your own poultry stand across the path from her. There isn't a central plan. The plan emerges from the voluntary action of the participants.

Maybe not from that one product (a single cathedral) but after it has been built knowledge has been gained on how to create flying buttressed, move stone around efficiently, build a foundation, etc. So if you had another large project you could benefit from this gain in societal knowledge.

Well the people that built those cathedrals generally kept the techniques of masonry a secret. Sure, people were free to use the fruits of their labors (for various shades of free) but that knowledge wasn't generally made available to all when the cathedral was finished. If you wanted another cathedral you had to go to the same (or similar) people who built the first one.

Masons of the time closely guarded their secrets, like source code from a vendor like Microsoft, Oracle, etc...

It's kind of weird, that we're starting to get a generation of new programmers who weren't even born when that essay came out.

I'm starting to feel that teaching a history of computer science (in terms of sociology and insights, not just programming) would be a great addition to the curriculum in college. Some familiarity with classics like The Cathedral and the Bazaar, Fred Brooks' The Mythical Man-Month (which predates even my old-timer ass), The Agile Manifesto, and other masterpieces would teach students a lot that they may otherwise have to learn the hard way, or won't learn at all.

Yeah. I had that feeling when I came across Turing's original article about the Turing Test a few months ago http://www.loebner.net/Prizef/TuringArticle.html. And other classics like Feynman on the Challenger disaster: http://history.nasa.gov/rogersrep/v2appf.htm

Maybe we should start aggregating the more "timeless" books/articles we recommend. I'll probably start something on Github or Medium like for my own recommendations.

College CS courses (at least in my limited experience) are so far from day-to-day software engineering that I don't hold out much hope that they'd assign these kinda readings though. A few months ago I wrote about how reading Joel on Software was useful to me while I was getting into the field: https://medium.com/@firasd/things-i-read-on-joel-on-software...

The one that really killed me on first reading was Alan Kay's concept of the Dynabook. It only took us 40 years to get it!

(Replying since out of edit time window) One more contemporary link I'd put on such a list is "My Heroku Values" https://gist.github.com/adamwiggins/5687294

Oh, that's a good one! I've had to learn a lot of that the hard way in this whole startup founder path... :(

OTOH, there are places starting to do Software Engineering degrees that emphasize real-world practice. They'd benefit greatly from a survey of those texts. I'd be mildly surprised if that doesn't already happen though, at least for holy tomes like Man-Month or Peopleware.

I graduated last year and my school has a computer ethics class that was required as well as an elective software engineering class that was more about team and project organization. Both were quite insightful. Edit: and I forgot to mention that we talked about those books.

Something a little bit like this?

http://ocw.mit.edu/courses/electrical-engineering-and-comput...

Not exactly a history course, but emphasis is on thinking about system-level issues, with plenty of reading (including both classic & newer papers) and writing (of essays, not code).

Yup, a history course would be a good idea. I still find it surprising when programmers, even on HN, don't know where the term "Open Source" came from, as if it always existed.

I was involved in a source code copyright lawsuit a few years ago as an expert witness. It made me realize how immature the software industry is. How many terms we use have no authoritative definition. There are no books that cover the basics that say "this is this" and "that is that"

Yeah, that's how terms like "real time" get overloaded and confusing for people.

I feel like that every time I have to explain that C wasn't the first systems programming language and the myriad of options we used to have.

No, there should be a web page that is dedicated to this so it can easily be referenced. :)

Also, new programmers aren't the only ones who don't know enough history.

Link?

I would add Steven Levy's Hackers to the list if you want to go back to the MIT years, and Mitchell Waldrop's The Dream Machine if you want to go back to the DARPA years. Both of those were totally foundational to my understanding of the social history of what computer science is.

What I think is so important about history like this I still draw on those ideas all of the time. There are still so many opportunities to apply ideas from The Cathedral and the Bazaar . These foundational works can be "harvested" for a century before society is fully adapted around them.

If you go there, you also want to take something that covers microcomputers. There were many things in Cathedral and the Bazaar I rolled my eyes at, but the one thing that arguably hurt was that he downplayed and ignored microcomputer history ("their language was BASIC, so primitive that..." - no ESR, it was assembly! "vast disconnected hinterland" - no ESR, we had a huge network of BBSes and internet access via it). For lionizing Torvalds, you'd thought he'd notice that Torvalds taught himself assembly on the VIC 20.

I don't see why new programmers need to read it. When I read it first time, I didn't understand what it really talks about. I thought it is about the difference between open source and closed source.

In fact, the book is about the difference between development models of open source projects. Cathedral is not necessarily a closed source project. When the book was written, Emacs, nethack and other open source software were cathedrals in a sense that core team developed software in private and released "tarballs" from time to time. Now, most new programmers don't even have such projects around. The first thing they see is GitHub, where forking a project is one click away. Even *BSD systems made their version control systems open for everyone.

Programming culture has assimilated superior development model and there is no way back. New programmers don't even imagine that developing open source software is possible in cycles of gathering patches and releasing tarballs, like the GNU project did before.

@beat Please list all the reading list. Not sure if the history of computing will reach curriculum in college. But I am interested in reading.

I know ESR is not necessarily the most loved person in the OSS/FOSS development communities these days, along with RMS and, heck, even Brice Perens and Ian Murdock in some circles, but I got into free software because of these guys. I had paid for compilers for DOS, and then Windows and then a new one came along and, etc, etc.

My searches for free C compilers led me to GCC, which led me to GNU and Linux and ultimately to a lot of writing by the above individuals. I am hardly unique in this respect so love them or hate them, ESR, and so many others that have been there since the early days have made huge contributions to free software and I for one am still grateful to this day.

Here's to another 19 years!

Same here. I was commuting to university about 45 minutes and I ended up either stuck in an hot, overcrowded dungeon of a computer lab or on an unreliable 19.2k modem at home and Telix to do my coursework.

Enter Linux... I bought some white label Red Hat disks from cheap bytes (or something similar), which let me do about 90% of what I needed to do on my PC.

I find it a bit chilling that you put ESR alongside RMS. RMS highlights the political aspects of software; ESR is just your run-of-the-mill disgusting internet misogynist[1], homophobe[2] and xenophobe[3] who once advocated open source.

[1]: http://esr.ibiblio.org/?p=6907

[2]: http://esr.ibiblio.org/?p=26

[1]: http://esr.ibiblio.org/?p=129

OK, fair enough. I thought about putting in a comment about how much I disagree with ESR on most things these days. But this paper came out before he started using his sudden exposure as a soap box. I was trying to keep things centered on the spirit of F/OSS as I saw it when I was much younger and way more interested in code and systems than politics.

But since we opened that can now, yes, homophobic, racist writing based on a flawed understanding of libertarian-ism and it's worst qualities have done a lot to dampen my opinion of him in the last 10 years or so...

RMS I still read and tend to agree with even though he is just as maligned in some modern communities as ESR, just for very different reasons.

> ESR is just your run-of-the-mill disgusting internet misogynist, homophobe and xenophobe

To be fair, he was pretty representative of Internet culture in the 90s. For whatever harms the commercialization of the web has caused, at least there's been a clear shift away from that as the dominant culture.

One of my strange life experiences: I was working for an ISP that was based in San Antonio and the 55th WorldCon was being held in SA that year ('97). The boss and some coworkers wanted to go, so I tagged along - it was either that, or go back to my extended-stay motel for the evening.

At some point we ended up in a small conference room with a few other people; mostly ISP-industry people (I think Avi Freedman was there).

I ended up in a lengthy discussion with the guy sitting across from me. We chatted for probably 20 minutes before he turned and I saw his con badge. "Eric Raymond".

"Eric Raymond? Jargon File Eric Raymond?" He beamed and was happy that someone recognized him. I joked and did a Wayne's World-style "I'm not worthy!" and thanked him for his work on it, as it had been part of the inspiration that made me end up in a career as a system admin.

I've been thinking about the history of computing spanning my 30+ years of involvement with it (I got my first computer when I was 8). Things moved so fast back then (C64 was released in 1982, and the Amiga just three years later in 1985, Linux came six years after that in 1991), but in many regards have only accelerated since then. The arc of Open Source and Free software is perhaps the most interesting and powerful invisible force in our world today.

Imagine the Internet without Apache, Linux, MySQL, etc. And, it's moved further out, as well...now Open Source powers most of the clients, too (Android, Chrome, Firefox; the way people interact with computers is, under the hood, Open Source). This is an incredible shift. Microsoft and Apple (and a few others) fought for years...and, eventually they didn't really matter anymore. Those companies are at the edges now; some people still choose to use their products, but they don't define the architecture of our lives in tech.

It's weird to think back to the world in which CATB was written. OSS was a band of insurgents. Today, OSS is the norm, and it's the biggest gift economy in the world. It's not perfect, of course; loads of companies make huge profits exploiting OSS without participating meaningfully (and I'm not saying I have solutions for that). But, I like the world that the band of OSS insurgents has built better than the world they (we? I've been around it, and participating in OSS projects my whole adult life, and since well before CATB was written, but I still don't really feel part of it) started with.

And, I find it weird/amazing that today more Open Source code is likely being written and released daily than was written and released in a whole year back then (I'm making up this ratio, but I believe it's a reasonable guesstimate). Also feeling a bit old right now. It really doesn't feel like that long ago.

Can't believe it's been that long - whew. Here is an interesting counterpoint to this work by PHK:

http://queue.acm.org/detail.cfm?id=2349257

That's a great counterpoint, too. Both essays seem right. I think the solution is a hybrid... something in the middle as usual. I think an exploration into hybrid models that build on both traits is the best route. I'm eyeballing Rust as an example as it has a team doing Cathedral model with guidance and ecosystem from Bazaars. I see good and bad in it that come from aspects of both. Very interesting.

ESR has said some wacky stuff over the years, but this was a hugely influential bit of writing, and I'm very grateful to him for it.

This book (and the essay "How to become a hacker") marked the start of my programming career. Thanks ESR!

Whoa you had a great start then ;)

Is 19 years a significant milestone? Let's see, 19 can be expressed as 10 in base 19. Ah, now I understand. 🤓

What has become of him?

I can't speak of the past couple years, but in 2012 I reached out to him to see if he might be interested in speaking to a users group I ran (Philadelphia Java Users' Group) because I knew he was local and would likely draw a crowd. He agreed to speak because it was convenient, and seemed appreciative of the gesture - I don't think he'd been presenting much at the time.

In his presentation he mentioned some things he was working on at the time. I videotaped the presentation and published it with his permission.

https://phillyjug.com/2012/04/18/esr/

Racist ramblings.

"In the U.S., blacks are 12% of the population but commit 50% of violent crimes; can anyone honestly think this is unconnected to the fact that they average 15 points of IQ lower than the general population?" -- http://esr.ibiblio.org/?p=129

Also: https://twitter.com/tqbf/status/669592461111599105

There's more, I'm sure you can find it. Honestly, makes me sick how he's celebrated here. "Sure he has said some wacky(really?!) things but whatever I'm grateful." Done with HN for the day.

Might be at a gaming convention. He has seemed to express enjoyment of variously complicated strategy board games.

http://mobile.linuxtoday.com/infrastructure/1999121000105NWL...

Reply on news.ycombinator.com