Comment by netcan | Hacker News Reader

Comment by netcan | Hacker News Reader

Despite being big and different and having developed some grumpy old forum symptoms, HN is still smart. As such, I think we have a great example of memetic churn and progression here. We start with manifesto-ey essays trying to cut through some habitual thinking and paradigms. The new fundraising model. Bootstrapping. Web based SAAS is an ocean with a lot of paradigm shifts, the tech options, the business models, the development cycle. Those get people excited and defensive and there's fun to be had arguing it out. There's all sorts of pedantic arguing in the fringes pointing out that this things not strictly true. Then there are manifesto busting pieces. Case studies and anecdotes about manifesto-ed approaches going terribly wrong.

I'm not down on this process, it's progress.

Anyway, I think this is a part of it. "Enough eyeballs" is a big hairy hefty concept, part of the big hairy open sourced concept. No one could have predicted open source. The cultural and economic (BTW, economy is a big part part of culture) machinations of open source add up to something big and impressive and unpredictable. "Enough eyeballs" is a (somewhat squishy) big concept trying to make sense of it. This is like trying to boil down 'platform wars' or 'platform openness' into slogans and roes of thumb. You may come up with an interesting, informative and useful way of looking at things. But, underneath it is a big harry tangle of machinations, circumstances and ungodly complexity. Hence the need for rules of thumb and manifestos in the first place.

This post is part of the more mature stage. The audience is expected to be on Linus' side already and the novel part is discovering that the rule of thumb is isn't working in all cases. Diging into the machinations. Who are all these eyeballs? What are these bugs? Are some types of eyeballs more or less effective against some bugs?

I'm not sure where I'm going with this… be excellent to each other?

Anyone who has a couple of year into software engineering knows that there is diminishing return over the number of "eye balls" and that some problems are just god damn hard or need to be looked at with the problem in mind.

Software quality is orthogonal to the openness or closeness of the source code.

I think it is more about the will to do something great, the ability to listen to the user base and of course the technical skills of the authors.

I don't think orthogonal is quite right either.

I think it's just a matter of being complicated when you get dig into it. Open Source has certain strategies for quality that work, certain advantages (and disadvantages), different economies which means different abundances and scarcities of different resources. This "Linus’ Law" expresses something novel about the way open source projects can benefit from the advantages of being what they are. It's not an empty statement just not an airtight, always-right rule.

Yeah, number of eyeballs helps but it's not a catch-all. In some ways it could actually be a hindrance, since less-experienced eyeballs may not be able to mind all the necessary details, or may introduce further bugs/vulnerabilities. For our purposes, these are "bad programmers", whether or not they're good programmers on other stuff.

A simple model for code quality is something like Q =(good programmers) / (good + bad programmers). I would actually argue that it's not even related to the average, and that bad programmers can degrade quality disproportionately to their actual number. I think it might be something closer to Q = (good programmers)^2 / (good + bad programmers)^2. This is what it seems like people are getting at with the whole "negative productivity" in the good/bad/10x programmer framework.

Openness/closeness of the source code isn't the whole picture but the open-source model can more easily run into the too-many-cooks problem if the source is not properly gate-kept and reviewed in the aggregate. Thus I don't think it's entirely orthogonal. Of course closed-source software is typically commercial, which runs into its own set of pressures that degrade code quality.

I think much more sophisticated testing systems are what will really boost code quality. More advanced detection suites that throw more warnings, mandated full unit test coverage, plus CI frameworks that make sure that your code isn't merged if it doesn't clear the warnings and pass the full-coverage unit test. Randomized address systems and mandated array-bounds checking that catch undefined behavior and off-by-one errors. Basically, making things fail noisily instead of silently and forcing contributors to pay attention.

The C compiler in particular is really bad on the "throwing warnings" thing, even before you get yourself into other kinds of trouble. As in, according to the C standard, "rm -rf /" (or literally any other behavior the compiler wants) is a valid output behavior if you miss a closing quote([1]), rather than a compile-time error. That's an absurd definition in a security-minded world, and that's just the most egregious example of undefined behavior allowed by the C standard.

JVM-style managed code and declared exceptions are annoying but do seem to be a step in the right direction from a security perspective.

[1] http://blog.regehr.org/archives/213

The article shows it's not about that

> Perhaps you think these 30 bugs are really tricky, deeply hidden and complicated logic monsters that would explain the time they took to get found? Nope, I would say that every single one of them are pretty obvious once you spot them and none of them take a very long time for a reviewer to understand.

For me the takeaway is that even something as basic and mature as curl can't get it right in C. Two buffer overflows in as many years, along with TLS and HTTP failures. It's past time to move onto better tools. I hope the "bitcoin piñata" calls attention to the fact that there's now a full SSL stack available in a substantially safer language.

It's the devil that we know. New environments: new exploit classes.

There are some great examples of that (return-oriented programming as a way of getting around non-executable data segments), but surely there are some tools where adopting them was a pure win for safety and correctness over the status quo.

> or need to be looked at with the problem in mind.

The article addresses how hard some of the bugs were (they were not hard), but additionally, sometimes (often?) code needs to be looked at without the problem in mind -- who hasn't written code (or a paper, or comment on HN) and proofed it immediately, committed it, and realized it has an error that your mind elided over because you were so sure "what you meant" was there that you glossed over a dumb error. Having a different mind review these things avoids that problem.

Edit: I glossed over this -- I was also referring to proofing your work much later, so it's not still fresh in your mind "as you think it is". So, bring a "different mind" to the table.

s/completely different mind/different mind/

It is a long tail distribution and it happens because of different computational complexity classes. I don't really know if there is anything to be done about it.

I'll give you a high grumpy old forum five though.

Reply on news.ycombinator.com