Hacker News Reader

Comment by idoubtit | Hacker News Reader

My own take of this:

- This study tries to enhance the original work by better metrics (e.g. confusion with C/C++), using better statistics (e.g. uncertainty).

- Automatically evaluating the quality of a code is hard and time-consuming. They had to use peer review of labellisation.

- Most of the old claims are not confirmed.

- Only 2 languages in this study had both a significant impact (impact coeff) and enough fiability (p-value): Clojure and Haskell. But the impact is still small, and there may be other influencing factors.

- They propose several ways to further enhance their work (e.g. taking regression tests into account).

View on HN

"... and there may be other influencing factors."

Selection bias? Consider the type of person likely to be attracted to languages like Haskell or Clojure.

(Note, didn't read the original study so I don't know the selection criteria. Were code samples in different languages from the same programmers compared, or were there different sets of programmers per language? My comment really only applies to the latter case.)

What is the type of person likely to be attracted to languages like Haskell or Clojure?

I can think of selection bias effects both in-favor of more defects and against it.

On one hand, Haskell or Clojure probably attract more academic types who have a reputation for (consistent with my experience) on-average worse-quality code.

On other hand, the barrier-to-entry for these languages is higher so they tend to be programmed by more experienced folks. I assume that people pick functional languages in part because they believe the functional paradigm leads to higher-quality code, so the sample probably cares more about code quality to begin with.

You can’t win against people’s insecurities.

They went to school, they where not bashed in the head with hammers.

And should be weighed against of the difficulty of trying to staff a team full of Haskell developers :).

> - Only 2 languages in this study had both a significant impact (impact coeff) and enough fiability (p-value): Clojure and Haskell. But the impact is still small, and there may be other influencing factors.

They also found a significant positive association between C++ and bug.

They found a statistically significant correlation: not a causal relationship and the correlation was small. Using "significant" without those qualifications presents a misleading picture of the results.

Reply on news.ycombinator.com