Hacker News Reader

Comment by Arare | Hacker News Reader

"The Computer Language Benchmarks Game" has never been a valid benchmark for any language or implementation.

Why?

Among others: because of completely arbitrary and I'd dare to say weird rules. Programs are not meant to be written in idiomatic language. They are meant to resemble the C implementation. In many cases you can see that the program representing the language in the comparison is neither the fastest nor the simplest of the submissions.

You've made a bunch of claims without showing any evidence to support them. Please point to specific examples we can all look at on the website which show -- "Programs are not meant to be..." -- which show -- "They are meant to resemble the C implementation." etc

I'm not going to back up these claims in any other way than pointing the "interesting alternative" programs that appear in some comparisons, which tend to be faster than the chosen versions. For example here: http://shootout.alioth.debian.org/u64q/benchmark.php?test=fa... Both Lisp and Java alternatives are faster than the "fastest" solution (written in Fortran). Finding any more support for the years-old opinion I voiced seems too much work to be worth it.

So we can immediately dismiss your "Programs are not meant to be written in idiomatic language. They are meant to resemble the C implementation." claims as baseless.

The only example you chose shows programs that manually unroll loops are not accepted. (And that's stated explicitly on the Help page. http://shootout.alioth.debian.org/help.php#unroll)

Yes, you can. Just one last mention: my opinion was formed in the times I used Perl and looked around the actual implementations of these. But that was years ago. It was reinforced quite recently by someone whining (probably on HN) about the same things. Apart from these anecdotes, I have no basis for these words.

> But that was years ago.

It's a sad nuisance when people opine without checking what's actually shown on a public website.

It's sad to see fact based comments being down voted on HN.

> It's a sad nuisance when people opine without checking what's actually shown on a public website.

Well, I could simply ignore the question. Or give the answer I got a few years ago, what I did...

> It's sad to see fact based comments being down voted on HN.

Fact based? More like facts querying. Which is the reason I upvoted every single of your comments...

> Which is the reason I upvoted every single of your comments.

And yet people have down voted.

"... we ask that contributed programs not only give the correct result, but also use the same algorithm to calculate that result."

(Except for meteor-contest.)

I'm sure you wish to make a point -- please don't make everyone guess what you mean, just spell it out.

An algorithm that's idiomatic in one language might be terribly unidiomatic in another. (I'm sure you know this, but) Haskell's pervasive laziness makes some algorithms tractable while requiring a lot of space/time complexity for other algorithms.

The benchmarks game URL was posted by someone who has put a lot of energy into promoting Haskell - dons makes the best of the opportunity that the benchmarks game provides to promote Haskell.

>>pervasive laziness... requiring a lot of space/time complexity for other algorithms<<

And GHC provides strictness analysis and explicit strictness to avoid reducing performance.

http://www.haskell.org/haskellwiki/Performance/Strictness

>>I need some sort of shock collar to teach me not to argue about the worth of the Computer Language Benchmarks Game.<<

Do argue - but argue better.

Unfortunately, to argue better does require learning what the website says about the benchmarks game - and for most that's far too much effort.

Reply on news.ycombinator.com