Benchmarking programming languages

The shootout makes clear:

"How can we benchmark a programming language? We can't - we benchmark programming language implementations.

How can we benchmark language implementations? We can't - we measure particular programs."

The other thing that I have to note is that "Time measurements include program startup time." Which I think would distort something like Erlang greatly. It takes a long(ish) time for Erlang to start up, but then it runs very fast.

Ruby 1.9 and JRuby seem to finally be tipping the scales for Ruby. Both generally outperform Python 3.0 (though not 2.x) in this test. Ruby 1.9 versus Perl almost shows equivalency with Perl just pipping Ruby 1.9 but JRuby beating Perl ;-)

Of course, just how canonical are these benchmarks? I consider them pretty definitive for my own casual uses, but if there are others let us know!