Hacker News Reader

Comment by mercurial | Hacker News Reader

Nice to see Rust doing so well, but I'm disappointed by the performance of the Haskell code.

Why? I had (and often have) exactly the opposite sentiment: it is surprising that a language that is as high-level as Haskell can compete with low-level imperative languages.

Regardless what one thinks of Haskell, it's a testament to the work of the GHC developers.

That's the thing about Haskell, isn't it? It offers enough guarantees that the compiler can work its magic. Which makes me think it can do better than that in terms of perfs.

Every benchmark is different. And especially if you use lazy lists instead of loops with unboxed parameters.

Still, it looks mostly like a test of the random number source. And also none of the nice high level optimizations are firing anyway, http://www.reddit.com/r/programming/comments/1ixnf6/benchmar...

This test isn't actually a great benchmark between languages because, as the author correctly points out, it's mostly a test of random number generation and presumably all of those languages use libc's random number generator. If you run a quick profile on the C code, you'll find the vast majority of time is spent in libc generating random numbers.

To get a fair comparison without completely changing the benchmark, you should probably subtract .3-.4 seconds from each runtime as that's the amount of time spent in random number generation (again presumably all languages use libc's).

I would be surprised if Go used libc's random number generator; Go libraries tend to reimplement libc functionality in Go. (This is in contrast to Rust, which usually uses libc.)

Indeed it is reimplemented: http://golang.org/src/pkg/math/rand/rng.go

Rust uses a cryptographically secure random number generator (ISAAC) by default, to try to protect users from themselves. This means that the RNG is going to be slower than non-cryptographically-secure random number generators. You can get a non-cryptographic random number generator, but you have to ask for it.

Interesting, although I think that makes the benchmark even less useful because the heart of it (generating random numbers) is not using the same algorithm across languages.

I do believe that there's no reason that any modern language should not have a secure random number generator by default. But while ISAAC is likely fine for this purpose, I'd recommend Salsa20/12 instead (Salsa20/8 would be fine too, but let's be conservative).

Salsa20 is parallelizable and can skip ahead arbitrarily, has smaller state, and is fast: on recent processors the amortized time per 32-bit integer is around 8 cycles. It was also thoroughly vetted as an eSTREAM cipher.

Rust has XorShift (http://en.wikipedia.org/wiki/Xorshift) as a "secondary" RNG in the standard library, for what it's worth.

What implementations do you suggest? I'd be happy to bring this up to the developers if there's a chance of speeding up our default RNG with one that's just as safe. Note that it must also be compatible with our MIT/Apache 2.0 dual license.

SUPERCOP [1] has a bunch of public-domain implementations. [2] is a portable one. To get top speed, you need SIMD, though. The available implementations in SUPERCOP are in (x86) assembly, but they really should be converted to intrinsics to be more portable.

I don't know if Rust has intrinsics or some other kind of vector register support. I'd even volunteer to implement Salsa20 on it.

[1] http://bench.cr.yp.to/supercop.html

[2] https://github.com/floodyberry/supercop/blob/master/crypto_s...

[3] https://github.com/floodyberry/supercop/tree/master/crypto_s...

There's support for SIMD in Rust, and the compiler intrinsics that LLVM supports. Inline assembler is supported as well. However, note that the SIMD support will quite possibly change to become more first-class.

Note that I'm working on this (slowly), and things of interest can/should be put here: https://github.com/mozilla/rust/wiki/Lib-rand

D doesn't use libc's random number generator because the C standard one isn't very random.

http://dlang.org/phobos/std_random.html

Of course, a better random number generator takes longer to compute. The blog's author switched the D version to use libc's random number generator, and got some significant speedups.

http://www.reddit.com/r/programming/comments/1ixnf6/benchmar...

Haskell doesn't use the libc standard number generator. The implementation is here:

http://hackage.haskell.org/packages/archive/random/1.0.1.1/d...

Go's implementation is here and doesn't use the libc generator either:

http://golang.org/src/pkg/math/rand/rng.go

Walter Bright has already commented on D.

it's mostly a test of random number generation and presumably all of those languages use libc's random number generator

In fact, that would be better than benchmarking different PRNG methods, as it's done now ;).

Anyway, I think the benchmark is still fair, since the programs use whatever the default implementation for a PRNG is in the language's standard library. Sure, the Haskell program will be faster if you'd usea MWC or MT implementation from Hackage. But you'd expect a standard library to come with a sensible default ;).

The most recent Haskell version uses the MWC PRNG rather than Mersenne, reaching 1.05s (53% the speed of C). That's a respectable number, especially for such an expressive language. I imagine if someone could adjust it to use XorShift it would run even faster.

The reports from all of these micro benchmarks seems to be consistent on the subject of Rust: it doesn't yet have the performance that justifies its existence. I'm guessing you could get similar numbers out of Java.

I'm hoping Rust's numbers improve. It's incredibly promising, but without C++ level performance, it's too much semantic complexity.

Achieving the performance that Rust does is amazing, given the focus so far been on language design and not performance. It's not even 1.0 yet!

I'm sure these numbers will improve.

+1 The same is true probably also about D.

No because D has C performance, now. Have you seen the blog numbers?

As pointed out in the other comments on this thread, this benchmark benchmarks mostly standard RNG performance, not quality of the code generated by each compiler. Here you've got much more computationally intensive benchmark, showing that D is "as fast as C" like it could be said about Java: http://attractivechaos.wordpress.com/2013/04/06/performance-...

It is close, but Java is just as close, too. Not a tremendous performance advantage that someone would choose one over another for and not something I'd expect from a language marketed as "performance oriented".

Ok you live in a world where Java is "as fast as C". Let's terminate this discussion.

I didn't say it. You said D was as fast as C now. Then, according to your definition of "as fast as C", Java must be fast as C as well. Seems your as fast as C definition must be something between 30%-95% performance of C because those are the most common performance numbers reported for D in various benchmarks (I cited one of them, and the one in the blog also confirms this - only one D compiler managed to match the best C compiler, the others were significantly behind). The same numbers are consistently reported about Java, so I don't really understand what you're arguing about. So I conclude both are the same league (with two minor exceptions: 1. I saw Java beating GCC many times, once quite severely by almost 2x on simple array numerical code, while I never saw that happening for D and 2. D has much worse GC implementation than Java or C# have).

In the real world it's not nearly as fast as C, but in a micro-benchmark like this with a bunch of loops crunching fixed-size integers it would come close.

The difference in performance between the languages is almost entirely due to differences in the implementation of the benchmark.

With XorShift as the PRNG, Rust is now almost on par with C and beating the DMD and GDC D implementations. If the syntax were stable, and the standard library and documentation improved, then we'd seriously consider it as an alternative to C++ for parts of our current project.

You can write slow code in any language, that's exactly what the problem was (and still is) here. The implementations still aren't using the same random number generator.

Rust has C++ level performance if you compare the same code in both languages.

Reply on news.ycombinator.com