Isaac Gouy is the gatekeeper for the shootout, and he refuses to include D for reasons he's refused to elucidate.
I stopped publishing benchmarks myself 15 years ago as people always assumed my thumb was on the scale.
I stopped publishing benchmarks myself 15 years ago as people always assumed my thumb was on the scale.
For the simplest reasons that I have stated to you many times!
Including every language implementation that the language author would like to have included is more work than I am willing to do, period.
Sept 13 2008: 63 language implementations were shown-
https://web.archive.org/web/20080913030117/http://shootout.a...
- currently, 27 language implementations are shown.
You could truthfully say "he refuses to include [at-least 30 language implementations]". In that regard, there's nothing special about D.
What about you create a technical specification (including both technical depth and common-sense breadth) for what kind of benchmark you'll accept, throw the whole thing on GitHub, and then refuse 99% of pull requests? :) (ie, only accept really really good quality benchmark implementations)
Eventually, enough developers unimpressed that Language X is not adequately represented would step up to the plate and maintain good-quality benchmark code. (This could get pretty interesting with rapidly-evolving languages like Rust.)
Obviously this is all very ideal and I can see so many ways such an endeavor could go horribly wrong, sure. I can also very easily see you having floated such an idea then discarded it for reasons I haven't even thought of.
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
The only thing I wonder about is allowing any given set of hardware to be compared with another set of hardware with anything approaching accuracy.
What would be awesome is if someone could figure out whether they'd need Raspberry Pi Zero to run some algorithm or if they could get away with just using a BBC Micro:Bit, by looking at a benchmark run submitted to the site from an i9-7940X.
Sadly I suspect the amount of realtime introspection needed (memory speed, practical cache coherency (per level), bus saturation, instructions used per cycle) would make this difficult - particularly because, even if a given benchmark was going to go fishing for performance counter info (and I just learned that even the Gen1's ARM 1176-based PMU provides a few, including one that counts instructions), benchmarking memory I/O is a bit harder; PCI DMA MMIO debug cards only map a small range of memory, like 256k or so, and I don't know if such cards can back the region with actual RAM. I suspect the access latency would be so different than from normal RAM that even if this did exist it'd never be used. Sigh.
And then differences in compiler optimization approach would have to be taken into account, and thorough understanding of assembly language for all target architecture(s) would be needed to have the time of day to page through the diff analysis...
Hmm, I think "simple" got left behind a few thousand miles ago. It's kinda (morbidly?) fascinating how different that different architectures are on the ground, hah.
- To clarify, I meant the Gen1 Raspi.
- I now realize (after having actually had a proper look) that while the Micro:Bit does use "JavaScript" and "Python", they're different, microcontroller-specific implementations, and I should have used a different example there, like an Intel NUC or perhaps an ODROID board or similar.
Just systematically enough to be reminded that a different context may have different relative performance.
There are many of benchmark projects that include D and don't appear to be struggling under any kind of massive burden. For example:
https://github.com/kostya/benchmarks
What are your actual reasons for not keeping D... "scala... and Clojure and ..."? The results in the previous link show D as a massive competitor (sometimes 1st place beating C and C++) on both memory and speed. Wouldn't the purpose of benchmarks be... to highlight useful, highly-scoring languages? Isn't that one of the primary reasons people read benchmarks?
(The D implementations are also often smaller in lines of code, as per this benchmark project:
https://togototo.wordpress.com/2013/08/23/benchmarks-round-t...
)
Do you see any mention at kostya/benchmarks of why ".NET Core" or Ada isn't included?
>> … benchmark projects that include D and don't appear to be struggling under any kind of massive burden.
Check when those kostya/benchmarks language implementation versions were released.
>> What are your actual reasons for not keeping…
I don't want to do the work.
Please share the secret of your tenacity and inspiration in the face of such "setbacks". How do you do it on a day to day basis?
Everybody likes to make more money :-)
Back when I was a brand new engineer at Boeing, I was talking to my lead engineer about the beauty of aerospace engineering. He smiled and said I didn't get it. Boeing wasn't making airplanes, they were building money making machines for airline companies. That the airplane turned out to be a beauty was just a happy side effect!
D isn't about my personal aggrandizement. It's about how effective it is for users at solving their problems and making money for them.
It can be though to shake "early license impressions" - D (dmd specifically) is one example, Ada (FSF GNAT vs AdaCore's GNAT Pro and Ada-gnat - GPL w/o runtime exception) is another.
[ed: if others are interested in "make your own measurements and host it yourself", relevant page with link to code etc appear to be:
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
]
Not since I asked him to stop dropping by the D forum to remind us that he wouldn't include D in the shootout.
I've personally lost all interest in it.
(The benchmarks are also small and too easily gamed by the author of the benchmark code, and the compiler developer. I encourage people to run timing tests on their own code, as that is what matters to them.)
This is the heart of the matter: benchmarking toy and unrelated bits of code amount to the same thing.
I think what people really need is a straightforward, accessible understanding of where a software implementation is inefficient. Design inefficiency can sometimes be "hacked over" (read "paved over") a bit like evening out technical debt; implementation inefficiencies, perhaps not so much. Making a complex database query faster by fiddling with (very) obscure SQL options because a given vendor's query planner is broken in a rare edge-case is one example that comes to mind.
I think it's especially sad when a software implementation becomes renowned for its inefficiency. It kinda takes the heat off in what I would argue is a very unfair way, like it legitimizes slowness like speed isn't a worthy pursuit... and then we wonder why our computers are slow. (Yeah, I'm thinking about Python here... and to a small extent many interpreted languages.)
In the context of D (and now DMC++ :D), so specifically compilers, it would be interesting to know what areas of the language don't generate especially fast code, or what bits might produce code that uses a little memory than it could, etc. Because that's what people really want to know before they take the time to write/port; and if they know about all the instances of "don't do XYZ in this very specific way" ahead of time, maybe they can write the best possible code on the first try! (Design and implementation are intertwined in practice.)
I suspect the list of such "avoid"s may not be long. It might make for a particularly efficient kind of developer user manual.
Gaming from the compiler/runtime side would be uglier - but I guess it is somewhat mitigated by real languages running the "general release" version.
No harm in "fastest way to list first 1000 prime numbers" being "print static list of first 1000 prime numbers".
Hand there's some value in having a standard benchmark harness that works easily across languages - as a helpful tool with "running your own benchmarks". Assuming the harness is any good, that is.
I just got tired of the vitriol leveled at me with no basis. Things like I must have "sabotaged" other compilers. Probably the absolute worst one was when the journalist decided that Datalight Optimum C ran the benchmarks so fast, it must be a compiler bug and removed the benchmark results from his compiler roundup, calling DC a buggy compiler.
The reality was Datalight C was the first C compiler on DOS to do data flow analysis, and it deleted dead code. (Benchmarks of that era did nothing useful, and dfa detected that.) No cheating at all. A couple years later, everyone did dfa.
(That said, D is lightning fast for me!)
If you really, really want to, you could use https://crates.io/crates/cargo-script to do the way you're talking about.
https://doc.rust-lang.org/cargo/ has installation, getting started, a reference, all of that. It needs some work, but for the basics of doing this, it's got it all.
https://forum.dlang.org/search?q=benchmarks+game&scope=forum