If I read this right, they reduce their competitors' benchmarks because they have better compilers? Can anyone justify this?
Generally, that is why we prefer to publish "compiler optimized" as best-case performance as well as "GCC" as more of the least common denominator. Both sets of data points are important.
Official SPECint published numbers will not use GCC because the organizations that submit them always want to see the best performance. Ampere used a scaling factor off of published numbers.
If you want to see the impact, we have some numbers from my ThunderX2 review: https://www.servethehome.com/cavium-thunderx2-review-benchma...
You can see the impact clearly there even though that was from a few years ago. Cray has a better performing compiler for ThunderX2 but we did not get to use it due to licensing restrictions.
I hope that helps. The bigger need is for more data since this is one view of performance. There are other needs as well such as FP performance.
When could we expect a review on Altra?
On an Altra review. Great question. I have been bringing it up for some time and live 15 minutes from their headquarters. The invitation is open on our end.
Oh, and don't miss the 3.0 vs. 3.3 GHz.
And even if you do it's irrelevant. Most common large/important frameworks won't compile with proprietary compilers.
Doing Bench's with ICC, XLC or other is hypocritical and often does not reflect anything useful.
Only the HPC world can afford to recompile everything with proprietary compilers and justify the man power to do so. And even so, they already have passed most compute intensive kernels on GPGPU with cuda a long time ago.
In benchmarking you have two choices: publish the real numbers or not. Which option you choose marks you as honest or not.
You can argue about why the numbers for your product are lower in the discussion section of your report. Not in an asterisk.
People can discard with the various SPEC* benchmarks as unrealistic -- they generally are -- but this is a bridge way too far.
The whole point of the SPEC* benchmarks is that you bring out the best of the best and do everything you can to achieve the pinnacle of performance.
And those SPEC* benchmarks are most interesting, and most applicable, to HPC users, who actually do use hardware-specific compilers. Of course another person mentioned that "everything is on GPUs now" (it isn't): If it's on a GPU, then you don't need a 60 core CPU. If it's on a GPU, then SPEC* benchmarks aren't relevant.
Using Intel's compiler or AOCC to compile binaries for normal server use is pretty rare.