Hacker News Reader

Comment by haecceity | Hacker News Reader

AVX-512 causes frequency throttling for thermal reasons.

https://lemire.me/blog/2018/08/25/avx-512-throttling-heavy-i...

https://lemire.me/blog/2018/08/24/trying-harder-to-make-avx-...

https://lemire.me/blog/2018/08/15/the-dangers-of-avx-512-thr...

Any performance gain from fewer cycle cost may be lost from the CPU being slower. Maybe there are situations where AVX-512 is faster but how is one to reason about performance when some instructions can slow the CPU.

View on HN

> but how is one to reason about performance when some instructions can slow the CPU.

The same way one should always do so on an advanced out-of-order CPU largely being programmed in modern high level languages: through thorough profiling and performance testing.

Empiricism? In my software development? No way.

I appreciate the humor but that's more time consuming than comparing instruction cycle counts and data dependencies. It's also workload dependent when it wouldn't be if they didn't come up with this throttling thing even if out of order execution makes it complicated.

Most programmers are not regularly writing primarily assembly or reviewing the assembly generated by their compiler. These are the scenarios where instruction cycle counts become relevant - everything else is abstracted too far to rely on anything other than profiling and testing. If that's you, then I'm sorry for my response which didn't really take you into account.

For the throttling operations you need to decide to commit to using them heavily or not. If you go all-in on them and use them well, you will get the same work done in less time and less joules than without them. If they won't help, don't use them. But, popping in and using them for tiny bits would be the slow and hot worst of both worlds.

Also, Intel almost never provides the frequency throttling details for AVX-512 heavy workloads, which is called AVX base frequency.

You can only see the performance values for yourself, or believe the SPEC benchmarks.

If you're working in an HPC center, you may ask your fellow centers and get off-the-record answers, sometimes.

This is absolutely not true. They are available for every publicly available SKU.

https://www.intel.com/content/www/us/en/products/docs/proces...

When we got our processors last year for our new cluster (I work in an HPC center), Intel explicitly said that it's confidential information and cannot be shared.

I remember reading the datasheets myself and not being able to find it, together with my teammates.

So they changed their mind two months ago, and shared this info for their processors.

Guess that the information lost its significance.

Either way, a processor significantly throttles under AVX-512 load. Nice.

Edit: I re-read the document and found our SKUs under the "newly added" table and, clarified the comment.

Original Datasheets:

- https://www.intel.com/content/www/tr/tr/products/docs/proces...

Spec Update - For re-referencing:

- https://www.intel.com/content/dam/www/public/us/en/documents...

It has been publicly available since years ago(I remember reading the specification explaining how it worked), it was never a surprise like Cloudflare made it out to be.

The base frequencies for first generation scalable was available however, not very publicly. You either needed to dig very deep in Intel's site or, find the blog post of some guy who nicely tabulated the data.

Our mostly used SKUs are released in 2019 so, its data was not available since years ago. Intel was very open about how their tech worked when it was new, needed to show their tech against the competition and not limited by thermal constraints.

When they became the dominant player, they didn't feel bad about hiding small but important details from their consumers.

Until EPYC came and pushed & prodded them around, they didn't bother to release both "R" processors and the related frequency scaling data for AVX-512 loads.

You measure. It's actually not that hard - try with AVX and try without and see which is faster for your workloads.

Reply on news.ycombinator.com