HNHacker News
TopNewBestAskShowJobs

Remnant44

558 karma · joined April 25, 2014

submissionscomments
Remnant44··on They're Breaking Every Retirement Rule to Be Off Now, Not Later
Which is a very important thing to amplify. Healthcare in the US is full of gotchas and small exceptions that can be absolutely devastating if you fall through the cracks. But Obamacare's mandate of coverage for pre-existing conditions was a huge step forward.
Remnant44··on What's new in C++26 (part 1)
It's basically like clockwork - you can assume that any post about c++ language evolution is going to have a number of people saying one or all of the following :

1) "CPP keeps getting complex in useless ways. Just use C (maybe C with classes style)". This viewpoint is correct in the sense that modern CPP is essentially a different language than C++98. But I disagree with the rest incredibly strongly - modern C++ is more expressive, safer, and often more performant than the old-school style. Things like unique_ptr, string_view/spans, RAII, etc are very useful and reduce boilerplate code as well as manage complexity.

2) "CPP is garbage, use Rust instead". I have not personally written in Rust, but I do find it to be a very interesting language. I would consider very strongly writing a new project in Rust. But most C++ projects are not new, and although us nerds always love rewriting perfectly working code, it's a good way to shoot your business in the foot.

3) "The template system is obscene". I mean, this is true. :) I do occasionally sprinkle metaprogramming into my code, because it solves some problems incredibly well. But it is essentially a different language grafted on at compile time. if constexpr, concepts, etc help enormously with this problem. And yes, those have all been introduced very recently...

Remnant44··on Understanding the Layout Process in Qt Widgets
I haven't had a high DPI issue with QT in quite a while. Early in QT 5, high DPI presented many problems, but they have been slowly settled over time, including by using sane defaults.

Source: I develop a QT widget app and develop on a workstation with mixed high and low DPI monitors.

Remnant44··on Zen, CUDA, and Tensor Cores, Part I: The Silicon
Hi Raph, first of all thank you for all of your contributions and writings - I've learned a ton from reading your blog!

A minor quibble amidst your good comparison above ;)

For a zen5 core, we have 16-wide SIMD with 4 pipes; 2 are FMA (2 flop), and 2 are FADD @ ~5GHZ. I math that out to 16 * 6 * 5 = 480 GFLOP/core... am I missing something?

Remnant44··on AMD Ryzen 9 9950X and Ryzen 9 9900X Deliver Excellent Linux Performance
I agree. I work in a similar field, and the value of AVX512 is clearly there - it just hasn't been worth implementing for the tiny percentage of market penetration. This is directly due to the market segmentation strategy Intel applied. AMD has raised the ante for AVX512 with two excellent implementations in a row, and for the first time ever I'm definitely considering building AVX512 targets.

Just as a small example from current code, the much more powerful AVX512 byte-granular two register source shuffles (vpermt2b) are very tempting for hashing/lookup table code, turning a current perf bottleneck into something that doesn't even show up in the profiler. And according to (http://www.numberworld.org/blogs/2024_8_7_zen5_avx512_teardo...) Zen5 has not one but _TWO_ of them, at a throughput quadrupling Intel's best effort..

Remnant44··on Zen5's AVX512 Teardown and More
Dynamic dispatch adds headaches to the build process; they are surmountable for sure, but in my experience the build wrangling to make it all happen is harder than the original work of rewriting your code with intrinsics!

The other major problem I have with dynamic dispatch, at least for the SIMD code I've written, is that you have to do so at a fairly high level of granularity. Most optimized routines are doing as much fusion & cache tiling as possible and so the dispatch has to happen at the level of the higher-level function rather than the more array-op-like components within it. And mostly, that means you've written your (often quite complicated) procedure several times uniquely instead of simply accelerating components within it.

I have not used Highway - if it dramatically simplifies the above, that's excellent!

Remnant44··on Zen5's AVX512 Teardown and More
Setting aside the complexity of "detachable" ALU clusters, the bigger problem is that essentially, we can currently drive all those ALUs _because_ they're SIMD and executing the same instruction.

For example, if you could un-gang even a single Zen5 ALU and do 16 independent scalar FMAs at once, you now need the front-end of the processor to decode and issue 16 instructions/cycle for each 1 instruction that it currently does! That's hopelessly wider than the front-end of the CPU can process. It needs to decode the instructions and schedule them, and both of those are complex operations that can be power hungry / slow (not to mention the instruction fetch bandwidth is now through the roof!).

SIMD bypasses that by doing lots of operations with a single instruction. It would be extremely difficult to achieve with just scalar instructions.

Remnant44··on Zen5's AVX512 Teardown and More
The short answer is "Yes, sometimes".

Clever hand-written SIMD code is still consistently better, sometimes dramatically better. But generally speaking, I've found Clang to be pretty good at auto-vectorizing code when I've "cleared the path" so to speak, organizing the data access in ways that are SIMD friendly.

On the Windows platform, in my experience MSVC is a disaster in terms of auto-vectorization. I haven't been able to get it to consistently vectorize anything above toy examples.

Remnant44··on Zen5's AVX512 Teardown and More
Agreed. My current best-case-scenario hope is that the success of the Zen4/5/etc processors will force Intel to adapt their strategy towards AMDs, and finally move us out of the avx512 mess they've segmented us into.
Remnant44··on Zen5's AVX512 Teardown and More
AMD's avx512 implementation is just lovely and they seem to be firing on all cylinders for it. Zen4 was already great, 'double pumped' or no.

It looks like Zen5's support is essentially the dream - all EUs and load/store are expanded to 512 bit, so you can sustain 2 512 FMAs and 2 512 Adds every cycle. There also appears to be essentially no transition penalty to the full-power state which is incredible.

The only thing sad here is that all this work to enable full-width AVX512 is going to be mostly wasted as approximately 0% of all client software will get recompiled to an AVX512 baseline for decades if ever. But if you can compile for your own targets, or JIT for it.. it looks really good.

Remnant44··on Fast Multidimensional Matrix Multiplication on CPU from Scratch (2022)
Yes, the figure was 250GFLOP for 4 cores instead of per core, I misread. Still impressive but more reasonable
Remnant44··on Fast Multidimensional Matrix Multiplication on CPU from Scratch (2022)
I honestly didn't realize how performant the decades-old 2013 Haswell architecture is on vector workloads.

250GFLOP/core is no joke - He also cross-compared to an M1 Pro, that when not using the secret matrix coprocessor achieves effectively the same vector throughput, a decade later...

Remnant44··on tolower() with AVX-512
Which is also why it's so attractive. :)

Those large shuffles are really powerful for things like lookup tables. Large tables are suddenly way more feasible in-register, letting you replace a costly gather with an in-register permute.

Remnant44··on I love you, HN, but you're toxic (2022)
I agree. It's not even that we over-weight being technically correct. Correctness and kindness are not mutually exclusive at all.

I suspect you're right and it boils down to the fact that a lot of us tech types have substantial pride. Worse, we support a substantial amount of our self-worth from being knowledgeable and correct, and so of course we have to both demonstrate and defend it. I recognize that truth in most of my real-life nerd friends as well as myself even as I try to temper it.

I don't know what the solution is for a community like this that will always be filled with people of so many unique stripes and neurotypes. Other than striving to argue our disagreements from curiosity instead of superiority.

Remnant44··on tolower() with AVX-512
I agree. I still hesitate to ship code that requires even AVX1, even though it was first introduced in 2011(!).

AVX512 really improves the instruction set. Not just from masking but from some really big holes filled in terms of instructions available that AVX2 doesn't have a good solution for.

At the same time I also DO have plenty of code that could definitely use the compute throughput improvement of 512 bit vectors. But it's definitely a more niche usage. You have to at least nominally satisfy that you: 1) Benefit from 2x the ALU throughput 2) Live mostly in the cache 3) Are not worth running on the GPU instead.

Remnant44··on tolower() with AVX-512
Neat and performant code like the article makes me very curious how the competition will shake out between AMD's AVX512 implementation and Intel's upcoming AVX10. The entire point of AVX10 seems to be to resolve Intel's P vs E core situation, while AMD seems to have taken a better approach of using either full width (Zen5) or double-pumped 256bit (Zen4, Zen5 mobile) as appropriate to the situation, while making the API seamless.

The big gains delivered in the article are all on a double-pumped Zen4 core! AVX512 brings a lot to the table so its quite frustrating that Intel market-segmented support for it so heavily as to completely inhibit its adoption in broad-based client code.

Remnant44··on tolower() with AVX-512
I've run into the same behavior with clang and intrinsics. Well, I appreciate the fact that they're trying to optimize the intrinsics usage, there really does need to be a flag or pragma you can pass that says more along the lines of "no really, give me what I asked for." In some cases I have found that the code it produces is a significant pessimization from what I had selected.
Remnant44··on Zen 5's 2-ahead branch predictor: how a 30 year old idea allows for new tricks
Because most code is not running anywhere near saturation of the available resources, and the problem is only getting worse as cores get wider. I mean, look at the Zen5 block diagram - there are 6 ALUs and 4 AGUs on the integer side alone! That's almost two entire Zen1 cores worth of execution resources, which is pretty amazing. Very, very little real world code is going to be able to get anywhere near saturating that every cycle. SMT helps improve the utilization of the execution resources that have already been paid for in the core.

I'll give another example from my own experience. I write a lot of code in the computer graphics domain. Some of the more numeric-oriented routines are able to saturate the execution resources, and get approximately 0% speedup from SMT.

Importantly though, there are other routines that make heavy use of lookup tables. Even though the tables reside completely within L1 cache, there are some really long dependency chains where the 3/4 cycle wait for L1 chains and causes some really low utilization of ALUs. Or at least, that's my theory. :) Regardless in that code running SMT provides about a 30% speedup "for free" which is quite impressive.

I was uncertain of if SMT had a future for a while, but I think for x86 in general it provides some pretty significant gains, for a design complexity that has already been 'paid' for.

Remnant44··on Zen 5's 2-ahead branch predictor: how a 30 year old idea allows for new tricks
For a fully compute-bound workload, you're certainly correct.

That's rare though. All it takes is a couple stalls waiting on memory where a second thread is able to make progress to make that "ideal speedup" certainly be nonzero.

Remnant44··on Kompute – Vulkan Alternative to CUDA
Those are totally reasonable caveats to deal with currently. Thanks!
Remnant44··on Kompute – Vulkan Alternative to CUDA
This looks great - I've been looking for a sustainable, cross-platform-and-vendor GPU compute solution, and the alternatives are not really great. CUDA is nvidia only, Metal is apple only, etc etc. OpenCL has been the closest match but it seems like it's on the way out.

Does anyone have real world experience using Vulkan compute shaders versus, say, OpenCL? Does Kompute make things as straightforward as it seems?

Remnant44··on Polychromatic Pixels
This is definitely a problem; if the control circuitry is up for it you could PWM the pixel color, basically dithering in time instead of space to achieve white or arbitrary non-spectral colors.
Remnant44··on Beating the L1 cache with value speculation (2021)
Very nice! I was struggling to come up with a realistic real world scenario for value speculation, but this is a perfect example. Any kind of multi-thread/contention related code seems like an ideal target for making a guess-based fast-path corrected by a slow-path consistency check
Remnant44··on Beating the L1 cache with value speculation (2021)
The linear node layout is not the point at all.

It's serving two purposes here:

1) Providing us with a correct "guess" of what the next node is. 2) Ensuring that in all cases we're running from the L1 cache.

In real world code, you'd be correct -- getting things to run out of L1/L2 is the most important attribute. This is specifically about a micro-optimization that allows you to beat the obvious code even when running completely from cache!

Remnant44··on Beating the L1 cache with value speculation (2021)
It's rare to need to work at this level of optimization, but this is a really neat trick!

Modern cores are quite wide - capable of running 6-8 instructions at once, as long as there are no dependencies. Something as simple and common as a summation loop can often be sped up 2-4x by simply having multiple accumulators that you then combine after the loop body; this lets the processor "run ahead" without loop carried dependencies and execute multiple accumulations each cycle.

This technique is similar in concept, but even more general. Put the "guess" in the registers and run with it, relying on a second instruction within a branch to correct the guess if its wrong. Assuming your guess is overwhelmingly accurate... this lets you unlock the width of modern cores in code that otherwise wouldn't present a lot of ILP!

Clever.

Remnant44··on TDK claims solid state battery breakthrough
A huge percentage of America. In most more rural regions, this is the only kind of coffee shop you will find.
Remnant44··on SIMD < SIMT < SMT: Parallelism in Nvidia GPUs (2011)
agreed! The basic categories remain, just blurring a bit at the edges.
Remnant44··on SIMD < SIMT < SMT: Parallelism in Nvidia GPUs (2011)
I think the principle things that have changed since this article was written is mostly each category taking inspiration from the other.

For example, SIMD instructions gained gather/scatter and even masking of instructions for divergent flow (in avx512 that consumers never get to play with). These can really simplify writing explicit SIMD and make it more GPU-like.

Conversely, GPUs gained a much higher emphasis on caching, sustained divergent flow via independent program counters, and subgroup instructions which are essentially explicit SIMD in disguise.

SMT on the other hand... seems like it might be on the way out completely. While still quite effective for some workloads, it seems like quite a lot of complexity for only situational improvements in throughput.

Remnant44··on Low-intensity exercise connected to decreased depression
Rather the point is that depression is... more of a shared symptom really. We're mostly grasping at straws with our treatments, but we do know that some cases (and we assume, causes..) are amenable to lifestyle factors. Others are assuredly not.

It's unhelpful for either case to assume that the other side doesn't exist.

Remnant44··on A (Draft) Taxonomy of SIMD Usage
I've done a fair amount of floating point simd programming for what the article calls type 1 - iterate large amounts of data as quickly as possible on the CPU.

I've been increasingly finding lately that integer-simd programming is incredibly useful to speed up all kinds of sorting searching filtering etc operations, even in otherwise scalar code. A lot of this kind of non-numeric code doesn't have any performance requirements, but it's very satisfying to be able to use when you run into situations that do.

← PreviousPage 3 of 6Next →