This is crazy insightful, thanks! I’d really love to learn how to get to this level of understanding, but can’t seem to figure out what curriculum I’d follow where I’d end up with this level of technical competence.
Then research how it is realized on the hardware with "warp"s or "wavefront"s (on AMD i think). How the cache works is also very important here. Sadly the information on the internet is relatively sparse here.