It's a shame more people aren't familiar with arithmetic intensity and the roofline model [1], which give a simple mental model for performance profiles on both CPU and GPU.
I always find this graph [2] covering decades of hardware evolution fascinating.
[1] https://en.m.wikipedia.org/wiki/Roofline_model
[2] http://www.nextplatform.com/wp-content/uploads/2022/12/donga...