I haven't studied it carefully, but it sure looks like 90% of the performance improvement is using a big cache, which is a totally obvious thing to do. Also the big x86 guys have more or less been asleep at the wheel for almost a decade.

My go to example: my 2011 x220 sandybridge stinkpad is faster than my 2017 kaby lake mbp. 2005 machines (I dunno, Lakeport?) aren't even in the same ballpark as modern machines. Had that pace continued up to current year, the M1 chip would be a stinker. As it is, AMD is close and could smoke M1 in next generation 5nm chips, restoring order to the universe.