Resonance would be a key factor as well, various sized rocks all at irregular angles would help greatly to prevent any overall resonance shaking and amplifying across the structure.

Now in contrast to standard sized bricks and regimental formations - like a row of pitchforks just awaiting the right note in many ways.

Then buildings of old had less engineering and material science to lean onto. That combined with a shift of building things to last forever towards a lifetime and even that lifetime gets shorter. We see things built with just enough materials to do the job and know more with a life of 100 years at best in so many cases (often half that with some housing).

Sadly it gets down to the numbers and odds as well - if there is say 1 in 100 years chance of a quake that would be over 6 on the scale and 1 in 500 of a 7 and the cost to build is 1m for upto level 5 survivability and 5 million for level 6 and 50m for level 7. You see how the insurance can push it towards going for the cheaper option on the bases that if you went for a level 5 - you could rebuild it 5 times in that 100 years. So money does play a huge part as always.

I do wonder, what would it cost today to build Machu Picchu as it is - a huge amount and can bet the modern approach would involve concrete and reinforced steel. Just can't beat natural stone still in many ways.