longbeard

How would you ensure that an instance launched on hardware bought 6 months ago is identical to the hardware under an instance launched N years ago and still running? Buy old hardware on eBay to prevent newer hardware introducing variation?

True. EC2 for example has 17 different instance types, but only one was benchmarked in the post.

http://aws.amazon.com/ec2/instance-types/instance-details/

If performance is the primary concern you wouldn't pick such small server/instance types to benchmark. It's like organizing a drag race between a scooter, a moped and a kid on a bicycle.

Another major flaw is taking results for a single instance type and implying that those apply to all instance sizes and each provider as a whole.

If you're going to do a benchmark at least pick something realistic like the m3.* types: http://aws.amazon.com/ec2/instance-types/instance-details/

At least the author had enough sense not to do the bench on a t1.micro