jread

We have also created a report that summarizes and aggregates manged DNS performance and availability globally and regionally, features and pricing. It is still under development, and we intend to sell it in the future, but I'm happy to share it, and would appreciate feedback (caution it is a 10mb file): http://dl.dropbox.com/u/20765204/CH-DNS_Report-12_2011.pdf - it is a good resource if you are evaluating different providers.

If you are evaluating 3rd party DNS, I developed a tool for testing recursive dns query latency here http://cloudharmony.com/dnstest - the test covers most major Anycast providers.

I created a similar open source javascript/ajax/xhtml only browser-based desktop, with an OS X like GUI. It was fun, but a waste of time, and it never caught on. There isn't much of a use case for extendable OS like browser desktops imho.

http://demo.sierra-os.org u: demo p: sierraos (takes a few secs to load)

Infrastructure as a service (IaaS) cloud generally implies a few distinguishing features such as hourly utility billing and no long term commitments, rapid elasticity (provision/de-provision in minutes, not hours) and usually (but not always - there are bare metal clouds) virtualization. People often refer to software as a service (SaaS) as cloud, which I think is where it starts getting very ambiguous (i.e. is any hosted web application in the cloud???)

For what it's worth, I ran the tests and wrote this blog post and can assure you that we made every effort to be fair with the methodology and honest about the results. Take a look at our blog and website. Since 2009, we've released many benchmarks publicly and have a good reputation for providing unbiased, objective analysis.

This seems sort of like CloudFlare built directly into the browser. Essentially, using EC2 as an http optimizing and caching proxy through which requests are routed. They might also geo balance traffic to the closest EC2 region.

True, but without Anycast you wont' be able to control which server users query, and your latency for EU or APAC users will probably be in the range of 200-800ms. Route 53 uses Anycast network infrastructure with many geographically dispersed POPs so users will hit the closest POP dramatically reducing latency. For $200/mo you could host a hundred domains and billions of queries with much better performance using Route 53.

Typically this is true as the only performance hit will be due to the initial lookup, and then your resolver will cache the record. However, if your DNS record uses a low TTL (required for things like DNS-based monitoring/failover) like 30 secs or 1 minute, DNS lookups will have a larger impact on site performance because user resolvers will have to frequently re-query the authoritative DNS.

Providers that utilize an Anycast DNS infrastructure (Route 53, Dyn, DNS Made Easy, Cotendo, Easy DNS) will generally be faster for lookups than those that do not due to lower network latency. This is especially true for international users where latency might be 500ms higher than in the US.

If you are interested in DNS performance comparison, I created a tool for comparing DNS query times of 16 different hosted DNS providers including Route 53, Dyn, DNS Made Easy, UltraDNS, Rackspace, SoftLayer, Easy DNS, Zerigo and a few others here: http://cloudharmony.com/dnstest

It is a browser/javascript based test that uses a wildcard DNS record and alternating between cached and non cached lookups to determine an approximate query time for each provider.

I run a site that monitors over 100 different cloud services. Our monitoring node in EC2 EU West went down at 10:48:51 PDT. Our monitored application with PHP PaaS cloudControl, also hosted in EC2 EU West, went down at the same time. For the past year, we've gotten 99.978% availability in this EC2 region: http://cloudharmony.com/status

Check out http://cloudharmony.com/status We've been monitoring Storm for over 18 months and never once experienced an outage. They are one of only a few providers with no downtime (we monitor over 100 different cloud services).

We just released this, and are looking for feedback. Here are the providers currently supported (only 5 of 10 support the geo-targeted testing): Route 53, Dyn, Cotendo, UltraDNS, EasyDNS, DNS Made Easy, Rackspace Cloud DNS (not Anycast), LoadDNS, DNS.com, SoftLayer

I've done some benchmarking of softlayer cloudlayer and their "SAN" back cloud servers have about the worst IO performance of any provider. Even the largest 16 and 32GB instances performed extremely poorly. EC2 EBS is a much better performing storage platform. Stay away from cloudlayer if you have anything even remotely IO intensive.

http://blog.cloudharmony.com/2010/06/disk-io-benchmarking-in...

In response to question #1. I've conducted some similar benchmarking across all instance sizes and using both ephemeral and EBS with and without Raid. EBS is notably faster on larger instances. We observed roughly 2.5-3X better IO with EBS backed m2 instances compared to c1.medium. Using ephemeral Raid 0 on the cc1.4xlarge was about 6x faster.

http://blog.cloudharmony.com/2010/06/disk-io-benchmarking-in... http://blog.cloudharmony.com/2011/04/unofficial-ec2-outage-p...

You also have the option to use ephemeral storage if you prefer not to deal with EBS. All instances provide ephemeral storage with the exception of t1.micro.

This study was sponsored by Rackspace. It is based entirely on 2 benchmarks - iozone and linux kernel compilation time.... not exactly a thorough study. It also only tests m1 instances and ephemeral storage. Not a great study imo.

I've conducted extensive benchmarking of rackspace cloud, ec2 and about 38 other iaas providers. Rackspace cloud is definitely not faster than ec2 by a long shot. Rackspace cloud utilizes homogonous infrastructure, AMD 2374 to be exact. All instance sizes are burstable, so you typically get about the same CPU resources on a 1GB instance as you do on a 16GB instance according to our benchmark results. Ec2 on the other hand scales CPU much better all the way to dedicated dual quad X5570s with the cc1.4xlarge. Both this and the rackspace sponsored bitsource study compared an ec2 m1.small. Any study that does this should be immediately discounted as that is about the worst possible performing ec2 instance size. Adrian cockroft from Netflix refers to these types of studies as benchmarketing. They do not accurately depict the performance capabilities of ec2.

http://blog.cloudharmony.com/2010/05/what-is-ecu-cpu-benchma...

Or without the redirect callback: http://cloudharmony.com/mydns

Thanks, I'll try a couple of those in the next interview.

Not many I suppose, a decent consulting rate is about all. Those are things I don't have much control over.

We are developing a new application that operates separately from the auction software. The team is small and we all have fairly diverse roles.

BTW, this isn't intended as a flame post, I just would like to know if this is typical or not. I keep recommending to my manager that we hold out for someone that has more breadth of experience, but at this point I'm not sure we are ever going to find that type of candidate.

I would highly discourage a non-technical person from trying to manage a foreign outsourced software project. For what it's worth, I've managed a few such projects over the years with developers in India and China. My experience has been that you absolutely must have a technical person in house who should expect to spend a significant amount of time very thoroughly documenting and verifying their work. For a smaller project, it really doesn't make sense because you'd be able to finish the project yourself in the amount of time it takes you to manage an outsourced team. For larger projects, expect them to spend and bill at least 2-3x the hours (and 2-3x the number of developers) that you'd find with a local dev team. Also, don't expect any creativity or initiative to fill in gaps in your requirements... you will have to be very specific and explain every minor detail of functionality and behavior in order to get the project done right.

I was at the Cloud Connect conference last week. In a session on cloud performance Adrian Cockcroft (Netflix's Cloud Architect) spoke and said they do not use EBS for performance and reliability issues. They initially had some bad experiences with EBS and because of this decided to stick with ephemeral storage almost exclusively.

The guys from Reddit also spoke about their use of EC2. Apparently they are running entirely on m1 instances which suffer from notoriously poor EBS performance relative to m2 and cc1/cg1 instances.

I just created a very similar http-based ping based on this, but with the following difference:

* Use JSON-P to avoid same-origin-policy request failures (this also fixed the Opera failures) * Repeat request 6 times (1 warmup + 5 measured) and display mean, median, min and max latency measurements * Allow ping against 40 different provides including 4 AWS regions

http://cloudharmony.com/ping

This is a follow up to the earlier AWSPing post. I just created a very similar http-based ping based on that, but with the following difference:

* Use JSON-P to avoid same-origin-policy request failures (this also fixed the Opera failures) * Repeat request 6 times (1 warmup + 5 measured) and display mean, median, min and max latency measurements * Allow ping against 40 different provides including 4 AWS regions

How are you getting around same-origin-policy issues with this script? I noticed the javascript debugger is reporting failures for every request: "Origin http://www.cloudping.info is not allowed by Access-Control-Allow-Origin.". Are you just counting on the request failing, but recording the time that it takes to do so anyway?

Capturing and recording these latency measurements and associating them with the user's location would be fairly easy with MaxMind's free GeoIP databases and Apache module mod_geoip.

This is great, very close approximation to actual latency. We had a similar need on our cloud speedtest http://cloudharmony.com/speedtest and implemented using a Java applet that executed and screenscraped an OS ping to capture latency. We've been wanting to replace this with a less intrusive javascript approach.