A pragmatic approach to Google AppEngine | Hacker News Reader

A pragmatic approach to Google AppEngine

nicklothian.com

68 pointsnl12 comments

think of it as an integration server for a legacy API you are integrating with: data inside it isn’t going to go missing, but you should expect your connection to it will break at any point. You need to isolate your users from it, protect you application from it and consider carefully how to protect your data from outages.

I get that in terms of practical advice - but the biggest thing I've been wondering in this whole GAE debate is why you'd select a service where failure is such a norm. Is it solely a cost decision because GAE beats someone like AWS by such a large margin? Theoretical development costs seem like they'd be much lower on platforms where you expect lower error rates.

It depends. For some applications it's going to suck, and suck badly.

OTOH, if you are primarily concerned with interacting with 3rd party HTTP based APIs, then you probably need to build that protection layer anyway. This is a scenario where AppEngine works really well - and once you've built that protection layer it isn't such a big deal to extend it to data access too.

The single largest benefit of App Engine is that Google does the sysadmin. I would prefer that Google's highly paid, top 1%, employees are responsible for the servers than whomever I could afford to pay. ;-) If your application fits with the App Engine paradigm, then it can be a great and inexpensive platform.

Where is this myth coming from that Google employees are highly paid? That's never been the case as far as I know. The number I'd always heard was 10-20% below market because "ZOMG, IT'S GOOGLE!". As far as top 1%, that's a meaningless metric and I've heard they don't even try for that anymore anyway (e.g. people complaining of a lot of "riffraff" coming in.

For me, it's because they move the problems into a realm I feel competent in. If I fix those issues, I get a lot of stuff for free where I have zero competence, and don't care to learn about.

Was it Foursquare who had the huge issue a month or so ago? They had many developers worrying about how to re-shard their data because suddenly it couldn't fit into memory. Keep that problem away from me...especially at 2am on a Saturday!

> Theoretical development costs seem like they'd be much lower on platforms where you expect lower error rates.

Then again, that theoretical becomes quite practical when your app that runs on EC2 breaks down because it isn't as scalable as you thought it might be. Google has already thought about the tough questions like "what happens if an entire datacenter goes offline?" and GAE has that designed in.

Developing for failure doesn't make sense when you're operating on a small scale, but it makes a lot of sense when you do need to scale. I'd rather take a bit of a development hit up front when there are few users than take the hit when a site really gains traction.

Regardless of the platform you’re using, defensive programming is a good way to improve the quality of your software, deal with expected and unexpected failures.

It is no different from sanitizing user inputs. You wouldn’t expect sane people to type alpha chars in a phone number field, yet they do. So you write code to deal with it. If you expect the datastore to fail every now and then, it helps wrapping it in an error handler.

Should it be that way? Certainly not, but software breaks, whether it’s designed by Google or by anyone else, and it’s our job to make it as smooth as possible for our users.

note: the datastore issues are not as common as they used to be a few months ago. And latency is low now.

> Theoretical development costs seem like they'd be much lower on platforms where you expect lower error rates.

What makes you think that GAE's error rates are higher than "the norm"? Better question - what makes you think that GAE's error rates are higher than what you can get/do yourself? (Yes, I'm sure that someone can beat GAE, but unless you're that person/organization....)

Yes, GAE encourages you to program for error. Unless you've got an error free alternative, you're either programming for error or hoping for the best.

"All too often a single HTTP request will have 4 or 5 database queries in it, and that is regarded as normal."

Is this really normal for a Java app? A typical Rails app (either built by myself or someone else) would have tens of queries in a request and still complete in under 1s. That was my experience with PHP too.

Well yeah, Java apps have tens of queries too, and they complete in under 1 second.

But that won't work on AppEngine.

That was the final nail in appengine's coffin for me. You get one query per page, preferably a get by id, and you'd better not want to do anything fancy with the result either.

Need more? It's simple. You just port away from appengine. Or you jump through the insane hoops the author describes just to get your mildly complex pages composed and out the door.

The choice of a platform has nothing to do with the response times. The author was referring to a typical "enterprise" app, which might call several backends at once. Java or no Java, some of them will not be that responsive, and it will increase the overall response time. His point was that the "enterprise" way of thinking of what is "normal" should be reconsidered when you switch to GAE -- Java has nothing to do with it.

Reply on news.ycombinator.com