App Engine for Go Now in General Availability | Hacker News Reader

App Engine for Go Now in General Availability

groups.google.com

134 pointsagonzalezro85 comments

I've been trying out App Engine for Go the last couple weeks for a side project, but it's been a bit of a headache. Having to use their data store was a little annoying to begin with, but I was able to get past that one. Once I started playing with user accounts I was very frustrated to learn that they only support Google Account login functionality... I have been looking at implementing my own but Go doesn't appear to have a common authentication package and the advice often seems to be "roll your own". Any advice for doing this?

That said, this is the only Go hosting service I could find that has a free tier and their development binaries are very well done!

The only reason I use AppEngine is because they offer free hosting. I guess we have to pay the price of using proprietary APIs for free stuff.

If cost is your main concern, you can get VPS hosting from a dollar a month.

Is it as reliable as AppEngine?

If you're willing to spend $5/mo then DigitalOcean is a safe bet with excellent reliability. If you're a new customer then google for a Coupon before you sign up (saves you $20).

Don't try to go even cheaper, that way lies madness.

You cant compare AppEngine with DO or Heroku with AWS. Yes, AppEngine is costlier than DO 5-10X is comparing cost of computation power. You pay the price for not having to deal with DevOps, scalability and some cool API/services.

Amazon EC2 has a free teir.

Not if you've already been signed up with them for more than a year.

It may be free for weekend projects but if you're planning to get some serious traffic, prepare for some hefty bills.

I've had App Engine projects that were on Product Hunt and Hacker News for a day, and the bill for those days were close to 0. (it did exceed the free tier, but not by much)

You probably wouldn't want to run Reddit on App Engine, but anything with medium traffic or decent monetization should work out just fine.

The only other caveat is that it may depend on what the app does and how you utilize the datastore. If you don't consider cost of the datastore when building an app, it could end up being expensive like you mentioned.

Our burn rate is about ~$200/day (not month), half of it for datastore ops.

Heroku just announced official support for Go as well.

I have been a python appengine user for last 2-3 years. It is really useful to get a prototype out fast.

But MAJOR MAJOR MAJOR Painpoints are:

- User : only google account supported

- email : only can use <appid>@appspotmail.com

- Image upload support

otherwise very nifty to get a site out in less than a day!

Can you not use the REST API of Postmark, Sendgrid, etc?

What do people mean by "only google account supported", I've been building a web app with app engine and I just ran my own auth using passlib and it's worked fine. I also don't see how they can prevent you from adding oauth from different services onto your app.

The problem I ran into was that with python you could only use certain native libraries provided by google. To use third part unsupported libraries in python you had to do a hack where you submit them with the app but I managed to get it working with a bit of fiddling.

Presumably, he means the built in Users service only supports Google Accounts [0].

[0] Though this is true only in terms of stable features; it does have experimental support for OpenID identifiers.

> User : only google account supported

Isn't google account support one account more than you'll get at AWS or Heroku or anywhere else? Why is this a MAJOR MAJOR MAJOR painpoint?

> email : only can use <appid>@appspotmail.com

Not true at all. You can use any email address simply by adding it to your permissions list.

> Image upload support

What does this mean? I've built a few photography apps on GAE and had no issues with image upload. What do you mean specifically?

These MAJOR MAJOR MAJOR painpoints aren't work mentioning in the review of a service if they are also MAJOR MAJOR MAJOR painpoints everywhere else.

> These MAJOR MAJOR MAJOR painpoints aren't work mentioning in the review of a service if they are also MAJOR MAJOR MAJOR painpoints everywhere else.

blaincate wasn't doing a full review. They merely provided some very useful observations.

Simmer down. You have been zealously defensive about App Engine throughout this whole comment section.

> They merely provided some very useful observations.

Except they're false. False claims + Heroku promotion in a GAE announcement thread. Stay classy, people.

There are lots of bad things to say about GAE without peddling nonsense.

They won't let you run any python module with C code... which means you can forget about using BCRYPT or SCRYPT, for example.

Their image manipulation and edge caching service won't handle any photo wider than 1600px.

Domain name management is shit... you have to use Google Apps if you want that.

E-mail is restricted to 100 mails per day. App engine doesn't provide any listed prices for paid accounts. What happens after 100 emails?

Email is only limited to 100/day for free apps.

https://cloud.google.com/appengine/docs/quotas#Mail

No, you have to explicitly request an increase.

See this page: https://cloud.google.com/appengine/docs/quotas#Mail

"The daily email quota for billing-enabled apps depends on which billing system your app is using. See Increasing your Daily Mail Quota below to learn how to increase the quota in each case."

Once you go to the referenced page (https://cloud.google.com/appengine/docs/quotas#up_mail_quota), you find this:

"When you enable billing, the email quota remains at 100 messages per day. You must explicitly request a higher mail quota. Go to the Quota Details page of the Admin Console and scroll to the Mail section. There you'll find a link to submit a request for additional email quota. After you submit the request you can monitor its status by visiting the same link again. This will display a page that shows the status of the request: whether the request has been granted, is under review, or has been denied. If the request is granted it may be several minutes before the new quota of 20,000 messages per day takes effect. If the request is denied you cannot make another request to increase mail quota until some time has passed.

If your app needs to send more than 20,000 messages per day, you can sign up for a support package which entitles you to ask for a higher quota."

So it's not "restricted to 100 mails per day". It's limited to 20,000 per day, and that's only until you sign up for support.

Regarding the email issue, the addresses that you are able to add to the permissions list are very limited.

https://code.google.com/p/googleappengine/issues/detail?id=2...

- I have to search for oAUTH and use it . And having used django user authentication, something similar would be really useful.

- mail : send email on behalf of app users : not supported. Need to integrate with sendgrid or third party libray . native support would be good.

- Image : I could use the blob earlier, but now blobs are deprecated, and need to use google cloud.

I am not regular web developer, so other people may have valid points.

> mail : send email on behalf of app users : not supported. Need to integrate with sendgrid or third party libray . native support would be good.

Could you explain this? You want to send outgoing email with your users email addresses in the FROM field?

> Image : I could use the blob earlier, but now blobs are deprecated, and need to use google cloud.

Could you please explain how this is a bad thing? Doesn't google cloud make things easier and faster?

> And having used django user authentication, something similar would be really useful.

How about exactly that on App Engine, rather than just something similar: https://www.allbuttonspressed.com/projects/djangoappengine

- User: I usually just skip this service and use OAuth + Login management library

- Email: I remember it works with gmail; you can overwrite the "reply-to" header.

- Image Upload: seems to work fine by uploading to blobstore or GCS. Image serving is superb with images.get_serving_url!

Bit of a shameless plug, but we just announced official support yesterday on Heroku (https://blog.heroku.com/archives/2015/7/7/go_support_now_off...), graduating from our community supported buildpack. We allow you to pick pretty freely which databases and other services you want to you use from our add-ons. We also have a free tier.

Finally if you do give it a try would love to hear any feedback you have.

Hey, great! I've been using Heroku for a Node apps and was sad that deploying Go apps required a custom buildpack. I'll check it out!

Except automatically deciding what buildpack to use, what does it offer more to have official buildpack ?

I mean, when you know about Heroku's buildpacks you have access to all of Heroku raw power.

Buildpacks are definitely worth understanding to get the most out of Heroku, but not having official support is a negative. Also, I can't recall what it was but I did check into the community buildpack and it seemed like more effort than it's worth.

That said, I didn't take a very close look so maybe it was easier than I anticipated!

Personally, I don't put stuff on Heroku because it's costly and inefficient.

Let's say I have a bunch of pet projects and applications I want to run (blog, IRC bouncer, some kind of photo management). They take negligible CPU most of the time and ~128MB RAM each.

If I throw these on Heroku, I have to pay $7/each for 3 separate dynos, for a total of $21. Alternatively I can pay nothing but deal with my stuff being down 25% of the time.

If I throw them on Vultr or DigitalOcean, I pay $5 for a 512MB (768MB on Vultr) box and they all run happily ever after.

Heroku forces me to buy 4x the resources I actually need, which is a huge waste. It'd be nice to share my pool of resources so that if I use 384MB of RAM, I pay for 384MB of RAM or at least only pay for 512MB.

I know it's very unlikely you care about users like me and that's totally fine, I know things like this aren't particularly lucrative. Just thought I'd mention it since you asked about feedback.

Heroku isn't selling blank VMs, though. It's a value-added service -- you don't have to configure servers, build docker images, manage logging, wire up routes, inject service dependencies ... it goes on and on.

PaaSes are expensive if your time is free.

I work for a company which donates enormous engineering resources to an opensource Heroku competitor (Cloud Foundry). As a note, Cloud Foundry lets you specify exactly how much RAM you want to use. You can try it on Pivotal Web Services[0] (hosted by my employers) or BlueMix.

[0] https://run.pivotal.io/

Another shameless plug -- Go has been an officially supported buildpack on Cloud Foundry for more than a year, based on the same buildpack which Heroku have now adopted as official.

You can try it on Pivotal Web Services or BlueMix.

Disclaimer: I was seconded to the Cloud Foundry buildpacks team for ~7 months.

If you're okay with OAuth providers then https://github.com/markbates/goth gives you the capability to handle more than just Google accounts.

Otherwise, you're right in that there doesn't exist a 'de facto' auth package for Go like Devise in the Rails world. Part of this is because you can't rely on stuff like ActiveRecord, ActionController, etc. to do some of the heavy lifting for you (Go being just a language, not a framework on its own).

FWIW, I'm partway through writing a concise cookie- (and optionally Redis-) backed store that effectively asks you to provide a `User` type that can satisfy `GetID() string` and `IsAdmin() bool` methods. The middleware/lib does most of the work once you call `mypkg.SetUser(yourusertype)` in your login handler.

This obviously doesn't cover features such as password reset emails, user registration, etc because you either have to make a ton of assumptions about your users' design/implementation, keep things limited, or fall into the trap of "provide options for every possible use case".

PS: Ping me via email if you want some guidance. Happy to share some code snippets and/or input as I've been down the roll-my-own path a couple of times already.

Will do, thanks!

I've hacked together a package targeting the auth/user accounts gap, albeit in a fairly opinionated way. Very rough, but maybe they'll help someone:

https://github.com/likestripes/kolkata //user accounts API

https://github.com/likestripes/calcutta //basic UX

(ping me if you've got an interest or opinion on these!)

I've been using Go on App Engine for the past 6 months for a weekend project. My goal is to build a little search engine for Swift language/iOS resources. At the moment, I don't crawl any sites, just search titles and tags. I've been happy with the performance. Seems snappy?

http://www.h4labs.com/dev/ios/swift.html

Anyway, I'd like to hear from other Go web developers. Libraries, frameworks, etc that work well on App Engine.

Are you making the code open?

Hard to say. My current plan is to simply cobble something together. I do have a simple Go Web Server that I used when I started my other Go web server on Digital Ocean:

https://h4labs.wordpress.com/2014/10/13/simple-go-web-server...

Direct source: https://github.com/melling/GoWebServer

Go on App Engine is not hard: https://cloud.google.com/appengine/docs/go/

is it supposed to look like this http://imgur.com/0vngGP1

or is the CSS messed up ?

Yes, I'm an engineer. :-(. I was going for a simple Google results look. I had something that I thought was nicer but Google wanted mobile ready pages so I made it sparse.

No worries, I was just wondering if something was wrong on my end. There's a lot of simple things you could do to make it look better, you can start by pushing the whole site up. Right now most of the valuable content happens below the fold of the site. A lot of the stuff on top either doesn't need to be there or doesn't need to be the size that it is, so just prioritizing the screen space for results would go a long way. Then you can do simple things like adding dividers between results.

This looks a lot cleaner and usable http://imgur.com/zLdnKuy

edit: also you can just twitter bootstrap everything together

Thanks. I'll give it a try.

Whether to use Go on App Engine or not is a much less important question than whether to use App Engine or not. Java is also a good choice on App Engine. But either way, using App Engine dramatically affects almost any decision you'll make about how to build, and it ties your fortune to the sometimes-capricious product decisions made for App Engine.

This cannot be stated strongly enough. App Engine is a powerful platform, but the lock-in is very real.

It's not that Google prevents you from getting your data, but as you develop you start to lean on more and more App Engine specific services (e.g. Datastore, identity, images, task queues, etc.) just because they're so convenient to integrate with. You also have to engage in a lot of funny little hacks to deal with App Engine idiosyncrasies.

The next thing you know, you've got 50,000 lines of code and the thought of porting it all to another platform with an identical topology is hardly any more palatable than the prospect of rewriting it all from scratch (since you've invariably accumulated warts you'd like to refactor anyway).

App Engine is great for MVPs and toy projects. And it can be great for large scale projects, but you better be sure you'll never want to run it anywhere else...

Khan Academy runs on App Engine: https://cloud.google.com/files/KhanAcademy.pdf

One of the techniques to mitigate that problem specifically in Go is to create interfaces for everything. For example, instead of calling the datastore directly, create an interface with only the methods you need. Then, if you need to swap out your data store, you do so from the implementation of the interface, not having to change the code your business logic runs in.

How is that simpler than writing a package that replaces the datastore, then changing your imports?

If that feels simple to you, knock yourself out.

I believe this is way simpler. Using interfaces will coral your datastore code into one location. Then, you only need to change that code in a very modular way. Heck, give the job to a friend and go for a walk. You can review the changes he made later without worrying that he crossed your routes up. Heck, you can even run your tests (same tests as before since you are using an interface) against your new datastore.

Basically just a proof of concept at this point, but I've been hacking on something that would help mitigate these concerns for people who like App Engine's Golang abstractions, but not the lock-in. Contribs / thoughts welcome!

https://github.com/likestripes/pacific

It's worth noting that there is an open source implementation of AppEngine, for projects that want to self-host:

https://github.com/AppScale/appscale

Red Hat also run a hosted one called CapeDwarf.

I have been really enjoying Go App Engine support for some time now. It's pretty convenient to write web apps in Go, although to be fair all my projects have been super simple. I don't really have any experience making anything beyond a simple forum.

Nice! I've been using this for a few years for my Scrabble solver app but it's good to know they've committed to it for the long term :).

This is good news. Just out of curiosity, how App Engine with Golang is different than Digital Ocean with Golang????

I haven't used App Engine+Go yet, but did a bunch of work with AE+Python a while back.

The biggest difference is that you don't have to do any server management. You toss your app to Google, and they handle all of the deployment, distribution to multiple machines, load balancing, etc. I'm assuming that's the case for AE+Go as well.

Digital Ocean is (I believe) a virtual machine hosting service. You get a VM with some OS and an ip address.

Google App Engine is a very tightly controlled, and very inflexible, system for deploying applications which use google's App Engine services, Datastore, task-queues and an array of other useful things.

App Engine provides a very restricted development environment, you can choose from three programming languages, Python, Java and Go. For each of these they provide a custom modified runtime/compiler for running/building your application. The trade-off for this is that App Engine is able to automatically scale the number of instances of your application running to meet user demand (if you are able to pay for it). The restricted environment is somewhat idiosyncratic and is designed for scalability of stateless servers with a large, also idiosyncratic, datastore with memcached caching for speed. Good for a wide range of web-service development preferably with lots of independent requests which touch separate pieces of data.

It may not sound like it from the description, but I really like App Engine. But, it is important to know what it is for, and what it is not for.

Give it a try :)

It's not so restricted if you use Managed VMs instead of (or in addition to) the sandboxed App Engine: https://cloud.google.com/appengine/docs/managed-vms/

Yup, I just did a tutorial on using MVM's: http://pokstad.com/2015/07/29/golang_and_docker_on_gae.html

Good point :)

You left out one of the huge benefits of app engine. Google are responsible for keeping the stack secure. You deploy your code, and can trust that it's running on a secure kernel, web server, etc.

PHP as well:

http://googlecloudplatform.blogspot.jp/2015/06/App-Engine-fo...

PSA: Google is notorious for blatantly lying about App Engine availability. Today’s yellow alerts and red exes, on the uptime/status page, somehow always become yesterday's green checks.

(This was a first-hand source of frustration for the team I was working with a few years ago.)

You have been warned.

On the contrary, check out https://status.cloud.google.com/summary. Serious incidents (eg https://status.cloud.google.com/incident/appengine/15019) include a full incident report, including root cause analysis,.

There have been incidents that Google didn't acknowledge at all. There was one day where all sites using custom domains were redirected to Google.com for several hours, including their own Chrome Web Store. (which, I presume, runs on App Engine)

It eventually resolved after a few hours, but both during & after the incident, there wasn't any public communication from Google.

https://groups.google.com/forum/#!topic/google-appengine/IIv...

https://groups.google.com/forum/#!topic/google-appengine/voR...

Well I can certainly hope they upped the honesty factor. (My experience is from 2010 & 2011. If the page you linked to is indeed the new version of the one I recall — is there no standalone App Engine status page? — then it’s certainly be massively redesigned.)

AWS is notorious about displaying outages as green (plus an "informational message"). I think we can assume that everyone games the numbers as hard as they think they can get away with.

¯\_(ツ)_/¯

Doesn’t make my comment untrue. Or worth downvoting. If we don’t like the lies, what else can we do but call attention to them?

didnt downvote you fwiw

off topic , Do you think they will do same thing for dart too ?

Probably not. But you use Dart, NodeJS and any other languages with Managed VMs

https://cloud.google.com/appengine/docs/managed-vms/

Reply on news.ycombinator.com