Going Serverless: From Common Lisp and CGI to AWS Lambda and API Gateway
medium.com
medium.com
I've tried both, and GCF is vastly superior in many ways.
1) Developer experience is pretty terrible in Lambda, whereas GCF is compatible with Express.js to the point where you can simply pass in an Express app instance to it (which makes vendor lock-in a moot point as well). Develop your app more or less like you normally would, and just export it to GCF. Job done.
2) Deploying is a simple cli command away, even with native binaries.
3) No API Gateway necessary for HTTP functions.
4) It's now compatible with Cloud SQL so you don't have to give up your traditional database.
5) No crazy IAM permissions hoops to jump through to get anything to work.
Disclaimer: I'm not affiliated with Google.
I think the reason AWS gets all the love is, GCF only came out of beta very recently. Azure doesn't generally get as much press as AWS or Google Cloud.
As a secondary issue, "why use something with a poorer developer experience"? My answer: AWS's tools are quite complicated but it's also clear to me that Amazon takes AWS very seriously. My interactions with Google Cloud's products, support, and account management have left me with concerns about their long term commitment. This is a personal gut feel. I trust Amazon and Bezos' commitment to AWS. Focus and commitment are things we're all aware Google has had issues with in the past. As far as Microsoft is concerned, I see the great strides they're making and I would consider Azure for future projects. I'm growing to trust Nadella's Microsoft. It's clear they're betting the farm on multi-platform and cloud. Time will tell. Again this is just a personal viewpoint.
Lastly, once you climb the hill of learning IAM, CloudFormation, etc... you can be quite productive.
Is this common practice? I've looked for guides on this without much success, but is it basically, just static link your binary and then write a thin js wrapper to invoke it and hope it is the right architecture? Anyone know how multithreading would work? Is it more cost effective to multithread or just spawn separate lambdas?
I agree that developer experience in Lambda is crap, I am a Ruby person though and my language of choice is not on their supported list. You can do it, but the runtime I found[1] looks like a toy and hasn't been updated since 2015.
Google Cloud Functions on the other hand has this[2] (yay! ruby support) but I can't get excited about it, because we are "all-in" on AWS and it's politically impossible to suggest more diversity in this environment. I am however very excited about KNative, as I've been following the Riff project for a minute and it was looking really good (and the same people that built Riff, eg Pivotal, are in the drivers seat of building Knative now, as I understand it.)
Scale to zero! Bring your own runtime... lots of nice features. But number one is, I can run it on my existing Kubernetes environment, and use it to make my deployments leaner and my base cluster scale can be smaller. Hand-in-hand with that, I can carry it with me to another cloud provider if need be.
It works anywhere Kubernetes works, which is everywhere, and I think it will be a while before our non-ephemeral (non-serverless) workloads are going to start going away. And I'm not even stuck with Knative or Riff if I don't like it! There's also kubeless, which includes some degree of Ruby support too.
(NB: I haven't actually tried "rumbda" and it looks like I'm exaggerating about how long it's been since an update, I don't want to talk shit, but when I found "traveling ruby" in there and issues such as "ruby 2.3/2.4 not supported" I basically stepped away, as I have no existing entanglements with Lambda and no desire to get into any new ones. It might be fine, maybe just needs some love. I'd rather work on an open platform...)
Chalice for AWS makes local testing and deployment a breeze.
Lambda existed first and was already all in on AWS. I haven't even tried GFC because of those facts. However I probably will now.
The cynic in me says that a in lot of cases this has something to do with people ensuring their own job security. A lot of AWS technologies I've looked into seem to be way, way more complicated to get started with than they should be. There is a also this curious incongruity of people sinking tons of time into "figuring out" how to make something prod-ready in AWS, while saying the technologies they are using are "simple" and "easy to use".
I think "all the love" is related to the fact that you can enjoy the variety of AWS services (which does not exist to the same extent on GCP, at least yet), and integrate them together relatively easily. Also, from developer perspective, AWS documentation and SDKs are by far superior to those of GCP (at least from my experience).
GCP does have many advantages around specific features/services - but at large, AWS is (still) much more capable and mature.
Disclaimer: I am the author of the above article :-)
You could just build a Docker web app eg Rails/Play, deployed in ECS or Elastic Beanstalk with the same scalability in easily 1/10 the time and complexity. And if you get sick of AWS trivially move it to GCP, Azure or on-premise.
Until then, it’s not a mature technology, it should not be sold as such, and you take significant risk in implementing with it.
1. Biggest issue with serverless is debugging when you can't make a local environment and can't install anything like a conventional debugger in the serverless environment. I know of at least one company with a new and very elegant solution for this problem.
2. Visibility and monitoring is also a challenge and there are a bunch of serverless monitoring startups.
3. Security is interesting because, again, you can't install anything to protect your code: no Web Application Firewalls, for example. And there are several interesting theoretical attacks on serverless apps. Already, there are specialist serverless security companies.
A lot of companies don't 'migrate' to serverless. They use a Lambda or GCP Function to handle one small part of a backend task like image processing or some other form of backend tooling. Serverless is often just a good way of gluing other microservices together.
Sure, but that "junk" is key to making things scale. Prototype on whatever you want, but if you have to leave the prototype phase you have to care about/get involved in that layer.
https://medium.com/@amiram_26122/the-hidden-costs-of-serverl...
https://cloudncode.blog/2018/03/22/is-aws-lambda-expensive-t...
The ease with which you can go from zero-to-production can make it easier to build something completely new than to fix or extend something old.
In time, you get a sprawl of half-baked services and a complex distributed system.
That can lead to each service's creator becoming the sole maintainer of their creation because they're the only one who fully understands how it fits into the puzzle.
I'm hoping this will get better as the newness subsides and mature practices leak back in. Better tooling could also help.
Lambda doesn't cost anything when there is no traffic, and it handles spikes very well.
My biggest issue with serverless is the high cost at scale, which could be mitigated with this type of architecture. Are there any other benefits or concerns regarding something like this?
EC2 autoscaling takes a minute or two. This means you need to run with enough headroom to absorb spikes before you're able to spin up more resources. Depending on traffic patterns this can get very expensive (or not a problem at all, with predictable loads).
Our customers are concentrated around N. America, with minimal activity at night and weekends. We have a complex product with a wide surface area. Some of our services don't see usage constantly throughout the day.
Our usage is unpredictable and spiky, worsening our ability to cost optimize EC2 autoscaling.
Then there's staging environments...
Smallest EC2 instance will fall over with a sudden traffic spike, and requests will fail for the couple minutes it takes for autoscaling to complete.
Additionally for HA, a machine must be able to die without exposing us to the risk of a traffic spike taking us down.
For example, your base application runs with t2.micro. As soon as traffic starts going up, you launch additional medium/large instances. The load balancer would have to understand this also, and do weighted routing (ideally based on app server loads) so the first micro instance doesn't handle the same volume of traffic as the rest.
I've never seen a setup like this before, and frankly, part of me wonders whether the savings would be worth the effort of building it in the first place (vs just running with medium instances, or just scaling to dozens of micros to handle the load, or just using serverless).
If that's not fast enough, you can bake an AMI and scale more proactively (e.g. scale when your server is 70% utilized rather than 95%)
I've seen just API Gateway costing more than entire infrastructure costs of similarly sized websites.
If you can properly saturate EC2 it will be significantly less expensive than Lambda but with lambda you have to pay API GW and the vendor lock-in price.
I've started flagging these submissions.
Salary is a hidden cost people often forget. Or time spent by your programmer’s debugging infra; time not spent creating value. Frustration from having to deal with TF or ansible in the first place.
These are all benefits of a serverless solution which cost aware criticisms should at least try to quantify and take into account.
Signed, not-a-shill.
* I’ll rephrase: a significant amount of companies don’t have the luxury of being so swamped by requests they can saturate EC2 boxes and make the devops overhead (which, I’m arguing, is big but relatively constant/“sub linear”) worth the price difference (which scales linearly). Significant enough for these articles to have a raison d’être.
Lambda and api gateway is nothing but expensive vendor lock-in.
If you want to argue about vendor lockin, sure. But removing the necessity of managing scaling, individual instances, and OS / software patching is significant.
Sharing the cost brings benefits but it also brings pain. First time you hit an AWS hard limit you will realize how much pain
This is no different from any other abstraction.
> Sharing the cost brings benefits but it also brings pain. First time you hit an AWS hard limit you will realize how much pain
Many AWS services have no architectural scaling limits. S3, Lambda, API Gateway, DynamoDB, SNS, and SQS are examples. And you can always take comfort in knowing that many people are actively using the service at far greater scale than you ever will.
That's not to say they won't ultimately limit (e.g.) how many S3 buckets you create, but being able to create unlimited S3 buckets is not necessary to use S3 as designed in a near-infinitely scalable way.
All of those services have limits, just because you havent experienced that scale doesnt mean others havent.
See the sentence literally immediately after the one you quoted.
People are already using those services all day every day at far greater scale than most of us can ever hope to achieve. It's meaningless to pontificate on theoretical limits when the practical ones are higher than you can reach.
These "limits" are real and are hit pretty quickly (and it's easy to ask for bumps but they are still there)
S3 buckets is one of the first ones most people hit
With AWS (and other cloud provider) APIs and tools like Packer, these difficulties are vastly overstated.
Building VM images is just another step in the CI/CD pipeline, and patching and deploying a zero day fix becomes "kick off the CI/CD pipeline". You can even do automated unit testing of an image with tools like Inspec.
Scaling is an API call to change a "desired instance count" value (if it's not already automated), and complex problems with any individual instance can be resolved with STONITH (terminating the instance).
There is a non-zero cost in maintaining that process, including paying people to know and understand things like Inspec, packer, and Linux troubleshooting. There are also full OS upgrades where assumptions made could be invalidated, along with revising your process accordingly.
> Scaling is an API call to change a "desired instance count" value (if it's not already automated)
That automation is significant complexity. You'll be maintaining whatever health / resource checks are necessary to determine when scaling up is necessary, when scaling down should be done, what initialization / teardown tasks need to be done, etc. You'll also need some kind of health checks / monitoring to ensure this process is operating as it should so that you can detect if there's a problem with it. All of that needs to be known / understood / documented / maintained by someone.
And that's only for the stateless part. If you're trying to do with same with a relational database, it only gets tougher.
> and complex problems with any individual instance can be resolved with STONITH (terminating the instance).
Only if the problem is truly non-recurring and only in a single instance. Otherwise, it will be Linux troubleshooting to find out if it's your software, an OS patch, a third party software patch, or some other issue.
> There is a non-zero cost in maintaining that process
Just as there is a non-zero cost associated with maintaining Lambda, API Gateway, and the associated CloudFormation scripts, and finding people who can (and are willing to) maintain them.
> That automation is significant complexity
99% of that complexity is already shouldered by AWS and their ilk. They implement log forwarding, metric dashboards, instance health checks, and simple (complete) examples of how to scale based on CPU and memory - the two metrics used for scaling in most cases.
As for OS upgrades, yes, those can require a bit more expertise. That said, those occur every two to four years, and for the past few OS upgrades I've had to handle, the pain was limited to converting sysvinit scripts to upstart scripts, to unit files (none of which were strictly required, as an aside, since both upstart and systemd support sysvinit scripts natively).
> If you're trying to do with same with a relational database, it only gets tougher.
You mean RDS? Databaes need to be maintained no matter how the application is run. For a quick personal anecdote, there's a world of hurt waiting unless someone is hired who knows how to manage and tune databases, no matter who runs the infrastructure.
> Otherwise, it will be Linux troubleshooting to find out if it's your software, an OS patch, a third party software patch, or some other issue.
How is this different? Linux troubleshooting skills won't help to identify if it's third party software or your software - and those pains don't go away magically with Lambda. In the exceptionally rare case that it is the OS, it will be fixable by kicking off your CI/CD pipeline.
A small tip: Like compilers, the problem isn't the OS. Even when you think it's the OS, it's not. It's your software. OOM killer taking out processes? Those processes are leaking memory. Running out of disk space? Clean up the logs. Cron is misbehaving? Fix the typo. It's also worth mentioning that all of those problems are at least temporarily resolved by STONITH; enough to give time to fix the application.
This is mostly true, although "maintenance" for those things is minimal. There are fewer moving parts you are responsible for maintaining, and the ones requiring ongoing changes (OS and software management) don't exist. To maintain an existing web application, you are on the hook to potentially ship updated libraries (but not runtimes) in your functions, and to pay your AWS bill. This is like a half-step above no maintenance at all.
If I build a web application for a client that I deploy using a modern serverless architecture, it will require virtually no hands-on maintenance from me for... years? If I build a web application with a more traditional stack, I will definitively need to charge some amount for maintenance because it's not feasible to ignore patching or assume patching won't break all the automation I'd have around scripting, health monitoring, deployment, and everything else.
That's a significant difference.
> 99% of that complexity is already shouldered by AWS and their ilk. They implement log forwarding, metric dashboards, instance health checks, and simple (complete) examples of how to scale based on CPU and memory - the two metrics used for scaling in most cases.
At what number do I scale up? At what number do I scale down? How do I detect when there's a problem with the instances coming up? And I'm familiar with AWS--they certainly help with those things, but it's still on you to have the log forwarding agent running on your box, to set up the dashboard, to ensure you have the separate agent running on your box to forward memory usage metrics, and to ensure you're not doing anything that won't break your automatic minor version upgrades for your AMI (or manage your own, if you're not using EB or don't use that feature).
It's a whole lot better than doing it without those AWS services, but it's a significant step away from what you get with a serverless architecture.
> You mean RDS? Databaes need to be maintained no matter how the application is run. For a quick personal anecdote, there's a world of hurt waiting unless someone is hired who knows how to manage and tune databases, no matter who runs the infrastructure.
If you use RDS, sure. If you're using DynamoDB or (soon) Serverless Aurora, it doesn't require nearly as much tuning or babysitting.
> How is this different? Linux troubleshooting skills won't help to identify if it's third party software or your software - and those pains don't go away magically with Lambda.
Sure they can. Linux troubleshooting skills would tell you if an updated third party tool is now leaking memory, for example. And they often do go away with Lambda because your functions run on a level of seconds or minutes instead of hours, days, or more. Every function is effectively terminated every few hours at most. You could have problems with your libraries, but that's a much more limited troubleshooting scope than an entire VM.
It was a complex environment with high availability though, with maybe 15 different types of software trying to work together...
After using AWS for a while, I have to say that things usually work and don't suddenly start to malfunction, even when not doing serverless.
Downsides of serverless is definently vendor lock in, but I think having the complexity of open source software in house, and being able to try new software, is a double edged sword that can cost huge money as well.
It's not a simple choice.
- How do I avoid a cold start? (Fake calls into the system to the point where AWS has to start more instances of your application)
- Why is my cold start time different between development to production? (Production is in a VPC, which has greater instance spin up times)
- Why has Lambda stopped executing on my Kinesis stream? (Oops, AWS' bad)
- How can we get more parallel Lambda processing on a Kinesis stream? (Clone the stream and attach to the cloned stream)
- What can I do to reduce the cold start costs for Java? (Specify your objects as static, since those are evaluated at JVM startup, which AWS doesn't charge you for)
- How do I do Infrastructure as code? (CloudFormation; expect an average of 4 CF objects per function. SAM makes the CF more user friendly, but it's harder to debug and encourages the creation of extra infrastructure)
- How do I increase the bandwidth allocated to my Lambda? (Increase the memory allocation)
Well Java was the first mistake if you want a reasonable startup....
Probably not a good argument though. If you're successful enough to be saturating servers you're probably successful enough to be hiring experienced competent help for the transition.
- When you start having to spend a lot of time working around its limitations. (As is the case with all managed services.)
- When the sum of its parts create a distributed system whose complexity costs you more in productivity than it gives back.
Performance limits will get better with time. The complexity cost is more difficult to predict. Hopefully it will improve as best-practices emerge and tooling gets better.
Organizing is also simple - if you know CloudFormation. You should learn that anyway to provision resources in a repeatable fashion.
I’m also working on certifications mostly as a method to force me to learn in a structure manner, my company will pay for them, and they still have some market value.
I mean, if you care about the infrastructure there are few reasons left not to self-host.
Could it possibly be things like they have 3 EC2 instances to run a cluster for Consul instead of using AWS services?
It’s slso about elasticity. It’s much easier and cost effective to spin up 20 VMs (whether it be EC2 instances or Lambdas which are basically VMs) to run a test and see how many you actually need.
Other times it makes sense to use a bunch of spot instances to save money and choose a cost optimization based on throughout vs. cost for backend processing.
Do you mean Api Gateway + the lambdas?
API Gateway is $3.50 per 1 million API calls received plus data transfer ($0.09 per GB or $90 per Terrabyte). What kind of API were they serving that was racking up those large bills?
Typically you would serve static content off s3 + cloudfront, and just the API off the API gateway/lambdas.
Also if you have servers you are probably auto-scaling redundant instances in multiple availability zones and then getting that high CPU utilization is unlikely.
In essence his point was to use Lambda as a tool only to work with tasks related to AWS infrastructure automation. And not as a tool for everything.
The old adage, of not treating everything as a nail while holding a hammer seems to apply here.
You can avoid that with the Serverless framework.
At my old job, we were terrible at wanting to use new shiny things (me included). We had terrible vendor lock-in with AWS, although it did make my resume look nice as I could fill it with fancy AWS products.
My supervisor was discussing AWS Lambda with me at lunchtime, and I said that it's not worth it, because any cost savings from using Lambda over a server are outweighed by the cost if we ever wanted or needed to switch vendors.
AWS lambda would make sense for large companies, where server time is more expensive than developer time. But for most companies, developer time is more expensive than server time, using shiny new AWS specific tools is just going to hurt later on.
I mean, sure, if you're doing something as a hobby and consider your own time free but have to pay $5 for a VM somewhere, then maybe.
But especially at large companies, my experience has always been that for any work done the cost of infrastructure is basically a rounding error compared to the cost of developer time. And that doesn't even begin to include the cost of support team time for the eternity that everything seems to need to be maintained for in larger companies.
As a recent example, I know that the amount each developer on a team I've worked on costs (not how much they are paid, all contractors from various companies), per person, more than it would cost to buy a mid range server, colocate it in a data center etc - per day. So after a year, if we just bought infrastructure instead of developer time, we could have had a handy ~15,000 mid range servers racked and ready to go.
BTW, you can use other self-hosted API gateways like Kong with Lambda if you want to cost optimize.
If your workload is a 'traditional' web application, saturating a full VM and not too bursty - don't run it on Lambda.
Regarding vendor lock-in, I don't think Lambda is important for the vendor lock-in question. If you just need 'compute' capacity, it would be easy to move from Lambda to anything else. But if you also use bunch of other AWS services - then moving the Lambda functions is likely to be your least concern, frankly. On the other hand, if you do leverage various AWS services (other than just IaaS) - switching to Lambda (or other managed platform) could be useful to make your entire architecture (more) serverless.
There are some edge cases where it’s useful - education, low use throw away bots - other than that it’s all marketing bs.
Serverless is the future of the Industry.
Lambda is not Serverless , it's just a FaaS which is a component of an entire Serverless Architecture.
Most specifically the majority of services provided by AWS and in the Cloud already are Serverless :
- Pay per usage ( GB , Execution , API Call )
- No Scaling or Provisioning ( SQS , CloudFront , Firebase )
- High Avaibility by default ( SQS , CloudFront , Firebase )
- Auto Scaling by default ( Azure CosmosDB , Firebase )
- Little to no technical limits for the service ( GCloud DataStore , S3 etc... )
If you have very little knowledge about something you shouldn't criticize it , that make you sounds even more ignorant.
Most of the stuff that people actually use on AWS (S3 , CloudFront , SQS , SNS ) are delivered with a serverless architecture in mind.
Again , if you are thinking "Lambda === Serverless" you really need to read more about this topic.
Here I thought most of those were just "cloud" services or "third party APIs".
You have indeed.
As I mentioned above , S3 is a serverless service.
Last time I checked , you don't provisioned , partition or scale S3 this is done by AWS behind the scene.
Same for Firebase , same for many services.
Now the move is into the "compute" aspect of thing and the architecture.
Lambda basically replace, in a limited way , the compute aspect of your architecture (EC2) but obviously it has it's limits.
There is a confusion between "Serverless Framework" which is an automation of Lambda + API Gateway + CloudFormation , and "Serverless Computing" which is basically an "event driven" , "pay per execution" , "auto scaling / self healing" architecture.
To my mind, serverless is exclusively limited to FaaS or similar means of deploying code without visibility of the underlying server it runs on. S3 isn't about code deployment, it's a third-party API that stores data. EC2, docker et. al. at least have instances of servers you see, even if you automate away the management portion- that makes them cloudy, but not serverless.
Maybe my understanding of serverless is completely at odds with what the industry is, and if so I have learned something new and am chuffed.
Well I take issue with this, I think Kubeless and Riff are definitely serverless, even though I need a Kubernetes cluster provisioned (and it obviously has servers in it). I can get visibility into those servers, or I can run them in a managed way where there will be no visibility.
Both ways are serverless, I think this because they are FaaS solutions and because of the way the runtime environment is dynamically provisioned. I don't have to know or care what server it's on. I'm not sure that's what makes it "serverless" but there is definitely room for questions between "serverless" and "Serverless" with a capital S. And in at least one of those (Riff) it's actually cold/pods all go offline at rest, so not warm processes running on any server until it is called (scale to zero.) You just have an API gateway that is always online, and it starts the workers in response to events. It's hopefully lightweight compared to your app / suite of microservices / the kubeless version of what I'm describing, where each function has a memory footprint that doesn't evaporate when it's not in use.
Regardless of whether you're using FaaS or not, the pricing model for servers as scale grows seems to be quite favorable compared to same size "all-lambda" solutions, and based on the feedback in this thread and prior ones, I think there's a compelling benefit to price predictability if you take the time to estimate how big your computers need to be and, say, order enough reserved instances to cover something like the capacity you will need for the year.
You can always scale past that to grow when needed, but reserved instances are cheaper than ordinary instances, and ordinary instances are cheaper than an equivalent amount of CPU time on Lambda. I get how this kind of negates the benefits, but not really. It depends entirely on the pricing model and how much you use. If you can fit in the free tier, it's much less compelling.
But you can start with a really small cluster if you can scale your worker deployments' pods all the way to zero without causing a downtime event. (But if you're now groaning because I've got you managing a cluster of nodes again, then your priorities might be different than mine too...)
Right up until it hits the frontpage of HN...
Also, many don't know this, but you can run and debug AWS Lambda locally with AWS' own tooling. Using this to build out and test Lambda's before creating them with Terraform has been priceless: https://github.com/awslabs/aws-sam-cli
Is AWS Lambda really "serverless", or is the server part just buried so deep in the stack that the app devs can disregard it?
And if it is truly "serverless", what mechanism is being used to "serve" the function calls? How is this radically different from say a REST-api that auto-routes function names from url POSTs and GETs?
I suppose I could look all this stuff up, but to be honest having a quick rundown of this tech stack here on HN sure would save me the time.
Thanks in advance.
It's only serverless in the sense that you don't have to use servers as a unit of account; at no point do you have to say "how many servers are we paying for?" or "what sort of servers should we get/lease/instantiate on EC2".
In ways it's similar to Heroku but with a very different pricing model.
Lambda is just code execution. API Gateway is the service that maps REST API endpoints to Lambda function executions.
And yes, in some sense, they do abstract this away for you, but then you just have different kind of problems. So your efforts just shift towards other difficulties presented by such technologies. But I digress :)