Hacker News Reader

Comment by badatnames | Hacker News Reader

These one-sided arguments come up every time there's a Lambda thread. They're useful and easily cheaper almost anywhere you have sparse compute or RAM requirements, even for heavyweight tasks like transcoding. A t3.nano meanwhile can simultaneously be overpriced ($3.50 for an idle task) and a bottleneck (30Mbit/s sustained network) in just as many places as a Lambda could be applied incorrectly.

View on HN

They're also good for extremely bursty loads. I use Lambdas extensively for a radar imagery API that delivers the images for an Android live wallpaper that shows weather radar.

That particular use case can go from hardly any traffic at all to a hurricane of traffic during something like, well, a hurricane.

For consistent processing loads, I think I could save money by just running on dedicated EC2 instances, but you cannot beat the instant scale of Lambdas.

Shameless plug: https://play.google.com/store/apps/details?id=app.radarlove....

There's nothing particularly special about Lambda's scaling. You can do the same with GCP Cloud Run[0] or Azure Container Apps[1]

Specify the maximum number of allowed instances and maximum concurrency per instance. Optionally configure more targeted scale rules. It will scale it from 0 - n as needed.

[0] https://docs.cloud.google.com/run/docs/about-instance-autosc...

[1] https://learn.microsoft.com/en-us/azure/container-apps/scale...

So long as you don't count >2 orders of magnitude difference in scaleup time as particularly special..

That's not how it works since each instance can handle multiple requests (up to 1000 per instance). So you may set this to scale at say 75% load level.

Indeed if you configured each instance as a single threaded handler, you will not have fun.

That's only how it works when misapplying mean() to characterise application performance. In your scenario, every request is hitting either an overloaded instance, or when scaled down, a 20-30 second wait for spin up, better hope they're neither interactive nor some jobs configured with a short timeout. The mechanisms are so completely different they're hard to even make meaningful comparisons between

    > In your scenario, every request is hitting either an overloaded instance,

Why would this be the case? Cloud Run containers and an app runtime like .NET, Java, Go, Rust can inherently be multi-threaded, multi-connection. It can handle up to 1000 concurrent connections per container using just threads and async (assuming sufficient size). Any workload that requires hitting the DB or backend service (S3, etc) is going to I/O bound and sitting in a future most of the time anyways.

The container instances auto-scale by default to a configured n (0 - n) with the option to define specific scale points. If I'm sensitive to high bursts, this can be a matter of scaling at 50% capacity and a warm instance is available while the application still has headway.

Cases where this falls apart: DDoS in which case, I argue that 1) it is better in any case that it falls apart rather than absorbing that type of traffic spike, 2) such traffic is better filtered upstream in any case in an API Gateway + WAF.

I have worked with both types of solutions for different use cases (was invited presenter at Azure Serverless Conf[0]), but my take nowadays is that serverless containers are almost always a better approach since it provides much more flexibility.

[0] https://learn.microsoft.com/en-us/shows/azure-serverless-con...

> this can be a matter of scaling at 50% capacity

At which point we're paying double just to work around cold start latency, which was the original point.

Cloud Run doesn't bill when idle; it only bills when active.

So if you're at 50% capacity with lots of idle time, it can scale to 0.

Scale to 0 doesn't mean "cold". One easy way to keep both Cloud Run and Container Apps "warm" but functionally scaled to 0 for costs is to just ping it every few minutes with Cloud Scheduler set up on a cron pointing to a health endpoint.

If you scale Cloud Run to zero and then send a request, won't you immediately wake it up and be charged until it powers down in at least 15 minutes?

When scale to 0, it only bills when actively serving a request. BUT, the instance doesn't go cold immediately; you are only billed while it is serving even if the instance is active.

This diagram helps visualize: https://www.gstatic.com/bricks/image/5eb3c8e9-6159-4e2f-ba98... (gray bar at the top shows when the instance is started and when it ends; blue bars show actual request, and green bars show the billing)

If there's a 10s gap between requests, you are not billed for the 10s gap, but it is still effectively hot; instance does not stop. Request that comes back in 10s later does not have a startup penalty because the instance is still up.

How do you leverage this to build a high throughput system with no cold starts? Just prime it with a short single request (e.g. `/health`) before instance is stopped.

I assumed you were talking about instance-based billing (what I've always used), but I think you're talking about request-based billing?

Request based billing indeed.

I often hear this argument that it is good for bursty load, but for real, there are a lot of inconvenient limitations. For exemple, you don't really control the number of concurrent instances or the scaling except if you pay for very expensive provisioning.

So you might have to wait for 10s or more to have a new lambda instance spawning up and replying to your requests. And you might have to have your function finish in 15 minutes, deals with the strange concurrency and scaling rules lambda is based on.

Reply on news.ycombinator.com